I think it is a fair concern, and I am sure it does drive some people away, and I think it is good to bring it up. Thanks.
Although in the case you describe it is about people not being able to be allies because of who they are, I don't see it as that much different as other kinds of divisiveness on the left. I think there a problem with judging based on dogma, and in feeling one has to squeeze one's self into a dogma.
My point is that it really eclipses dealing with the problem and changing things for the better. It just turns it into team sports. On the other hand, not everyone is going to like you or consider you an ally anyway, so why not just be honest about what you support and what you don't, and just leave the judgment and exclusion to them?