Well, they do collectively separate themselves from society. They have their own art, their own "furspeak" (not as relevant as it is mostly onomatopoeia), and generally their own culture.
I wouldn't call furries a fandom or a race, but more like a miniature cultural movement. An NRM, per-say.
Anyhow, it doesn't matter. Discrimination against any group in particular is a pretty bad vice to carry around.