Don't let anyone bring you down, if people are saying things just brush it off, you're here to have fun and no one should be standing in your way of that.
I find when I face a playerbase I do not like, I switch to another. Perhaps if the Humans aren't to your liking you should make another character with another race?
As Lefty said you're here for fun. People will always say things about each other and gossip. At the end of the day it doesn't really matter. Since they're just random people on the internet. Albeit a tad more arrogant with their identities masked.