It is usually due to bad previous experiences: more likely to be aggressive during sex, have issue with payments (try to negotiate), lack manners, lack hygiene etc. Every sp probably has a story of a black guy trying to aggressively negotiate for lower payment, while smelling awful. Same is true...