I am not accusing you of this because I'm sure you don't have bad intentions but when a client told me I "turned him off" personally it's because I set a boundary, I was firm, or felt hesitant to see him...
I'll say it again we aren't public property and we're allowed to decide that something...