the difference between being single and incels, is that the later have given up, consider this as a permanent situation probably because they are aspiring to far more than what they can get in terms of partner or because they don't have the desire or don't consider it possible to improve themselves both physically and in terms of social skills.
I think the first part of your statement is correct, but not the second part, which I bolded. If you listen to the Elliot Rodger Retribution video, which may be considered a template for incel mindsets (including a few I have met in my life), they don't believe self improvement is necessary or the issue at play in their involuntary celibacy. They do not see themselves as losers or socially awkward. They blame women for the rejections they are receiving. They see women as having made irrational choices in terms of their sexual partners, and in not choosing them. Women therefore are deserving of violence and death, as are the sexual partners they choose.
Rodgers probably creeped out every woman he ever met, and watching and listening to that video, we can all see why. Looks was not his issue. It was his attitude. He could not see it. Women do not want to be creeped out - they want men who act normal and who they are comfortable around. Would you be comfortable having Elliot Rodger in your house after hearing that kind of nonsense? Most women have radar for creepy behaviors and as soon as they sense them, it is all over.
It's a sort of irrational jealousy that drives the incel. Most of these guys cannot see in the mirror what is looking back at them. Some people who met Elliot Rodger, family friends, claim they knew something was wrong with him at age 8 or 9. The kid was in psychiatric treatment from age 6 til his death at 22, was never actually diagnosed with anything, and refused all proffered medications. His psychiatrist was sent a copy of the video and his manifesto before anything happened and told Rodgers' parents, who then contacted police - too late and after the fact.