I'd say that his lack of "manly" qualities stack so much in the negative values that he just can't be considered a man anymore. Saying that you'd fuck Shinji doesn't mean you are gay, since he isn't a man. A neGar character isn't considered a woman, just.... not a man
. More like a really, really, reeeeeaaally crappy living thing. Just because you are not considered gay because of wanting to fuck Shinji up doesn't mean that you are not free of any other lulzy labels!
Think of it like someone into dendrophilia, or zoophilia, or maybe even necrophilia but... more into wanting to screw really
crappy character, like Shinji!
"I settled for that shriek. Those dull vacant eyes... oiled cabbage stench of her...! [...] You know what? I'm sorry if I'm not gonna do this the way you want me to or the way you might. But I will not make an angel out of someone who wasn't an angel...!" -Chief Tyrol, on his beloved wife.
Have you heard about the saying that goes something like "drunk people and kids tell no lies"? Well, that's just a fallacy.