Not sure you know how LEDs work. LEDs are not bulbs even though we call them bulbs. LEDs emit straight ahead only, they do not radiate like a filament bulb where the light goes in every direction. LEDs are directional, if you point em in the right direction, you won't have an issue, glare even. I've got morimoto LEDs on my car and haven't had an issue with glare or the light being a bother at all. Another factor is headlight aiming, yes, aiming. A lot of people don't have their headlights aimed properly and even running regular incandescent bulbs aimed improperly will blind people in front of you. Like you said, only reason LEDs get a lot of crap is because they're 2x, 3x, 4x times brighter so it's more noticable/annoying. With LEDs, direction and aiming is VERY VERY important.
Only people my LEDs bother are ricers with cars slammed to the ground.