saw this forum last night trying to figure out what to do about the same problem in my 2007 Camry. The driver side sun visor was drooping down and wouldn't stay up. Here is what I tried.
I took down the sun visor first (instructions here: http://www.toyotapart.com/SUNVISOR_-...SB-0093-09.pdf
I pulled out a couple of the stitches on the fabric covering the sun visor along the top and side where the arm enters. I then pulled out the arm/bracket from the sun visor - the sun visor plastic was cracked and deformed. The bracket arm has a metal tab that rotates around and keeps friction on the arm, so that the visor stays in place when dropped down. But it looks like all the torque eventually causes the plastic clips on the sun visor to pop apart, ruining the visor and causing it to droop. If you twist the tab, you can find the locked position where the visor is flush against the roof.
I tried to lubricate the rotating metal tab with some silicone grease, but it remained really tight. So I resorted to brute force and got out the hammer and pliers and pounded the tab open a little bit, to decrease the friction. I squeezed it back into the sun visor (the tab should point down, into the body of the visor).
Then I used a large binding clip for paper (from the office supply store) to clip across the visor where the arm is - this helps to counteract the torque from the tab and keep it from popping back out of the visor. Then following the installation instructions, I reinstalled the visor into the roof bracket.
It's not pretty, the vanity light doesn't work (probably because the contacts inside have been pushed apart), and I don't know how long this solution will last. But for those who need a temporary solution, this might help. Of course, your mileage may vary, and I don't assume any responsibility for damage, injury, etc.!