I've been thur this, its most likely the master cylinder, which runs $150 OEM new. It also could be air in your brake lines or the brake booster, although less likely. My did then same thing, and then one day it almost had zero brake pressure. In my case it was the master cylinder. But the dumbasses at tire kingdom kept replacing it with a fake aftermarket one. This caused the brakelines not to bleed and also no brake pressure. Because of this, they had it for 3 weeks. Finally they towed it to toyota at their expense. Toyota put in a new OEM master cylinder that same day and it was ready in 1 hour. Toyota said that the camry brakes won't work with fake master cylinders. Dunno if it is true but it worked.
Read my horror story here:
http://www.toyotanation.com/forum/t113645.html