I just took my car to the shop for this problem today and it's fixed for the most part. My car was pulling to the left when I drove and when I hit the brake. The steering wheel would also shake when I hit about 60 mph or brake from a high speed.
The shop rotated my tires, balanced them, did an alignment, resurfaced my rotors and adjust my rear brakes for $160 bucks. I don't know if that was a good deal but all of my problems are corrected except for the drift to the left which is much improved but still drifts a bit to the left. The mechanic said that camry caster (2000 camry, I4) isn't servicable so I would either have to live with it or maybe get non-directional tires so they could be rotated better (criss-crossed I assume?).