Study this topic (replacing differential):
[gen3] Grinding noise from A/T (car won't go). !
Unfortunatelly, differential (driver's side gear) is very weak point of Camry. If there was not enough oil, it's probably damaged somehow.
Worn CV joints can make clicking sound when you turn and that's all (sometimes on straight line direction noise can be also present and it's similar to worn bearings, easy to mistake).
Worn differential gears will also start clicking during turning, but there is always force on steering wheel, that resists when you try to turn.
First check again your wheels bearings, brake system, CV joints on axles (take off boots and inspect it). Then go to differential.