I've always remove the steering rod end from the hub to do the axles in my Camry. Just don't move the adjusting nut and everything will be fine after re-installation. You will need a 2-jaw puller to pop it out of the hub. Good luck!
-Charlie
-Charlie