Get it inspected. I had the steering rack replaced in my '05 as did a co-worker also in a '05. Not saying this is your problem, but these types of things are hard to diagnose on an internet forum.
Tell me about it! When they replaced my steering rack they also replaced my two lower ball joints which included the full lower control arms, control arm bushings, and alignment cams. All repairs totaled $2800 including labor and a wheel alignment. Thankfully Toyota extended warranty covered it!I really hope its not the rack ... my '96 needed one replaced - a rebuilt unit installed was 2000+$$. I wouldn't want to guess the cost on an 05+