More than likely either the snap clip that holds the outer handle rod broke, or the snap clip is intact, but the rod came out of the hole. I am going through the same problem as I thought I was the only one here with door issues, but from what I see I am just as much a victim. I took off my panel. Do a search there are some good threads telling you how to remove the panel. Then I found that the snap clip that is supposed to be attached to the outer handle rod was severly broken, and not connected. The 2 snap clips cost me around $8.00 from my local Yota Dealership. The passanger door went in fine, but my drivers door doesn't open from either side

I have a slim jim, and need sometime mucking around with that to get the drivers door to open, and then I can replace the clip. If you have anymore questions feel free to ask, but definety do a search for door panel removal. Good luck.