The following link refers to a Gen 4 problem, but it might be relevant to yours.:
Hard Start after Hot Soak
...might be something to try.
...Another part to look at ... the IAC solenoid.... could be stuck...not moving freely due to carbon or dirt. You would probably have to disassemble it...connect it to a battery and see if it moved...Instructions in the repair manual on how to test it....Just hook up to 12 volts.
How many miles on your vehicle? Are sparkplugs and wires ok?
__________________
98 Camry LE, 2.2L, automatic
50k miles, drop in K&N A/F

recent timing belt, water pump