That's strange, my 05 LE came with three sets of keys. Two black with the key fobs, and a third that was gray. The gray one was the valet key. The two black ones with the key fob would start the car and open/lock the trunk. The third, the "valet key", would only start the car, but NOT open the trunk if you used one of the other two keys and locked the trunk by turning the trunk key lock to the left rather than right. Turning the trunk lock key to the left would also prevent the inside trunk release latch in the car's cabin from opening the trunk.
I don't believe '03-'04 had immobilizers. My '03 doesn't, anyway. When I bought my '03 last year, all the dealer had was a plain metal Toyota key with a keyless entry fob and the black rubber Toyota key with a keyless entry fob. I didn't get a valet key, and according to my manual, my black key SHOULD be able to lock the trunk as bennor mentioned, but I have yet to make it work.
To answer the OP, if you have a black head key, it should technically do everything.