First start with the easy stuff...
- Check all your fuses, make sure none of them are blown.
- Check behind the lights and see if there are any loose or frayed wires.
If everything seems kosher... it could be an electrical problem... which is supposed to be a pretty bad thing on gen2 mr2's from what I have heard.
When you take it to get the oil changed... take it to your local Toyota dealer, tell them you are having some problems with your reverse lights that you would like that have them check over while they are changing your oil. <--- That's how you get a cheap estimate

.
If you have a good mechanic you use... he should check it out for free... but I don't know your situation.