The balance shafts were to reduce the engine vibrations. However, form what I've gathered, most mid 90's Celicas(which also had the 5sfe) never even left the factory with balance shafts. I also, read that Japan built Camrys did not have balance shafts either, but the U.S. built Camrys did.
Anyway, Chris Crash is right. If that thing is starting to go bad, toss it. I haven't read of any ill effects of removing it. Do a search on balance shaft removal, and you find a DIY on the whole process.
EDIT: it's driven by the crank shaft, not the belt. Here's an image taken right out of the repair manual...
You can see the balance shafts are driven by a gear on the crankshaft.
.
.
.
.