Thanks for the posts guys. Based on what you folks stated, I decided to swap the engine.
Here's a summary of the process I followed:
In considering my options, I carefully looked the rest of the engine over. I removed the valve cover and examined the cams. I noticed that the cams were actually worn - most likely this was due to a low oil condition.
The reason for the low oil condition was increasing oil leaks due to loose oil pan bolts. If you have a 4AFE please check your oil pan bolts and make sure that they are tight!! Also, make sure your gasket is good. If there is no gasket, then use a silicone high-temp RTV.
Obviously, overhauling the existing engine would mean replacement of a large number of parts, including the camshafts. Therefore, I explored the option of buying a used engine.
I found a used 4AFE from a 1992 Corolla with a claimed 104,000 miles on it. After purchasing the engine, I performed a basic teardown, removing the valve cover and the oil pan. Visual inspection showed the cams looked like new and no sludge in the oil. Rod bearings on all 4 cylinders were quiet.
A leakdown test was performed @ 100psi input pressure through a 0.060" orifice. Initial results showed less than 8% loss across cylinders 1-3. Cylinder 4 initially showed 35% loss through the exhaust valve. I used a screwdriver and tapped the valve shim a few times with a screwdriver and a rubber mallet to dislodge any carbon. Loss dropped to less than 10% after a few taps. All was well.
Total time for the swap took about 40-44 hours, since I had to do it alone and this was the first time I had ever done an engine swap. The paper reference I used was the HAYNES manual, which had torque specs for the critical bolts. However, I must say that it is a pathetically incomplete resource. It had enough to get the job done, but if you have the ability to get a more complete reference, by all means, please do.
I also replaced the following in the process:
* Intake Manifold Gasket
* Exhaust Manifold Gasket
* Distributor O-ring
* Camshaft Front Seal
* Rear Main Seal
* Rear Main Gasket
* Clutch / Clutch Throwout Bearing
* Front Main Seal
* Timing Belt
* Timing Belt Tensioner
* Water pump
* Transmision Fluid (Mobil 1 MTF)
Here are some things I learned from the swap:
1. I needed to use two impact wrenches. One that is small and makes 250-300 ft-lbs of torque but is light and can get into small spaces. The other is a heavy 1400 ft-lbs monster with a 3/4" drive. I had to use this to remove the crankshaft bolt, the 3 lower a-arm bolts from each of the front wheels (which I had to remove in order to remove the driveshafts). Just a few seconds with the monster works where 30 seconds of constant hammering with the smaller torque wrench @ 120 psi fails.
2. The front crossmember is a b**** to remove. The reason it is difficult to remove is that Toyota designed the rear engine mount bolts to fix the crossmember in place. The problem is that you cannot easily lift the engine high enough to have the rear engine mount
subframe bolts clear the crossmember. It took me literally 4 hours to figure this step out. The correct way to do this is to remove the driveshafts first, which then allows you to remove the rear engine mount
cross-bolt, then lift up the engine and remove the engine mount.
I also used my plasma cutter to modify the crossmember so that it no longer is clamped down by the rear engine mount. No structural problems doing this. The benefit is that I can remove the front crossmember without having to remove the rear engine mount, which means that I can drop the oil pan in the future without needing an engine hoist!
3. Removing the intake is a pain in the butt. The secret to this step is to remove the driveshafts first, then the starter motor. Then disconnect the intake manifold reinforcement bar and the vacuum manifold, then any vacuum lines. The vacuum lines will prevent you from moving the intake manifold away from the firewall.
4. Timing is easy to preserve, just make sure that you always leave the engine at TDC and mark the TDC points for three critical places: Distributor, Camshaft Gear, Crankshaft Sprocket. If you find it difficult to remove the Camshaft Gear bolt, you can use a small impact hammer. Because of the way that the impact hammer works, it will not rotate the camshafts when you loosen the bolt.
5. Putting it all back together takes about 3x less time than taking it apart.
6. Easy way to remove the engine from the transmission with one person:
Remove the rear and front engine mounts. Remove the left engine mount bolt. Keep the right transmission mount attached to the car.Lift the engine out of the engine bay with a chain attached to the lift-hook located next to cylinder #1 (left side of the engine). Lift it straight up until the entire assembly is cocked at about 30-35 degrees. Place a jackstand, hydraulic jack, or other stable object underneath the transmission. Undo the four transmission bolts and gently lift out the engine. The engine should remain at about 30-35 degrees and slide easily out of the transmission. Keep the transmission where it is. When it comes time to replace the engine, just lift the engine to the same point and re-connect the clutch to the input shaft, gently sliding together the engine and the transmission.
7. I know I mentioned it a few times, but
remove the driveshafts first. It will make your life a lot easier. You will need a 30mm deep socket driver to remove the driveshaft nuts. Then remove the two nuts and one bolt holding the bottom a-arm to the knuckle (do not mess with the balljoint.)
The easiest way to remove the driveshaft from the knuckle is with an air-chisel using a single-point tip. Just press into the little hole in the tip of the driveshaft.
Once you are finished with the swap, fill the engine with oil, disconnect the plug wires from the plugs, and crank the engine continously until the low-oil light turns off. Then a few more times to be sure. This took me about 60 seconds of cranking the engine. Replace the plug wires, start the engine, and check idle. Everything worked the first time for me.
I plan on changing the oil and filter within the next 100 miles just to make sure that no dirt/gunk ended up in the bottom end during the swap
Here are some pictures from the engine swap:

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04

By
sonoronos at 2009-10-04