For some reason, this trio gets overshadowed so easily. The RSX stole the glory from the Celica at a time when WRX and EVO were getting popular, and the XRS's became less popular than the first Scions. And when the tC came, and everyone forgot about those cars. But yet, people still remember "MR2, Supra, rwd Celica, etc.".
That is a separate issue. Still, the fact remains, those were the most sporting cars that were blessed with one of the very best 4 cylinder engine Toyota ever built (Lotus Elise was using the same engine and transmission in the same period).
As far as popularity goes, 95% of the cars were sold in automatics and you cannot sell a car in big volumes with only a 6 speed manual close-ratio gearbox and an edgy, high-revving screamer engine that demands a lot from the driver, but is very rewarding in return when you put in the proper effort.
RSX also had the same issue where base RSX accounted for over 90% of the sales particularly in automatics. RSX was discontinued just a year after the Celica was discontinued because they both suffered the same problem.
That is where both Honda and Toyota shifted their strategy to sell in bigger numbers, which is where it brought in Scion to appeal to the mass market. The 2nd generation XRS with Camry engine, 5 speed auto/manual and a softened chassis, suspension was also a move in that same direction.