Welp, I spun off those dust caps a few or several hours after my previous post, but again was greeted with the sound of gas whooshing out as the cap loosened.
So I cranked those plastic caps back down as tight as can be without breaking them, and I hope that my AC will last the summer - although I'm having my doubts.
My car is a 2000. I got one valve core from a 2001 or so Celica, and the other two I got from later Avalons. Comparing the three, one of them had a weaker spring like the one in my Corolla, but the other two seemed a little stronger, so I put them in - I'll assume that they are the later valve cores.
The Schrader valves aren't supposed to leak like that to my knowledge. I did some reading and saw that some HVAC pros swear by the cap, saying that the Schraders will always leak, and the cap can stop, or slow, that leakage. Those guys seem to swear by brass caps with O-rings, which sounds right to me. But I'm not aware of anything like that for R134A fittings.
I converted my previous Rolla, a '93, to R134A and it worked for the five years that I owned it, with probably close to zero leakage.