Yep, the ventilated seats just pull ambient air from under the seat, NEAR the HVAC outlet. HVAC air is not "piped in."
To add ventilated seats, you would need to buy the seats, then add the appropriate wiring and the switches. The cost of all that stuff might be equivalent to the price difference between the XLE and the Limited, plus you would likely get nothing for your "upgrade" at trade-in time.