Comparing apples and oranges between those two models. Most RAV4 V6 owners know what they've got, an almost muscle car in the shape of a little SUV. An "iron fist in a velvet glove." You would have to be a driving enthusiast to appreciate it. The Highlander is larger, still plenty of V6 pep, with a much smoother and quieter boat ride. It is a minivan and a good one, more practical if your "load" involves more than four people with gear.
You might take a look at a late model Kia Sorento for grins. Consumer Reports is very favorable on them--including reliability-- and it is more in the same class as a Highlander for a few thousand $$ less. They don't hold their resale value as much as Toyota, but that might work in your favor.