Yeah, if you have a factory external amp, you'll need to bypass it becuase you'll be using the internal amp from your new aftermarket head unit.
Near where the wiring harness is that plugs into your factory unit (I assume it's a single plug), there should be two other harnesses tucked away in the dash. Those two harnesses are for hooking up a head unit with an internal amp.
See the "Remove Factory Radio" section of this PDF. THis is for the Gen3, but the wiring for the Gen4 may be similar.
http://installdr.com/InstallDocs/Toyota/PDF/869013.pdf