2.5? not sure what you were measuring, but that doesn't sound like any light on the camry except maybe the license plate light.
I just measured my 98's outer tail lens section. at the front edge (side of car) it's just shy of 4" tall with the tape measure vertical (not following angle of tail light). at the trunk side it's again almost exactly 4" tall, and again not following the angle. it seems like if you did follow the angle, it might be the 4.5" you mention.
just so there's no confusion, here are the pics:
worst case scenario, you can shave down the bumper cover, since it's the bottom edge of the light that gets wider (taller).
that said, i think gen4.5 outers would look weird next to gen4 inners, even if the lines match up, because the inners are so "foggy" compared to the crystal clear outers of a 4.5. plus the inners on a 4.5 double as brake lights, whereas the inners on a 4.0 are only tail lights. This last bit is minor, but still worth mentioning. Now if you were to paint the inner lenses body color so that only the reverse lights were visible, then maybe it wouldn't be such a big deal.
I myself prefer the look of the 4.5 rear (and front too) better than my 98, but it's not worth the work to me to convert the rear. I might still eventually invest in some gen4.5 headlights though - that is pretty minor work to me, and doesn't have to be perfect since all the cuts you make are not visible after the headlights are installed.