Damn. Sorry to hear about the early demise of your Camry! I AM glad to hear that you guys walked away from the accident though. Funny you mention owning a Jetta. My neighbors had the last gen Jetta (looked to be about an '03) and I guess they repelaced it with a used Fit. My guess is after the years piled on as did the miles, it became too expensive to keep up.
I test drove the Camry SE (both I4 and V6) and it didn't seem noticeably sportier; it just LOOKED more sporty. I think you'll have to mod it a little to tighten the suspension and get stickier tires. In the end though, it will never drive like a German sports car, but it also will not cost you an arm and a leg to maintain. Life is about trade offs I guess.. Personally, I would go with the Camry and do simple mods to tighten the handling.
As far as the Accord, I can only compare my Camry to my aunt's Accord of the same generation. I most definitely prefer the steering calibration of her Accord but the ride is ROUGH and road noise is higher. I manage to drive just fine in my Camry in all its stock LE glory