Not impossible - but you would need to do a fair amount of fabrication. Transaxle will probably have to go as well - not only due to compatibility with the engine, but also because the 4-cylinder transaxles most likely cannot handle the extra power.
To keep the FF layout (front engine, front wheel drive) - you're best source for potential donors would be from V6 powered Camry/Solara - in terms of finding potential donors. This would also include tThere are few that have stuffed V6 engines into MR2 and Celicas - I don't think there were any on a Matrix - atleast off the top of my head.
Also keep in mind, the amount of fabrication, wiring, etc. - would most likely be more expensive that what the Matrix is currently worth - but if you want to do something different. Go for it. Here is a link to the V6 powered MR2 page - has some specs on the different Toyota V6 engines that some have used for their swaps. Lists engine code, transaxle codes, sources for said engine - a great resource.
http://www.v6mr2.com/specs.html