even though i traded my matrix for a corolla, i recently test drove a caliber 1.8l 5 spd. my matrix was a 5 speed base model.
utility--id say caliber and matrix have exactly the same utility.
looks--ok, but the lines to me are very sloppy, the body just has no smooth flow from front to back. from the front, it does look like a dodge pickup truck.

the center console stack(where radio and heater/ac controls are) looks like a separate piece was glued on to the dash; uneven gaps between stack and dash. matrix interior much nicer in my opinion.
ride--about as smooth as matrix, and really no quieter. feels slower than the matrix cause its pushin somewhat more weight.
gas mileage--no contest here. from visiting caliber forums, owners are struggling to get 30 mpg on highway. Chrysler has never been involved in fuel efficient engines, as is the case here. plus, caliber is more bloated weight wise than necessary for a car in its class. my matrix always got 36-38 mpg on the highway and 32+ in the city.
overall, there is not one single thing that would cause me to choose a caliber over a matrix.
beyond every factor mentioned, there is still the issue of reliability. anyone can do the research there: looking at frequency of repairs over the long haul, chryler products have faired poorly historically. will caliber be better. who knows? go ahead and buy one and let us know