I think mine has been like that for a couple of years. On a friend's '94 Corolla it has never been cleaned and the only problem that developed like a year ago, was that the throttle plate was kind of sticking in the closed position. Nothing major, just that it was a slight resistance when depressing the throttle pedal in the morning. And also the idle air control valve got somewhat sticky.
Of course a clean manifold is better than a dirty one, but it is not a major problem. The oily residue comes from the crankcase vapours that are directed into the intake through the Positive Crankcase Ventilation (PCV) valve.
Haynes manual recommends to clean the area around the throttle plate and be done with it.