2x4 in the dirt is ok if you have two things: momentum (speed) and flattish surfaces. Most of us cannot control these two factors unless we live on a hog farm. You have two choices if you see a common a washout on a dirt road. Either balls through, or just turn around and feel kinda weenie ("That rut wasn't near as huge last time I saw it! Damn rain storms!) .
4x4 Inspires confidence!
Descending hills is easier in a 4x4, espescially if you have a drop-off ledge on one side.
Taking a trail till it ends is easier because the 4x4 feature lets you get turned around in tight dead ends. 4x4, especially Toyota, can fit where the larger Detroit trucks cannot. As 80toy says, the low gear range is awesome, and can be the difference between walking/waiting for help. I have seen trail drops closed by the forestry service because these trails were snagging too many people with 2x4. My 4x4 would take these trail no problem, even though I have only 96hp.
Another benefit: 4x4 allows trucks to "tread lightly". The same trail obstacle that forces a 2x4 truck to balls it through and tear up the earth, can be accomplished with zero wheelspin in LOW range at walking speed in a 4x4.