First,
A long block comes assembled with the cam, head, block, pistons, crank, rods, timing set, and usually oilpan and valve cover.
A short block is only the bottom end, as in the block, crank, rods and pistons.
Both would come with an oil pump and maybe a water pump most likely unless it is a really cheap outfit or something.
As far as your rebuild, there are some companies out there with good reps, like engnbldr.com
I don't have any experience with purchasing one as I like to rebuild them myself.
For a solid stocker,
Make sure the block has been THOROUGHLY cleaned before and after machining. It should be bored and honed to fit each piston, not just all honed to one dimension if possible. Crank should be either reconditioned or new, with properly sized bearings. Rods should be reconditioned, and it's a good idea to use ARP rod bolts for extra insurance. The deck should be checked for flatness and resurfaced if necessary. Cast pistons are ok. A very good idea to have the whole assembly balanced. This lends to a smooth running engine and longer life.
The head should be totally rebuilt including pressure checking it, careful inspection for cracks, etc. Replacement heads are available if needed, also from engnbldr and others. You can re-use the intake valves if you want, but they are cheap so it's a good idea to buy new ones. Exhaust valves should be replaced, IMHO. Springs should also be replaced with new units. If the cam is in good shape, you can re-use that, but it's a good time to upgrade if you want a little more oomph. Make sure the springs match the cam. Get a good three angle valve job.
New oil pump, new water pump, new timing chain and gears, and new timing chain guides.
While you got it apart, it's a good time to replace the vacuum hoses and water hoses. Maybe clean out(or replace) the EGR valve if you have one, replace the fuel filter, replace the O2 sensor(should be done every 60K miles or so) possibly have the injectors cleaned and balanced, the list goes on. Depends what you want to spend.
Take some time to research reputable shops. It could easily make the difference between a 300 mile engine and a 300,000 mile engine.
That's what I would do, I'm sure there are other opinions.
(actually, I would do more performance oriented stuff as well, but thats another story

)
Good luck