I initially returned a D800 based on the 'concerns' you have detailed - ended up with a D700, which I love - and then added a D800e which has quickly become my most used camera.
I will say this about the D800e, even with downsizing I find it more sensitive to camera shake than my D700. That being said, the D800 has done what seemed to be impossible, it has tripled the MPs while actually improving ISO performance over the D700. Although the ISO improvement is not vast, the fact it is coupled with the MP increase makes it very impressive.
The dynamic range at low ISO is incredible, and is clearly better than the previous generation of Nikons (D700/D3/D3s).
The amount of detail it can resolve still amazes me.
Ultimately, my original concern about getting sharp photos while handholding appears to have been unwarranted. While I still believe the D700 to be less demanding, I am content to pay more attention to my technique and use higher shutter speeds on the D800, which provides enough advantages over the D700 to have relegated it to the backup or 2nd body.