"I have taken some DX mode shots of surfers with the 70-300mm VR trying to "fill the frame", whereas in FX mode I could not do so, obviously the shots were from the same distance. It SEEMS that the DX shots were more clear, and at any rate, did not suffer. Same apparent result with some racing sailboats. These shots were at distances I could not "reach" well with the FX mode and 300mm lens.
This suggests that you were racked out to 300mm in FX and may have been able to use a slightly shorter focal length in DX. On the D600, my 70-300 is significantly sharper at 250mm than 300.
OK- went to look at properties of files, and YOUR GUESS WAS CORRECT: the FX images are (of course) racked out at 300mm; the DX images are at 250-280mm. I would not have thought this would matter, but it does.
So, I've learned 2 things: 1) images shot in FX, then cropped, are ordinarily going to be of same quality as a DX shot image of the same apparent size in the frame; 2) my 70-300 3.5/4.5 VR performs better if I avoid racking it out to 300mm - even 280mm may "look better".
I already knew to try and fill the frame with subjects like surfers, boats, and birds, surrounded otherwise by less interesting back and fore ground.
ONE QUESTION LEFT: could it be that in DX mode the autofocus points, which are then "larger" have an easier time focusing than in FX when the FX shot is at such long range that serious cropping is necessary? I ask because the surfers seem more clearly in focus in the DX mode shots. ISO, aperture and shutter speed are the same.