I did a quick and dirty comparison between a 35mm 1.8 DX and a 50mm 1.4D. My conclusion? You're right. Once FoV is equalized the ONLY way I can tell which lens was which was by the DoF: obviously the 50mm @ 1.8 will be shallower than the 35mm.
To my eyes one image doesn't look any more "distorted" than the other. I imagine that if I staged a even more rigorous test and equalized the DoF, used a tripod, light meter, etc., I could produce images that you'd have to look at the exif to determine the lens used.
I've got some thinking to go do. Thanks everyone for the guidance, and correcting some obviously "distorted" information about lens distortion!