For a variety of reasons, I never delete as I go, but frankly, whether or not I would do that, I find it's easy to use up both cards in the D4 in a morning of shooting even if one was 32GB and the other 64GB. I've done it many times.
As to the original question, what size cards to use, I think it's a matter of philosophy as much as anything else.
Right now, I'm using 32GB cards. The 16GB cards are just too small for me and would require too many cards during a full day of travel or wildlife photography. At 32GBs I think I've struck the right balance of number of photos per card vs. if the card goes south, the number of photos I could potentially loose. At 16GB there's not enough capacity as mentioned above, and at 64GBs the loss potential is too high for me.
On the other hand, this isn't written in stone, as I haven't yet taken into account video needs. I need to experiment more before deciding on video memory capacity. That might push my requirements to have a 32GB CF card for still images and a 64GB XQD card for video. Then again, I'm extremely pleased with the speed of the XQD cards for write speed when shooting at 10fps. It's noticeably better than the Sandisk Extreme Pro CF cards.
In fact, I'm almost to the point I wish the D4 had 2 XQD slots instead of one CF and one XQD. Sooner or later, I will be at that point. I can definitely see new upper level cameras using only XQD memory and discontinuing all CF use in the near future, if there are at least 3 quality XQD card manufacturers. With Sony and soon Lexar manufacturing the XQD cards, I've got to believe it will only be a short time before Sandisk finally jumps on the band wagon.