I have both the 24-70mm and 16-35mm. Luckily, I have very steady hands and although very useful I can get away without VR in many occasions and by upping the ISO. I tend to keep my elbows into my chest and support the lens with my left hand using my right to click. I am similar to you, but I always disengage VR when the focal length exceeds x1.5 of a chosen speed e.g. 300mm = 1/450.
Below are three images taken 2 days ago at a museum and all handheld. The first two are taken with a 16-35mm with VR on, the third with a 24-70mm and of course has no VR anyway. In most cases I will use a tripod, but in this case I had left it in the car.
Hope that helps
Image 1. D800 + 16-35mm 1/15@f4 ISO800 EV -0.33 Handheld