Responding to what I see in the test: I can't tell much of a difference. That said, I believe that the only reasons you need f2.8 is to reduce motion blur, reduce DOF, and get a brighter viewfinder. There are workarounds for the last two (accept a slightly darker viewfinder, carry a small prime for DOF issues/shots), but there is no workaround for shutter speed motion blur.
But, if YOU don't need f2.8 for motion, the f4 lenses are a no-brainer. Image quality is essentially the same, weight, size, and price is lower. I personally, don't need anything bigger than f4 for anthing I shoot but (indoor) sports. In fact, except for the occassional portrait, I stop everything else down to f5.6 or f8 anyway.