I'll try to give you a response from two different perspectives. First, through a telephoto lens, the slightest of movements is accentuated at a distance. Regardless of technique...there is more movement transfered between the photographer and camaera than one might think...there are a number of variables....breathing, pulse...the subject...wind and others. All play a part. Even at 1/500 of a second...enough of the variables togther may compound into enough movement to blur an image. I see this no differently as what a skilled marksman must do to hit a distant target.
The second perspective is completely different....one of taking the time to compose the image you want. For me personally, I tend to "settle" for an image...roughly composed in the haste of capturing the image. That said, i am much more deliberate about everything when using a tripod. I can work with one element at a time...framing, exposure, composition...etc.
So to my answer...it depends on what you're trying to capture. The image you are aiming for may be strengthened by an impulsive mood....where in other images precision is needed. I usually carry a tripod...and when I don't...I usually wish I had one with me. For me it goes back to all the elements I try to put into an image.