>>"I get that the combination of my test subject and lighting were sub-optimum, but that was somewhat by design. Real world shots don't occur in idealized conditions."<<
This is one that I have struggled with as well. But, in reality, it is necessary to test the "lens" in an optimal environment to see if it's the "lens" that has the problem. Every time I get a new lens, macro filter, TC, etc. I am surprised by the results.
Once, the lens can be eliminated as the problem, then it is time to discover what you have a right to expect in the conditions in which you are shooting. Then, like you said, you may find that your expectations aren't easily achieveable; with the type of gear you have, and in the environment that you shoot.
Then you need to discover if there is some skillset that you may achieve to help match your expectations. There is bound to be some Nikonian on here that can help with that. Or, you could just explain what you are trying to achieve, and get advice on how best to achieve it.