Not all "screwdriver" AF lenses have an AF/MF switch, and those that do, don't all behave the same.
To answer the question, on some AF lenses (the 180mm f/2.8D is one), the focus collar cannot be turned unless the lens is switched to MF, but with the lens in MF, turning the focus collar rotates the screwdriver clutch. That means the camera has to be switched to MF as well, otherwise the camera AF motor will be driven in reverse, with risk of damage.
There are a lot of annoying/awkward solutions implemented in screwdriver lenses.
I own Sigma, Tokina and Pentax lenses which use a clutch mechanism. If you pull back on the focus ring it engages autofocus, push forward for MF. Or perhaps it's the other way around on some or all of them -- I can never remember. Even worse, sometimes you have to turn the focus ring in order to get the clutch to engage and sometimes you don't.
The earliest 180mm f/2.8 AF lenses didn't have an A/M switch at all.
You don't have to sort of enhance reality. It is already pretty weird ... you know that there is nothing stranger than truth. Annie Leibovitz
I moved to a Tokina 28-70 briefly before my first AFS lens which was the affordable 24-85 G. I thought the manual clutch was the coolest ergonomic design ever. On reflection, it's very instinctive as opposed to reaching to the side for a toggle switch.
Oops! So it's lens dependent and I've been operating my 200 and 700 screw drives in reverse as well. Thankfully, risk of damage hasn't yet occurred and now I know better. Thanks B1 for splaining. And I'm with you, B2, annoying. Still, I'm willing to live with it on the 180 because it's so darn good.