I believe I read that horses will avoid eating from the roughs for as long as 3 years after they have been used as manure areas; so dragging won’t really change that, nor will rotating, unless you have tons and tons of pasture.
I drag my pastures for aesthetic reasons more than anything else. I don’t want to look at piles of manure in them, nor do I want to pick 4 acres of manure. But I try not to drag the manure into the lawns, since that is only likely to make the grazing area smaller.
I’m not particularly worried about parasite contamination by dragging as I have big enough pastures and supplement with enough hay in a drylot that my horses don’t have to graze from the roughs (nor will they anyway). And I do fecals and deworm twice a year and/or as needed.
So, I would say go ahead and drag your pastures; but don’t expect that by doing so it will necessarily force your horses to graze the roughs more than they do now.