Is where you turn him out a place with distance between fences, an alley at least, so there is no direct across the fence aggression?
Would that help, to put him where he can see but not interact with others right across the fence, so no kicking over or at the fence?
In breeding farms of old the turnout paddocks for stallions were some that didn’t have direct neighbors for the aggressive ones and others where they could interact across a fence for those that played nicely with others.
I know a mare that has always liked her neighbors fine.
Then this new gelding was put in the pen by hers and she took an immediate dislike to him.
He was not doing anything we could tell, didn’t even get up to the fence, flat ignored her and her posturing.
The fence between pens was solid wood vertical planks and heavy duty pipe top and bottom.
After a few days, in the morning the fence was torn in some places into the gelding’s pen, the heavy duty metal the boards were attached to bent, boards in splinters.
Luckily no one got hurt and each were still in their paddock.
It was impressive to see what some well aimed kicks can do to a solid fence, when a horse is mad for whatever horse reason.