My situation was similar when we bought our property, but with a few differences. Our farmette is part of a larger farm that was divided into small horse properties. After we bought the house, I had an arena built on the only suitable spot on a our gently sloping acreage. Our next door neighbor had lived here for 30+ years and had always had horses on her one acre lot. She was unhappy with the arena, claiming it ruined her view. (There is never any dust, as I water before each ride.) We did discuss it and remained friendly.
One day her hubby moved a large decorative windmill to their fence line right next to the arena. The noise and movement spooked both my horses. I politely talked to them about it and they moved it. Whether they did it on purpose out of spite I will never know. I do hope your efforts to mitigate the problem work and your neighbor stops his disturbing behavior. Behaving in a way to potentially cause an injury is something that should be discussed with a lawyer. (Only if it continues.)