Our first farm was set in the middle of a rural area --no neighbors at all. We built our house and barn and life was good --UNTIL the area around us sold and houses began to be built around us. One house was built only 19 feet from our back fence. Soon, there were problems --some our fault (we were used to living in a rural area and slow to adjust to a more subdivision vibe.) DH was not good about walking the dog and had developed a habit of just “letting him out,” and neighbors complained (justified). Their kids fed the horses through the fence and climbed the gates breaking them down. Our manure pile attracted flies (yep, it did) and was close to their back sitting area. Finally, after the umpteenth time someone complained, DH said, “sell it.” So I did.
I found a place with 20 acres and a house and barn set squarely in the middle --no more problems.
Now all our neighbors are wonderful Amish people who maintain their fences, keep their kids on their side of the fence (except when they are over at my place painting gates, mowing, or doing other kid-friendly work for me) --much better!
So, in my opinion, with such a close proximity to neighbors, you might find non-horse neighbors a problem eventually.