There’s some apple trees a ways off from my barn that attract critters—deer, raccoons, squirrels, rabbits, bees, my dog, and I’m not sure what else. I just see lots of critter tracks and apples with bites and holes and scratches on them. They’re in a tangle of forest though, so a bit hard to pick up all the apples.(I pick up what I can reach that hasn’t been eaten and leave the rest for the critters) I’ve also got some designated apple trees in a clearing, and I pick all the apples before they fall on the ground and don’t notice much if any critter activity there. And I’ve got a crab apple tree in my pasture, which my horses run to first thing in the morning.
I’ve got a couple of mulberry trees closer to the barn, which the raccoons will climb up and lay in like cats, getting drunk on mulberries and pooping all around the tree, which is annoying. I think that’s worse than the apple trees. (Plus, I love mulberries, but between the raccoons, squirrels, and birds, there’s none left for me)
In general though, I think if you keep the area around the tree tidy and pick the fruit a few times a day they won’t be a problem.