Don’t “force” him out on the trail… once he gets to the point where he gets anxious he will turn and head for home, let him. Put him to work a bit (small circles, walking, trotting, changing direction, disengaging, etc) when you get back to the barn yard. When you feel him start to search for another answer (where can I get left alone?) point him back towards the trail, let him walk on a loose rein - and when he gets far enough, gets anxious and turns around for home… let him. Work him again when you get back to the yard.
When we grab our horses faces, and put leg on - to convince them to go somewhere they don’t want to go - it’s doing the opposite of what you’re trying to accomplish - and you’re putting more and more pressure on them the farther they go, the more anxious they get - and it just reinforces the idea in their heads that “yep, this is exactly where I do not want to be!”. Of course they want to go home, right? That’s where the pressure is off, plus all the good stuff is there.
Work on little bits at a time, going further and further each ride. If you can’t already walk / trot / canter on a loose rein at home, then you need to work on that first.
OR get used to it and ride through it. But honestly, the former works quite well.
