I have definitely day dreamed about doing this, especially when I see openings with trainers that I really admire/would love to work with. I did attempt to do the whole barn manager/work in the horse industry gig right out of college, and only lasted about 6 months. I honestly think it was the trainer I was working for and not so much the industry itself, and hopefully now I would have more life experience to be able to pick out a trainer and program that would work better for me. I have too many animals to truly consider taking a break from my career right now, but just wanted to say you are not alone in thinking about doing this!
