Given the additional comments about your goals, I do think the trainer is out of line (more from just being polite vs being unprofessional). There is definitely a different mindset (speaking generally again) between owners and pros. Pros want to get horses going so they can make money, either by training fees, show fees, or a commission. Owners are usually enjoying the journey vs the destination. It could be that mindset coloring those comments. I still wouldn’t take super kindly to them. IIWM, I would ask “what about pony makes you say he is not where he should be?” Or “what do you think needs improved to get where I want to go?” Those answers would help me decide if I wanted to continue to ride with this trainer or not.
Second is an extremely reasonable goal and should be well within reach for an OTTB, especially if you have no deadline.
It sounds like you are greatly enjoying your boy! Slightly off topic, but I think bringing along an OTTB is one of the most rewarding things you can do in the horse world. They are fantastic partners with great work ethics and so much athleticism. I hope we hear more updates about your journey with him!
