I feel you. I spent 30+ years in the hunter/jumper ring and I never thought I’d “give up” jumping. I fought with myself over the fact that I just didn’t have the confidence that I had when I was younger (after a few bad falls) and that I was “giving up” or “giving in” by focusing on Dressage (a discipline that I had found incredibly boring and “fiddly” in my younger years). I wrestled with notions of being “that middle-aged ammy” and all sorts of super self-limiting and self-critical beliefs.
However, what I’m finding is that I actually have to be a bit braver to do dressage than I was doing a hunter course. I could dawdle along to a 2’6"-3’ jump course at the pace of a snail, but I really need to have the engine going to get my horse engaged at all. Instead of looking constantly to “quiet him down”, I’m looking at muscling him up and helping him engage his body with the end result that he is more/bigger/more expressive. I didn’t understand that before I really started diving in. And of course, as I ask him for more, I get more (not always the way I want it, but we’re working on that). Not always comfortable, but it’s the right way to progress.
What really happened was that I changed my mindset from “giving up” to “gaining”. Transformed the whole thing for me. I think I’ll ultimately be a better jumper if I ever go back (not sure I will, but if you would have asked me 10 years ago if I’d have been focusing on 20 meter circles, I would have told you you were crazy). And yes, we’ll probably still pop over a fence or two here and there because we’re dabbling in working equitation as well, but this journey has transformed what I think about riding.
You can always, once you’ve spent some time in dressage-land, go anywhere. In my equine journey I’ve stopped off in western-land, saddleseat-land, driving-land, hunter-land and eventing-land. I’ve learned something from all of them
Good luck to you!