Doing the giraffe thing, star-gazing, dropping his back? I’ve had some luck re-schooling some gaited horses with these habits; it is possible.
What bit have you transitioned the horse to? Does he have an effective half halt? Does he have a regular walk? A trot? A hard pace as his gait, or is he square?
I’d probably use one of my go-to bits: Myler D with hooks or a KK eggbutt, and might even let the horse feel comfortable enough packing it to lean a little bit, then work on getting a true half-halt, using one-rein in more of an upwards (rather than pulling back) fashion. Just guessing here, but his previous rider(s) may have kinda water-skied this horse.
Really work on the timing - a quick check with a fast release. Hate to use the word snatch, but it is a very brief checking, just enough to re-balance without causing him to stiffen up and hollow his back (if he has the time to get into that posture, the check took too long).
I’d also be very conscious of my seat – no feet on the dashboard (just guessing that he’s probably been ridden like that by the same rider who used the long-shanked, ported bit, as they seem to go together often) – making sure I’m in the middle of the horse with my legs underneath me. I like to make a clear differentiation between what I’m doing and the horse’s past. He can learn the difference!
I’d throw in some groundwork, too (not PP stuff), such as yielding to the bit from the ground (like what Alfredo H. does), where the horse doesn’t have to deal with weight on his back and I can see (and reward) a response immediately. I’m also a big fan of long-lining.
Lots of cavaletti (at a walk) to drop his head and build up his abs, using hills, too, if available. I’d want him to develop the musculature to engage his abs and raise his back, because that will help, IMO.
I’d start out at a slow speed because, IME, if one of these gets excited, then they can get into a zone where it’s “Earth to Horse, come in,” so I don’t want to go there. It’s a bit like riding a wave, finding the point where the horse is in balance, not leaning nor hollow, and working there for just strides at a time, building up the horse’s strength and muscling until he can move correctly. My goal would be to back off right before he loses it.
Hope this is helpful, sorry if I’m telling you stuff you already know, just mean to describe what’s worked for me. Good luck!