Go forward. Go in a field and establish a good hand gallop. Then come into the arena and establish the same thing. Contact, etc, comes second to actually going forward. Go for a brisk canter on a track, march out on a trail, etc.
Forst and foremost you need to get him going forward. All you are doing with your aids right now is nagging. One of my instructors had me do a hand gallop in two point (yes, in a dressage saddle) with no contact as part of my warm up when I was on a horse who has a tendency to really suck back behind the leg.
Once you get some form of forward you can work on refining it (contact, straightness, rhythm, transitions within the gait and between the gaits) but you need forward to actually be an option first. So long as it isn’t, you’re basically driving a compromised vehicle so long as the engine isn’t working. Nothing else can happen until you fix the engine.
Once you have him thinking that forward is actually a thing then I would look at sharpening the aids. There are a lot of good excercises out there depending on your horse’s level of education. As a four year old I assume he’s not up to advanced questions but even something like starting with a transition at every other letter and then as he gets
better at it, a transition every letter… between the gaits, within the gaits, as much with the seat as possible. And then a preparation for a transition down oh wait just kidding I actually want a bigger trot now. Can be done on a circle at the cardinal points but I prefer the whole arena to avoid so much time on 20m.
That said there is a good and fairly simple exercise on the circle in the canter… pick up your canter at 12oclock, and then canter one whole rotation and then trot just past twelve - canter within three or four strides of the trot again. the inverse is that you spend more time in the trot - trot at twelve on the circle and trot a whole rotation, canter just past twelve and canter only three or four strides before trotting again. Trot a whole rotation again and pick up the canter just past where you transitioned down to the trot the rotation prior. This form is a little more helpful with horses that are too strong and want to run in the canter though - the variant with more canter is generally more helpful to me with horses that aren’t so forward because you are both not cantering for too long, transitioning and repackaging, and then transitioning up again. Keeping them packaged and the quality of your preparation and half halts is key.