You have just opened the door to learning “equestrian tact.”
You say the horse is using you for balance. So, don’t allow it. If he starts to lean, dont’ hold him. Ask him to go forward. This is counter intuitive that if a horse is heavy, you ask for forward. The reason the horse leans is because the hind legs are trailing. So you ask the horse to bring hind legs under. IOW you ask for forward. If the horse then leans, you say NO! Don’t lean. It is a training exercise where you develop the understaning in the horse.