Could be the twist of fenders making ankles hurt, over the (light weight) stirrups being aluminum. Have you had any issues using any model of wooden stirrups? Even well broke-in fenders will pull your toes a little, trying to go back to being flat on the horse sides. This pull can also bother folks with knee problemss when riding. Rewrapping your fenders to be twisted all the time can remove that pull. I have seen this rewrapping demonstrated on Utube video and our local tack store will do it for you. They call it The Texas Twist, it is very popular with Trail Riders here.
If you still want wooden, what kind if riding do you do? Stirrup shapes are made for different reasons, each better suited to some jobs than others.
Just wondering, how do you feel the weight of the stirrups so it bothers your ankle? I can’t figure how you could without lifting them, which does not sound reasonable. A constant pull on foot from fender twist, could cause pain in the ankle.