I disagree with this statement. I’m “average” height for a woman, 5’5", and I can easily fit on a 15.1h horse. The barrel makes much more of a difference in how a horse takes up the leg/how you look on a horse than the height at the wither. This notion that you “can’t” ride smaller horses or that you are “too big” to them is not as common as we’re led to believe. Again, Scott just won Grand Champion on a 15.1h horse - the judges didn’t seem to think he was “too big.”
Not saying it DOESN’T happen - but more folks are fine on smaller horses, they’ve just been told they aren’t.
With ponies, the “small medium” does have a disadvantage because the jump heights are different. Not so with the horses, so I’m not seeing that logic (though I can see how that story can be told).