There are rules and then there are traditions. In the equitation and hunters, even though the rules say that white is acceptable, a lot of people choose to not wear white because they don’t want to look different.
Even though some judges may not actively penalize someone for wearing white, it may call attention to the rider in a way that is not positive.
A rider wearing white in a sea of riders wearing beige is going to stand out. If that rider exhibits a flaw such as a swinging lower leg, heels that aren’t down, etc. it may be more noticeable to the judge. Wearing the same colors as everyone else is like camouflage in a way.