When I first started hunting it was drilled into me that it was the riders responsibility to keep up. If you didn’t it meant the field master or hunt staff had to go search for you. Unfortunately as a field master I’ve had to do that on occasion. At that point, barring an injury to horse or rider, someone got a piece of my mind.
The other version of keeping up is within a field. We’ve all been there where someone riding in the middle of the field keeps falling behind and then they trot to catch up. Of course, this causes a great slinky effect to all of the people behind them which eventually leads to horses jigging. To me it is very inconsiderate to be a person who won’t keep their horse in the proper line of march. If you have a horse who is just slow, whether you have colors or not, your proper position is at the back of the field.