Barns are very different, and add wind, and all that “radiated” heat can really disappear fast. My pole barn is not insulated, and has very high ceilings (24 feet) so it does not heat up enough to warm horses. It isn’t air tight, and when the Gorge winds are howling (you know, steady at 45mph, gusts past 75mph like normal!) it’s uncomfortable inside. My retirees with unclipped coats wear sheets until it gets windy, then the layers go on (medium/heavy liner depending on which horse–the TB grows virtually no coat).
The boarding barn I was last in was in a historic dairy barn with very low ceilings, concrete walls, below a hay loft. With the doors shut, it was very warm with all the horses in their stalls, so layers were often removed down to a stable sheet unless it was bitterly cold, like you all are experiencing.
Honestly, if my water and bedding were freezing, I’d never remove blankets. If you’ve been blanketing horses up to this point, I’d never remove them (maybe remove a layer). If my horses were clipped, those blankets would stay on.