What you describe is pretty much what I get in my hands during the summer in super air conditioned places. (I’ve actually been known to carry around fingerless gloves in the spring/summer when going to classes, because some of the rooms are so super-cooled that I can’t write otherwise.)
I have confirmed arthritis in both hands due to one of the systemic forms of arthritis. (As opposed to injury-related or use-related osteoarthritis.)
In addition to the gloves, I also might take an NSAID, or go through a routine of gentle stretching and movement to help encourage circulation and disperse swelling.
I also have arthritis in my ankles, knees, and hips but they’re not as sensitive to cold places so much as just lack of movement - so when sitting I tend to shift around a lot, maybe straighten one leg some and flex the other more, shift my weight to one side of my butt more than the other, then reverse, etc. All very small relatively unobtrusive movements, but they keep things moving.
If it’s a real problem for you, you could also look into neoprene wraps. They’re relatively easy to come by (most large supermarkets around here stock them) and while I think they do exactly nothing to help the arthritis from a support pov, they do hold in your body heat, which helps.
(I will also add, regarding NSAIDs, that I’ve read that you tend to get better results for chronic arthritis if you take the NSAID routinely as per the dosing instructions, rather than only taking it when it hurts. I don’t take NSAIDS all day, everyday, but if I’ve done something that I know is going to have particularly P.O.d a joint - like spending a day walking around on pavement in shoes without enough cushioning, which is hell on my knees and ankles - then I do take aspirin regularly for the next couple of days - every 6 hours or whatever - rather than just when it hurts.)