We use the BIG dog crate when training outside cat to use a litter box. Cat is locked in the crate with a litter box, food, water, milk crate with towels over holes for laying on. Cat is contained, no other batroom options to use except litter box. Just put cat in crate when it comes inside. Then put cat outside when he wants out. You can’t be “too late” to let him out if he is crated.
Sorry, I have not found outside cats to be reliable for potty things if not contained by the litter box. Eventually most cats will use the box reliably, won’t need to be kept contained all the time. Might take several weeks though, so don’t turn him loose too quickly. True that they will often NOT use the box even if cat is locked in bigger spaces, like the laundry room for example. They just never learned house manners.
Is cat fixed? He might want to mark territory by spraying, which is very difficult to train him out of. They can still smell the spot, even if you cannot, likely to remark spot again. Sorry to hear about your rug. Nice of you to try bringing him inside with him being older!