I’m trying to picture your set up because it sounds very similar to mine, but I have limited options for in/out access. My last barn I built from scratch and it had paddocks off of each stall which opened to the pastures. It was SO convenient!
My new barn has an enclosed lean to off the left side which is where hay is stored (and will remain!!) The right side of the barn is adjacent to a giant waterfall and koi pond which is several layers of boulders. So, the only option I have is to put doors off the back of the barn, which is where the drylots are. My plan is to put a door off the back left stall which will then go out to the dry lot, and a second door at the end of the aisle which will end up opening to the same paddock as the stall door, but maybe I can create a chute or something so that horse can go in/out at will too? It sounds like that’s kind of what you did so I’m really trying to picture that!
ETA: It would probably be helpful to give you my current barn layout for ideas… when you walk in, there’s a feed room on your left, tack room on your right. Then the left side has two stalls, right side has a wash rack, two stalls, and a people door with a small deck and some stairs (another reason I can’t make the stalls on the right in/out)