Assuming you’re in a temperate part of the northern hemisphere, it’s pretty late for a nest to be starting. By late August I suspect what you’re seeing is the rapid population increase of a well established nest, not a new population.
I’d inspect the exterior of the shed and spray if you find the nest. If you think the nest is inside, then you have more trap options available than if it were outside (the bait type traps that are best outdoors aren’t great for catching paper wasps IME). Sticky traps are terrible outside because they can kill songbirds and get all sorts of other stuff stuck to them, but if you put one inside the shed you might be able to knock down a large portion of your wasp population without pesticides near the straw. If you can figure out where they’re entering and exiting, locating a trap inside the shed near the entry point might work.
For next year, putting screen across points of entry (in my shed a screen across the vent grate keeps them out) and inspecting a few times in June/early July for the first signs of nest construction (and killing/cleaning up incipient nests early) might help prevent a repeat problem.
Good luck!