The issue with barn cats and responsibility reminds me of the Great Kitten Summer of 2022. 
Owner moves and leaves 3 pregnant, feral, barn cats behind (two were semi-feral). They asked for two kittens from us. We did right by the kitties, but made zero effort in saving two kittens for them. It never came up, but I had no intention of perpetuating their version of cat care. The kittens would have never been spayed or vaccinated. Sure, they provided shelter in the winter and low grade cat food, but since that didn’t meet my standards, we looked for indoor homes for the 14 kittens.
If they had reclaimed and collected the cats right after we moved in, fine. They did not. In fact, they had told us they were taking two with them before closing, but then changed their mind the day we arrived. Once we brought those cats in and cared for them our way, once they had names and a vet record, they were ours. That means we got to choose where the kittens would go.
Had I been in Neighbor X’s position, I would have asked about who owned the skinny cat that I saw coming by and, knowing me, visited the owner to ask more details, such as age, spay or neuter status, and vaccination status. Sometimes, you can nudge people into the right direction, if you aren’t overly confrontational. If you want to take over care, yes, you can be more confrontational. Hey, I’ve got kids/livestock and I want to make sure your outdoor cat doesn’t turn up with rabies, there’s a low cost clinic, do you mind taking your cat there?
The the way Neighbor X handled it was horrendous. They made friends with the cat for six months, essentially taking over the care, but did a lousy job by not vaccinating or providing flea meds, and then abandoned the cat at the pound. And people wonder why cats at the pound look depressed.