Should I edit an existing puzzle when loopholes are found?

Question

I recently read a Meta question pertaining to question quality. This answer had the following text in bold:

If a mediocre question ends up generating great content, your first inclination should be to clean up the question to make it shine.

Which made me think about a recent question of mine. Shortly after my intended answer was given, several 'loophole' solutions were proposed, all valid but all reliant on a single loophole in my wording.

Should I update my question now that this loophole has been discovered? Or should I leave my question as-is and reward those answers that came up with a clever solution based on the original text?

Answer 1

This is tricky, because it pits question quality against answer quality.

On the one hand, questions should have a single right answer. If the loophole opens up several alternative answers, the question probably isn't that great in the first place, even if the answers draw water out of a stone.

On the other hand, in various parts of SE, you find the same sentiment that bad questions should be edited to fit good answers (like in your quote). The way I see it, if the answers are really all that good, it's safe to keep the question as-is. If there are only a couple correct, high-quality answers, leave it be.

Additionally, if you think the loophole drastically changes the question and generates different answers, consider posting the "fixed" question as a new question. Then you get to keep the good responses while keeping the answers.

Answer 2

I do agree that it is a hard battle to know whether you should edit a question to try and focus answers in. However, if you're far too detailed in the question, it may be too guiding of the answer. While, on the other hand, being too open will do what happened to you and allow multiple reasonably sound answers. I would love to say that it is impossible to put a strong guide on when to edit or not because puzzles are meant to open the mind up.

In my personal experience in https://puzzling.stackexchange.com/questions/3051/alexander-and-the-burning-forest/3057#3057 , I based my answer on the word island (which guarantees water) that was used in four places. It seemed like it was relatively sound and that this particular wording was used to give hints. However, without an edit, I'm not the only person to choose a water-based answer.

Side-note: Change a question after multiple answers have already went through may lead to the situation where others have to edit answers to say "This was before (insert new rule)". Happens every so often in the code golf when changes come around.