Consensus has been reached.

I say this on Rubio's authority - the contents of this post are considered to be a part of the definition of this site's scope.

So, I posted this open-ended puzzle a while ago, looking forward to all the fun around it and the upvotes that would pour in.


That question ended up with a score of +1, and 4 downvotes. (now +2, 7 downvotes. Meta effect. Also, I've updated the score multiple times, and I'm tired of it XD. My point doesn't really rely on the current vote count, anyway.)

It was my first and last open-ended puzzle. It also sparked some discussion on TSL, where Deusovi said "I don't think open-ended "puzzles" should be on the site at all, to be completely honest. They're fun games, but IMO they're not puzzles."

To which Rubio replied "I don't mind open-ended puzzles. At least, not if they're any good, and are well constrained," only to say "Ok I think we actually do have a couple [of good open-ended puzzles], but - the bad ones are way more prevalent."

Deusovi mentioned that he didn't have time to bring up the idea of removing open-ended puzzles on Meta, and he was unsure that the proposal would be viewed positively in the first place.

So, armed with my poorly-performing question and Deusovi's agreement, here comes the main point:

Open-ended puzzles are too broad for this site.

Some reasons why:

  1. There is almost never a final answer. Even the tag wiki admits as much! Questions with no final answer are unhealthy for SE's Q&A format. Also, to quote Deusovi again: "puzzles should have only one solution, and anything more than that is either a broken puzzle or not a puzzle at all."

  2. Minimal effort is required on the part of the asker, encouraging lazy questions in general. (Credit to North for this idea)

  3. The community isn't really a huge fan of these puzzles. I'm not the best puzzle-maker, but I'm not too bad, and I thought that my open-ended question was formatted and posed well. It still got 4 downvotes. That's a good indication that the type of puzzle it is isn't very popular.

With that being said, I hereby propose that we lock the "good" open-ended puzzles for their historical significance and close the rest.

Perhaps getting rid of the "open-ended" tag would be in order as well?

  • 2
    $\begingroup$ Or should we just combine open-ended and optimization? $\endgroup$ – athin Apr 20 at 21:00
  • 3
    $\begingroup$ I think the tag should just be reworded instead of gotten rid of as a whole $\endgroup$ – North Apr 20 at 21:46
  • 2
    $\begingroup$ Related: meta 1, meta 2 $\endgroup$ – PiIsNot3 Apr 24 at 8:05
  • $\begingroup$ Huh, the user that gave the only upvote to your question — that was me! Boy, do I feel a little warmhearted, considering that four other users downvoted... but I do believe you are right: open-ended questions are too broad, and questions that are too broad are unfitting, thus so are open-ended questions. Yes, some open-ended questions are popular, but when the third most popular open-ended question is closed, this is saying something. $\endgroup$ – Feeds Apr 26 at 1:24
  • 1
    $\begingroup$ @user477343 5 upvotes, 4 downvotes. You were one of 5. (thanks, though!) $\endgroup$ – Brandon_J Apr 26 at 1:25
  • 2
    $\begingroup$ So now that consensus has been reached, what do we do now? Do we deprecate the open-ended tag? What happens to the questions with that tag? Despite reaching consensus, I don't think we've defined a course of action to take besides that open-ended puzzles should be considered off-topic, and even that doesn't provide any guidelines on what to do. $\endgroup$ – PiIsNot3 Apr 30 at 19:30
  • $\begingroup$ That's true. We have defined "off-topic" moving forward, but we haven't said what to do with the old ones. Paging @Rubio ooooh-yi-aoooooh!! (or another mod - I just know that Rubio approved the consensus, so I'm paging him) $\endgroup$ – Brandon_J May 4 at 15:46

(Much of this answer comes from a discussion in chat with Rubio, Brandon_J, and North.)


I see several problems with the "open-ended" format. But first, a definition, to make sure we're all on the same page:

An question is one where multiple answers are expected; answers are ranked by some sort of rule, and the answer with the best score by that ranking is the accepted answer.

(Without the second part of that sentence, it's just a question that is intentionally too broad. These are already disallowed.)

So, what problems do I see? Well:

Dependence on other answers

In questions, an answer can be invalidated by other answers. If Answer A scores 18 points on whatever scale a puzzle used, and then answer B is posted (scoring 19 points), then Answer A is retroactively no longer a correct answer.

Compare this to regular puzzles: if someone posts a strategy for a "prisoners and hats" puzzle, it does not get disproven because someone found a simpler strategy. An answer there can only be invalidated by an inherent problem with that answer, not just "not being as good as this other answer".

Similarly, for another comparison, if someone posts a partial answer (to, say, a crossword), and then someone else posts a full answer, the partial answer does not become wrong. Even if the partial answer was posted after the full answer, the partial answer is still correct.

One goal of Puzzling Stack Exchange is to be an archive of high-quality puzzles that are solvable at any point in time, whether it's immediately after the puzzle is posted or ten years later. This seems to me to be in direct conflict with that: an answer at the start is not necessarily an answer later. Which leads me directly to another reason:

How do you know the best answer is the best?

With these questions, you can almost never know when an answer is correct. It's always possible someone can come along a year later and one-up an already-posted solution. So these questions are implicitly of the form "What's the best way to do X... that the PSE community has thought of so far?". That's why the ranking system exists: it allows you to pick out of the answers of the PSE community, not

So the top answer is in the same position as all of the other answers were: it's always waiting to be outdone. questions don't have a clear correct answer, and having a clear correct answer is a quality necessary to be a puzzle. And speaking of which...

These aren't puzzles. They're games.

questions may be fun things to think about. But they're not puzzles. As Rubio put it:

The danger - and I think we're correct to avoid it, and it's something Deus alluded to in an earlier comment - is that instead of a puzzle what we really have is a game: find the best thing you can come up with. (I've even seen some that are: best solution in days is the winner)

There is no "solution" to these questions, and that's by design. They're meant to be fun things to think about, but they're fundamentally not a thing that can be solved. Unless you have a proof of optimality (which the question is generally not designed for), all you can do is incrementally improve on previous answers and hope that nobody will improve on yours.

We don't treat analogous situations the same way.

I've already pointed out that we don't allow other questions without a single best answer. (Note that the ranking system doesn't satisfy this: it allows you to find a single best answer out of the ones posted by PSE members, but not a best answer in general.) We don't allow other questions whose answers change based on the answers that PSE members have posted -- that wouldn't be a self-contained puzzle. It's why was deprecated.

Consider a question that starts like this:

Here are 20 lines taken from various riddles. What object fits the most of these lines?

Assume, for the sake of argument, that whether something fits a line is generally not up for debate: they're fairly clear qualities like "this object is smaller than a breadbox" or "this object is green".

This question should probably be closed. Yet this seems completely analogous to, or even better than, questions as we have them. Any problems this question has apply equally to questions, yet we allow the latter.

The inevitable problem of HNQ

questions attract many answers. This is one of the criteria used to pick questions for the Hot Network Questions list, which is the first impression many Stack Exchange users get of Puzzling. We're putting our worst foot forward here, and showing questions that are (hopefully) not representative of the rest of the site's content.

There's unfortunately no way to show that this is driving people away without something to compare to. But I know that if my first impressions of the site were questions, I probably wouldn't have stayed.

So why do we make this exception? Why do we allow the tag to absolve askers of all responsibility for making sure their questions have solutions, and let them break some of the core principles of PSE (solvability at any point in time, a single best answer, self-contained puzzles)? I don't believe that there's any benefit to keeping these.

A proposal

In the aforementioned discussion, we eventually came to a conclusion that would draw a fairly clear line, while still allowing well-designed optimization questions. There are three things that might be called "open-ended" questions, that we'd still like to allow:

  • Optimization questions

These should have a provably best answer (ideally, not one that needs heavy programming or mathematics to prove, in the same way that we currently don't want puzzles that require heavy programming or mathematics to solve). Answers should come with justification of why they are optimal; an answer without this is not a full answer, but more of a comment.

  • Questions about whether something is provably optimizable

These are allowed, but answers are of the form "yes" or "no", with justification. Again, "you can get this much but I'm not sure if you can do better" is a comment, not an answer.

  • Questions about puzzle design that might have many possible answers

Here, we can follow the same rules as other sites on the SE network. These questions should not be asking for solutions to puzzles, but for the best way to accomplish some sort of design for a puzzle.

  • 1
    $\begingroup$ Soooo.....when do we reach consensus? $\endgroup$ – Brandon_J Apr 23 at 12:26
  • 2
    $\begingroup$ "...then Answer A is retroactively no longer a correct answer." How so? It's not wrong unless it doesn't solve the problem posed in the question; it's just no longer the optimal one. Someone can still solve the puzzle later and try to optimize their answer; they just might not come up with the most optimized answer. I don't see a non-optimal answer as any different than a partially solved crossword. The answerer got partway there, just not all the way. (I don't know what's best for this site, but that point just seems wrong to me.) $\endgroup$ – jpmc26 Apr 23 at 20:57
  • $\begingroup$ @jpmc26 The partial crossword is part of the completed crossword, but the non-optimal answer to the open-ended problem might have nothing to do with the optimal answer. $\endgroup$ – noedne Apr 24 at 1:50
  • $\begingroup$ @noedne Are you sure? Less optimal answers are often important steps in reaching the optimal one. $\endgroup$ – jpmc26 Apr 24 at 4:59
  • $\begingroup$ @jpmc26 not all open-ended questions are optimization problems. Optimization problems have a provably optimal solution, while most open-ended questions have an arbitrary set of criteria to determine the “best” solution. Optimization is OK; open-ended is not. $\endgroup$ – PiIsNot3 Apr 24 at 6:29
  • $\begingroup$ @PiIsNot3 The definition states that, "answers are ranked by some sort of rule." This rule is the "function" to be optimized, even if it's not presented as a formal mathematical function. By definition, it would seem that all open ended questions must be optimization problems of some kind, though they may not be provably optimized. Regardless, my point is that answers are still not invalidated by a new, higher ranked (according to the rule) answer being given. They're only invalid if they fail to meet the other constraints. $\endgroup$ – jpmc26 Apr 24 at 6:46
  • 1
    $\begingroup$ @jpmc26 The big issue with that is no single rule or even set of rules is enough to guarantee there be a single correct answer. For “longest word/sentence” type problems, the answer changes depending on what dictionaries you use and which words you allow. For “formation of numbers” problems, you have to define which operations to allow, what constructs are valid/invalid, etc. etc. Most questions tagged open-ended don’t have that level of rigor and instead devolve into a game of “what loopholes can I exploit that give a technically correct answer.” $\endgroup$ – PiIsNot3 Apr 24 at 6:59
  • $\begingroup$ In contrast, optimization problems are often grounded in math and logic, two subjects that have very clear-cut rules that don’t allow many loopholes. So that’s the difference between optimization and open-ended (or at least, what my understanding of the difference between the two is). $\endgroup$ – PiIsNot3 Apr 24 at 7:01
  • $\begingroup$ @PiIsNot3 The core of your point seems to be: "Most questions tagged open-ended don’t have that level of rigor and instead devolve into a game of 'what loopholes can I exploit that give a technically correct answer.'" That's a fine objection to the tag, but it has nothing to do with making one answer invalid by posting another one. As I said, I don't have any answers for whether the tag is bad or not; I just found the first argument to be very flawed, which hurts the persuasiveness of this answer. $\endgroup$ – jpmc26 Apr 24 at 7:01
  • $\begingroup$ @jpmc26 It’s an unwritten rule on PSE that all puzzles must have a justifiably best solution. Allowing accepted answers to be beaten by “better” ones goes against this rule, as Deusovi alludes to. Perhaps that section of the post should be moved to the top to make the point clearer. $\endgroup$ – PiIsNot3 Apr 24 at 7:05
  • 1
    $\begingroup$ Let us continue this discussion in chat. $\endgroup$ – PiIsNot3 Apr 24 at 7:10
  • 3
    $\begingroup$ @Deusovi I'd like to ask you to read over our chat conversation. I think you may find our discussion helpful in evaluating your answer. These two messages are perhaps of the utmost concern: 1, 2 (though the context around them is also important, of course). $\endgroup$ – jpmc26 Apr 24 at 7:51
  • 3
    $\begingroup$ @Barker Yes, I agree that it is a direction the site could go. I never claimed we couldn't do it due to Stack Exchange standards, just that it would be bad for the site and go against some of the goals we have. $\endgroup$ – Deusovi Apr 26 at 22:56
  • 4
    $\begingroup$ @Barker: codegolf.stackexchange.com has actually been strongly deprecating all the puzzles (i.e. one intended solution with no open-endedness); the "programming puzzles" in its name turn out to be better handled at Puzzling, because we're so set up for handling open-ended posts. I can't see any reason why having a clear contrast between the two sites would be a bad thing in this regard, because things like the site rules on meta need to be quite different in the two situations. (A simple example: on codegolf accepting answers is rare and looked down on, here it's almost mandatory.) $\endgroup$ – ais523 Apr 29 at 7:00
  • 1
    $\begingroup$ @Shinjo We've tried mandating a sandbox for often-poorly-made puzzles before. It did not go well. $\endgroup$ – Deusovi Apr 29 at 9:28

What is an open-ended puzzle?

If we're going to declare these off-topic, then let's have a clear definition so that the policy doesn't get misapplied. Deusovi has attempted to do this in his answer above, but I want to make it even clearer. (The reason for this answer is that I've already seen discussion over how/where it should apply, and I want to avoid a worst-case scenario where we ban all problems or "what's the most possible X" puzzles.

An question is one where there is no absolute "best possible" answer, so that any answer could conceivably be outdone in the future - the green checkmark will never be guaranteed to stay permanently in one place.


  • Growing anagrams - What is the longest possible list forming a sentence? is . The currently accepted answer has 16 words, but there's no way to "prove" that a given answer is optimal. Any accepted answer could conceivably be beaten by one with even more words.

  • Numbers on a Blackboard is NOT . The accepted answer proves that a particular number is optimal, but even if that answer hadn't been posted, the question still wouldn't be open-ended. It's clear that there will be an optimal number, and it just requires some mathematical ingenuity to prove it. Even if there had been several answers finding successively lower bounds, those would count as partial answers rather than open-ended attempts at the solution.

In an open-ended puzzle, any answer posted could be the best one, but even the accepted answer could be sub-optimal. If an absolute best answer exists and seems likely to be provable, then it's not open-ended: even if some answers are posted with successively better but non-optimal solutions, those are partial answers with increasing progress towards an upper/lower bound, not just iterations in an open-ended answer stream.

An open-ended puzzle could conceivably have arbitrarily many answers. That's literally what "open-ended" means: there is no upper bound on the potential for more answers, each one better than the last. (Here "arbitrarily many" might be literally "infinitely many" or it might be just "too many to reasonably fit on one Stack Exchange question". We don't want questions with ten billion answers posted either.) But if there's a question seeking an optimal number, and it's logically proven to be e.g. between 152 and 208, that's not open-ended either.

At the end of the day, this means we're likely to discriminate a little by tag. A "find the most X" puzzle is much more likely to have an absolute, attainable, provable, optimal solution than a one. That's just the nature of the beast. But hopefully the above discussion will help prevent the policy from being applied too zealously.

  • $\begingroup$ I agree with your ideas, but I most certainly disagree on GrowingAngrams being without an upper limit. As there is a bound on the length of English words, this will be the absolute upper bound of growing anagrams as well, wouldn't you agree? (And if there is a the argument of 'new' words being possibly invented, then one could easily restrict it by pointing to one of the "english-dictionary-lists" people have used for the answers anyway.) So, good arguments, maybe not the most convincing examples. $\endgroup$ – BmyGuest May 13 at 8:02
  • $\begingroup$ @BmyGuest No offence to your question (which I upvoted years ago), but I'm not sure if that "bound on the length of English words" falls into the category of "too large a bound to reasonably fit in a SE post". I'm happy to edit my answer and replace that example by another one if you can find something more clearly open-ended. $\endgroup$ – Rand al'Thor May 13 at 8:06
  • $\begingroup$ No offence taken. (And I think thanks to your post, the questions has recently seen a bit of traffic ;c) ) But your answer made me wonder a bit, and I'm actually becoming less and less convinced of the opinion. I'm thinking of coming up with a counter-arguement in this discussion, but am unfortunately running low on time right now. $\endgroup$ – BmyGuest May 13 at 8:09
  • $\begingroup$ Or maybe not. Considering that "consensus" has been reached. (Even if I might not consent ;c) ). $\endgroup$ – BmyGuest May 13 at 8:10
  • $\begingroup$ @BmyGuest Personally I'm not sure if I like this consensus either, but I came here too late to turn around the decision, so it seems this is our new policy. I'm just here to make sure it's not applied over-zealously ;-) $\endgroup$ – Rand al'Thor May 13 at 8:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .