16
$\begingroup$

A while ago, consensus was reached that open-ended puzzles are no longer welcome on our site. That means, if a question has no absolute "best possible" answer, so that any answer could conceivably be outdone in the future, then we're supposed to close it as too open-ended to work well here: there's no way to know when/whether an answer is optimal, and there could conceivably be arbitrarily many different answers.

We still have an tag, though, and not every "find the least/most/largest/smallest" puzzle is off-topic. Quite often such puzzles have an answer which is provably correct. Hence why I posted an answer on that previous meta thread to clarify exactly what "open-ended" means. I also said:

If an absolute best answer exists and seems likely to be provable, then it's not open-ended: even if some answers are posted with successively better but non-optimal solutions, those are partial answers with increasing progress towards an upper/lower bound, not just iterations in an open-ended answer stream.

Nowhere on that meta thread was anything said about deleting answers. (You can check by searching "delet" on the whole page.) However, a few hours ago one of our moderators deleted all answers to this question except the one which brute-forced all possibilities with a computer - including the optimal answer which Bass skilfully found, as well as other answers which were optimal at the time of posting. The deletions were justified by the following comments:

Based on the rules established by this meta post and the consensus around it, an answer must have justification for why the solution is optimal. Without that, this is a comment, not an answer.

Apparently the "rules" he was thinking of was the following quote from that previous meta:

Optimization questions should have a provably best answer (ideally, not one that needs heavy programming or mathematics to prove, in the same way that we currently don't want puzzles that require heavy programming or mathematics to solve). Answers should come with justification of why they are optimal; an answer without this is not a full answer, but more of a comment.

But since the main discussion was about closing open-ended questions, it's arguable whether there's a clear consensus on deleting certain answers to non-open-ended questions as well. Neither the quote just above from Deusovi's answer, nor the other one from my answer, received any disagreement from the community ... however, they weren't really integral parts of the discussion, which was primarily about the handling of certain questions rather than certain answers to different questions. So here's a new meta to get consensus on this point.


If an optimisation-type puzzle is NOT open-ended, and answers are posted which give bounds on the solution WITHOUT proving optimality, should those answers be deleted?

For example:

  • Q: Find the smallest possible number of X needed to satisfy Y.
  • A1: Here's a way of getting Y with seven X.
  • A2: Here's a trick that enables Y with six X.
  • A3: Here's a proof that it can't be done with four X.
  • A4: Here's a proof that six X is the least possible.

After (or even before?) A4 is posted, should any or all of the other answers be deleted? A4 is the only one that proves optimality, but the others all establish bounds, A2 is the actual optimal solution (only without proof), and A1 was the best solution found at the time of posting.

| |
$\endgroup$
  • 3
    $\begingroup$ Without making judgement in either direction, another aspect that muddies the issue is A5-A9: Here's a bunch more examples that also get Y with six X (often with no clear distinction as to which is "better"). $\endgroup$ – Alconja Sep 13 '19 at 14:13
  • $\begingroup$ It looks like this question warrants the creation of a chatroom because of how large the issue is. You can find the chatroom I made here $\endgroup$ – Adam Sep 18 '19 at 19:12
8
$\begingroup$

Having written a lengthy meta post on this same subject myself, I took a deep breath, rethought the matter, erased my post, and here's my final policy suggestion:

Let's not have a policy on this.

Let's face it: the problem here was not that we didn't have a policy for this particular situation. Instead, the trouble was entirely caused by the fact that we had one policy too many, which led to a misunderstanding in the enforcement step. I'm pretty certain that no-one would have even suggested squashing an entire boatload of decent answers, unless there was a written policy somewhere that seemed to prescribe that the mods must do so.

At the very core of the Stack Exchange network we already have a mechanic for judging whether an answer is useful for the site: the user votes. Since everything we do should be for the good of the main site users, there cannot be a more direct indication for what's good and what's not.

If there's ever a reason to go against the user votes by mod action, it should be something quite extraordinary, like demonstrated plagiarism, or hidden personal attacks on other users, or something on that order of seriousness. If there seems to be a policy in force that suggests deleting a 10+ upvote post, it's more likely that we should review (and possibly abolish) the policy, rather than the well received post.

Deleting useless posts that usually stand at zero votes or under, like late dupes and such, is of course a perfectly common task, and should go on as usual.

TL;DR

Let's not solve the problem of too much legislation by adding even more legislation.

| |
$\endgroup$
  • 7
    $\begingroup$ I like this answer. In the ideal, stack exchange sites should be mostly self-moderating through the upvote/downvote system. If your deleting an answer with 25 upvotes in the name of consensus, something has gone wrong somewhere. $\endgroup$ – hexomino Sep 13 '19 at 14:04
  • 3
    $\begingroup$ @hexomino Keep in mind that many bad (or non-)answers have gotten plenty of upvotes due to HNQ, and then eventually been deleted. This is a particular problem with improperly specified riddles that attract lots of 'lateral thinking' answers, but that's not the only place it happens. I'd say a lot of upvotes, especially on an HNQed question, are not as much evidence that an answer is good as one might think. $\endgroup$ – Deusovi Sep 13 '19 at 16:38
  • $\begingroup$ @Deusovi I think you may have it backwards. If the community (yes, it includes the "Hot Network Questions" casuals) likes a post, it doesn't really matter if the post is up to all standards, or even gives a plain outright wrong answer: the site is nothing but its community. I agree that sometimes pure garbage gets upvoted, and the other way too, but this particular matter is up to the community to decide. If the community upvotes sub-par answers, that is the community's right. If you disagree with the community, there are many correct actions to take, but mod-delete isn't one of them. $\endgroup$ – Bass Sep 13 '19 at 17:24
  • $\begingroup$ I'm not attempting to disparage people coming from HNQ as 'casuals', or anything like that. But we have put in place policies earlier on deleting 'lateral thinking' answers to non-lateral-thinking questions -- that was one of the earliest problems the site faced (after the migration to allowing "challenge questions". $\endgroup$ – Deusovi Sep 13 '19 at 17:30
  • 1
    $\begingroup$ If we're going to ignore all previous policies just because something gets a lot of upvotes, then why do any rules have any meaning? Why isn't the only close reason "the community doesn't like this"? Why isn't the downvote hovertext "I don't like this answer"? Because the community does have rules - the rules are set by the community through meta posts, yes, and they can be changed by the community. But those rules are still important. $\endgroup$ – Deusovi Sep 13 '19 at 17:33
  • 3
    $\begingroup$ (I'm not trying to defend the deletions here - I acknowledge that that was a mistake on my part. But I don't think "anything goes, as long as the people who see it like it" is a good, or viable, way to run a site.) $\endgroup$ – Deusovi Sep 13 '19 at 17:35
  • $\begingroup$ @Deusovi I'm with you on all your points. I went a bit overboard, and certainly should have used a less confrontational tone of voice. Sorry about that. Of course we need rules and policies, and I am overall happy with the way they are handled. I guess I'm trying to say that math textbooks sometimes contain very good puzzles, and some uninvited lateral thinking answers are really entertaining, and if they are appreciated by the community, we probably shouldn't instakill them on the technicality of them infringing on a policy that was instated to block some other (often unrelated) problem. $\endgroup$ – Bass Sep 13 '19 at 18:34
  • 1
    $\begingroup$ I agree that math textbooks sometimes do contain good puzzles - which is why we disallow "textbook-style" questions that are solved through rote application of an algorithm, rather than "questions that happen to be in textbooks". And uninvited "lateral-thinking" answers may be entertaining, but they hold no value for the site as an archive of high-quality puzzles and answers. (I'm perfectly happy to see joke-y "solutions" posted as comments, but answers should be reserved for... well, actual answers.) $\endgroup$ – Deusovi Sep 13 '19 at 18:42
  • $\begingroup$ @Bass Your point is similar to one made by Shog9 on main meta: rules should be applied with knowledge of why they are the rules, rather than just applied blindly because they're the rules. That said, I agree with Deus's reservations wrt HNQ, and there are some highly-voted answers which really should be deleted. Still haven't decided how to vote on your meta answer. $\endgroup$ – Rand al'Thor Sep 13 '19 at 18:44
6
$\begingroup$

First of all, I'd like to apologize: I vastly misinterpreted the community consensus on this issue, and I didn't realize that I had inadequately communicated my "open-ended" proposal's intended effect on this type of question. I should not have deleted the posts there, and I fully acknowledge and apologize for my mistake.

That being said, I would like to make an argument that calling these types of answers "Answers" (even partial ones) doesn't make sense, and we should reconsider how we deal with them.

I think single answers to optimization questions, without proof of optimality, should be compiled in a single Community Wiki post rather than as their own individual answers.

I think this is especially true for questions, but a lot of my thoughts here apply more generally.

Answers invalidating other answers

This was one of my core arguments against allowing open-ended questions: it didn't make sense to have a question where answers became retroactively invalid. That's one of the reasons we don't like people making major changes to their puzzles after they have been posted. As Rubio says in his stock comment mentioned in this post about changing questions,

Changing your question after you've received answers is inappropriate, as it invalidates the answers you've received. It can even make those answers wrong, and adversely affect the reputation of those who answered. If you now have a new or additional question, create a new post and ask it there; you can link back to this one if needed for reference.

[emphasis mine]

The major problem here is that answers do invalidate other answers. In Rand's helpful example, once A2 is posted, A1 is no longer a valid Answer. If the question itself said "I can do Y with six X - what's the least number of X possible?" then A1 would not be an Answer at all. (And these answers are posted for the express purpose of collaboration: they're not hidden information, so it's not analogous to a full answer to a different type of puzzle 'invalidating' a partial answer.)

So now we have a state where later answers to questions can change whether other answers are An Answer or Not-An-Answer. This seems to be a problem with how we determine Answer-hood - the status of an answer shouldn't change based on a separate answer.

Analogy to Partial Answers

Adam draws an analogy to partial answers in his answer above. I don't think this is entirely analogous, though: the word "progress" seems to be used for two different things.

The progress of a partial answer must be extendable into a full answer. As I argue in the recent discussion on partial answers with 'list-of-clues' questions, a partial answer that is not demonstrably extendable into a full Answer is a comment, not an Answer. Even if it happens to be the correct path, if it's not shown to be the correct path it's not a Partial Answer.

For instance, say someone gives a simple puzzle where eight crossword clues each give a word with a double letter, and those letters spell out the answer. Someone else gives an answer to the question that gives a guess at one of the clues and guesses that all the answers have a double letter (maybe based on the title). This is still just a comment rather than a Partial Answer, even if it happens to be entirely correct: it's not known (or even thought very likely) to be extendable to a full solution, so it's a 'fragmentary thought' rather than a (Partial) Answer.

The same applies here: the nature of these optimization problems is that a single solution does not necessarily lead to any better ones. Giving an answer with 6 Xs might be the right one, but it also might be entirely useless for finding the way to make Y with 5 Xs. There's no way to tell whether a solution happens to be the correct one without that proof.

Overall impact on quality of answers

It also seems to me that these types of questions encouraging partial answers may give the mistaken impression that more "gamelike" open-ended questions, where a bunch of people each give their own maybe-valid answer -- the exact type we decided were bad in the other 'open-ended' meta post -- are encouraged as high-quality questions. This is not necessarily a problem with the question itself (as Rand said, these types of questions are not off-topic according to the previous meta post, because they are not necessarily open-ended). But it may be good reason to reconsider how we organize answers to them. At time of posting, the two relevant questions have 16 and 13 answers, respectively. That means the first must have at least 6 invalidated answers, and the second must have at least 12 -- most of which have nothing even attempting to show optimality (which is the actual hard part of the question).

These questions are not off-topic, but they seem to encourage lots of low-effort answers. I'm definitely not saying that all answers there are low-effort -- even the ones that don't try to show optimality can still be difficult to find if they're optimized well! But there are several answers that don't particularly try to be optimal (and even some that intentionally break the rules of the question by adding new operations).

Community Wiki: the best solution to the problem

So, we have a lot of answers that are not Answers, but still could hold some value. They often either give a good bound for the problem, or (as Adam helpfully points out in his answer above) a strategy that may lead forward. These answers are explicitly meant to be built upon by other users.

This sounds like the exact type of thing that Community Wiki answers are meant for. They have a much lower reputation barrier for edits, so people can more easily contribute to improving them. And using a CW answer has some other benefits too: less answer clutter in general, and an easy-to-find list of so-far-optimal answers. TheSimpliFire's answer on one of the two relevant questions is a good example of how this might look.

So, I think that when a question of this type is posted, there should be a CW answer made (ideally by the asker, but if they don't anyone else could make one). Answers that solely consist of "here's how you can do Y with only n Xs" should be edited into the CW answer with credit rather than posted separately. Answers that have (part of) a proof that a certain number is optimal can be posted separately as (Partial) Answers.

| |
$\endgroup$
  • 1
    $\begingroup$ This seems like a very sensible compromise, point taken. $\endgroup$ – hexomino Sep 13 '19 at 22:31
  • 2
    $\begingroup$ This is a nice idea in theory, but it would have a lot of downsides in practice if you start deleting all non-CW answers to such questions. Of course not everyone will be aware of any particular meta policy, so people are going to start posting "here's the best I've got" answers to establish bounds. If a mod shows up hours/days later to delete all those answers in favour of a single CW post, as happened this time, people are going to be rightfully annoyed - even if they edit their work into the CW, they lose whatever rep they gained, and some work won't end up part of the CW anyway. $\endgroup$ – Rand al'Thor Sep 14 '19 at 17:10
  • 2
    $\begingroup$ Also, I disagree that any answers are "invalidated" by later ones. A1 is no more invalidated by A2 than a partial answer solving half the clues is invalidated by a full one which solves all the clues. I think any answer which was the best at the time of posting is worth keeping, since that's always a contribution towards a bound on the optimum. Sometimes a series of results approaching the final answer tells a nice story! Answers which were not the best at the time of posting, I could go either way on - they're usually not valuable and could stand to be downvoted, maybe deleted. $\endgroup$ – Rand al'Thor Sep 14 '19 at 18:16
  • $\begingroup$ @Randal'Thor The answers "tell a story", maybe, but is that a good reason to keep them? I'm not sure about that. And if an answer is only the best at time of posting, but is no longer an answer, it's not helpful anymore -- because of the vastly different strategies required to get lower and lower bounds in these problems, there's no way you can really turn A1 into a full answer, and so it's not a partial answer anymore. $\endgroup$ – Deusovi Sep 14 '19 at 18:33
  • $\begingroup$ That's the issue I take with this: a partial answer should be both true and extendable into the full answer. A1 is true after A2 has been posted, but it's no longer extendable. And that seems to me to be a pretty big problem with A1 in the first place, if it can be un-Answerâ„¢-ified. $\endgroup$ – Deusovi Sep 14 '19 at 18:35
  • $\begingroup$ That being said, I do agree that mass deletions are a problem - that's why I've said what should be done, but not that it should be enforced strictly. I think this policy is probably best 'enforced' through community nudges: if you see a question like this, post a CW answer even if you aren't going to actually answer this. If you see someone post a non-CW answer that only gives an attempt, edit it into the CW and leave a comment directing them to this answer. With enough people aware that this is The Correct Thing To Do, hopefully deletion of that sort won't be necessary. $\endgroup$ – Deusovi Sep 14 '19 at 18:39
  • $\begingroup$ (I do worry about people intentionally posting their progress to get rep, even though they know it should be edited into the CW answer. But I think that's the type of thing that could be handled on a case-by-case basis by the moderators, just like any other "bad behavior from a specific user" issues.) $\endgroup$ – Deusovi Sep 14 '19 at 18:42
  • 1
    $\begingroup$ Sometimes - not always, but probably fairly often - one "best at the time of posting" answer might inspire another. In the example which inspired this meta, I noticed two occasions where an answer with $N$ eights was tweaked by another user to get an answer with $N-1$ eights. Similarly to bounding problems in mathematical research: sometimes the same method of proof can be used with a few tweaks to get a sharper estimate than the original one. $\endgroup$ – Rand al'Thor Sep 14 '19 at 19:14
  • 1
    $\begingroup$ ... which means that, at least in some cases, A1 might be "extendable into the full answer" as you desire. Also, continuing the analogy with "partial answers", sometimes a partial answer might seem correct, but (maybe due to a red herring, or some ambiguity in the solved part which is deambiguified by the still-unsolved part) it's actually barking up the wrong tree. Happens very often in the riddle tag, for instance. But wrong answers to riddles don't get deleted (or even, in many cases, downvoted) - they can still be a good attempt without being extendable into the correct answer. $\endgroup$ – Rand al'Thor Sep 14 '19 at 19:16
  • $\begingroup$ @Randal'Thor I don't see how just proposing a strategy that may or may not work is enough to count as an answer, though. That's exactly the type of thing I was talking about with the "double letter" example. I think that type of strategy isn't a partial answer, but a comment. And that doesn't change just because it's posted as "here's a way to get $N$ eights" rather than "here's a way that could potentially be used to get the actual minimum number of eights". $\endgroup$ – Deusovi Sep 16 '19 at 0:54
  • $\begingroup$ As for the "seems correct but barking up the wrong tree" example, I have no problem with those. The problem isn't "wrong [partial] answers that look correct but aren't", it's "wrong [partial] answers that there isn't significant reason to believe are correct". $\endgroup$ – Deusovi Sep 16 '19 at 0:54
  • $\begingroup$ (I think a flawed proof of optimality is a fine answer, even if it's not actually the optimal solution - that would be the analogous case to "an answer to a riddle that seems to explain most lines, but turns out to be wrong". The analogue to the answers in question here would be more like "I think the number of words in each sentence of this puzzle are strange - the answer is probably something to do with those". It may be correct, but it's not enough to be a full answer without some sort of confirmation that it's correct -- some sort of reason to believe it is.) $\endgroup$ – Deusovi Sep 16 '19 at 0:56
  • $\begingroup$ I think I'm on Rand's side rather than Deusovi's as regards answers of type A1. More generally, I don't like the idea that the only proper way to understand a question of the form "do such-and-such a difficult thing; the simpler/longer/bigger/smaller, the better" is as asking for a proof that one's done it as simply/... as possible. More often, I think, the fundamental challenge is to do the thing at all or reasonably; anyone who's done that has solved the puzzle; and the minimality/maximality/... criterion is just there in order to provide a way of picking a winner. $\endgroup$ – Gareth McCaughan Sep 18 '19 at 15:34
  • $\begingroup$ So, e.g., in questions like the one we're looking at here, there are two entirely separate challenges. First: form the number using a small number of 8s. Second: prove that a particular number can't be improved upon. Deusovi's preferred philosophy here privileges the second challenge, but I think the first is more what the question is about. If you want to argue that the second is more interesting or intellectually satisfying or something, I might agree -- I'm a mathematician, and I love proving things. But it really isn't what the question is about. $\endgroup$ – Gareth McCaughan Sep 18 '19 at 15:37
  • 1
    $\begingroup$ (I'm contemplating writing an answer to the previous question expressing my misgivings, for which a lengthy series of lengthy comments here is probably not the best medium :-).) $\endgroup$ – Gareth McCaughan Sep 18 '19 at 16:41
3
$\begingroup$

Optimisation answers without absolute proof = partial answers

*within reason


An answer to a typical optimisation question that doesn't prove with 100% certainty that better cannot be done is exactly the same as a partial answer to a non-optimisation puzzle.

Here are the similarities between these types of answer

  • They are both incomplete
  • Both may not have reached the stage which can invalidate them (i.e may not lead to the final answer)
  • They both exhibit direction which may lead to the final answer
  • Both are somewhat useful to other puzzlers
  • They are a display of progress and current knowledge

Due to these similarities, I propose that we treat them as equal so all queries should be resolved by referring to the discussion on partial answers here.

Admittedly though, this creates the rather difficult question - how do you quantify the specific value of an optimisation answer? Its not as if there is a clear "List-of-Clues" to value answers with.

Short answer: we do this on a case-by-case basis, there is no general rule which can 'calculate' the importance of an optimisation answer, however...

Long answer: an optimisation answer which doesn't show any indication of progress is not even partial. If it isn't possible to build upon the existing answer to lead to full proof and the answer in itself does not profoundly pave the way for another answer (e.g find restrictive bounds) then it is not even partial. Answers which meet the requirements for being an "alternative answer" are, however, exempt from this. The answer requires at the bare minimum some proof to even be considered as a partial answer. An answer must standout on it's own for it to at least be partial. Anything that these general rules cannot be applied to will require special consideration (e.g an answer which offers a radical new technique).

A small reminder to solvers: if you do not feel like your answer could standout on it's own in some regard then it should be posted as a comment

Enforcement


I propose that all optimisation answers which aren't at least partial should be deleted. Many optimisation questions tend to attract a lot of attention so I also propose that the question should be protected if it is likely to attract a lot of non-partial unfinished answers in order to protect the integrity of the question and current answers. If a question is packed with partial optimisation answers then the bar to qualify for a partial answer should be raised and all further additions under the bar should only be comments. As a preventative measure to stop a question from ever reaching the stage of getting packed with partial answers, the bar should be set in some way prior to answers, preferably by the original poster.

| |
$\endgroup$
  • 1
    $\begingroup$ In reference to the question - in question, all answers which were best at the time of posting should be left as is. Any answers which were inferior at the time and also didn't bring any extra value should be deleted. The question already made it clear that "the minimum amount of digits 8" was progress $\endgroup$ – Adam Sep 13 '19 at 12:32
  • 1
    $\begingroup$ I mostly like this, but two quibbles. 1. You say things like "not partial" when I take it you mean "not even a partial answer"; to me, "this answer is not partial" means "this answer is complete" rather than "this answer doesn't even clear the bar for being a partial answer". $\endgroup$ – Gareth McCaughan Sep 13 '19 at 13:53
  • $\begingroup$ 2. I think there are other ways in which an answer can be worthy of posting besides improving the known bounds and progressing towards a proof. E.g., on the question that motivated this, someone posted an answer that (I think) didn't improve any of the bounds but introduced an amusing new technique (huge long strings of percent signs). I don't know whether that counts as "profoundly paving the way" for something else but it seems worthy of posting. $\endgroup$ – Gareth McCaughan Sep 13 '19 at 13:55
  • $\begingroup$ (posting the two quibbles separately was deliberate since they are of very different kinds) $\endgroup$ – Gareth McCaughan Sep 13 '19 at 13:56
  • $\begingroup$ I agree with most of this, but I'm not sure it makes sense to say that answers' validity can depend on the order they're posted. I don't think "this is the best right now" is quite enough for something to stand on its own as an independent answer. $\endgroup$ – Deusovi Sep 13 '19 at 18:25
  • $\begingroup$ (I gave an example in a chat with Rand in The Sphinx's Lair: if the question is "find the 5 Scrabble-legal words that have the highest MD5 hash when concatenated", I don't think we would want to accept people just posting 'lucky guess' sets of five words with successively higher hashes. You'd want some form of proof or at least a general strategy for something to be An Answer. This is a ridiculous example, but IMO, optimization questions like this are very similar to this -- having one strategy for getting a certain amount often tells you nothing about how to (or whether you can) do better.) $\endgroup$ – Deusovi Sep 13 '19 at 18:27
  • $\begingroup$ @Deusovi One argument for allowing "this is the best right now" is that it's necessarily a new bound on the optimum. Of course that might allow people to post random guesses just because they get in early, but those would probably be downvoted if it's clear that they're random and nowhere near optimal. (I also agree with most of this answer, but am also considering posting my own if I have wifi over the weekend. Haven't read yours yet ...) $\endgroup$ – Rand al'Thor Sep 13 '19 at 18:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .