11
$\begingroup$

Recently we've been getting multiple questions asking for answers to pattern-spotting questions from external sources. Many of them are in the form of a simple (e.g. given some numbers, predict the next number) or (e.g. given a matrix of pictures with one picture missing, choose the best one for the missing picture).

The problem is that, in many such cases, no one (including the OP and the answerers) knows either the correct answer (because e.g. it was from an online test), or the intended explanation of the original creator (≠ OP) of the problem. Also, many online sources of such problems (and even some books) are not very high-quality, giving possibility of multiple equally probable answers (as in here) or, even worse, expecting a counter-intuitive answer when an objectively better pattern is visible (as in here). Even if we come up with a seemingly clear pattern and no one on this site disagrees (therefore it looks like a good question), no one can be really sure that it is the intended answer/explanation by the original creator.

As I answered multiple times, all these problems boil down to a guessing game between us and the original creator, and we won't ever get to the intended solution/pattern/whatever because we can't get any feedback from the original creator. (Contrast this to the case when the OP has actually created the question: we can at least know which pattern is intended and which is not.) We could answer those with our best possible guesses, but they're inherently speculative and cannot be judged as correct or not. We even have a community-specific close reason for this: a question inviting speculative answers.

In conclusion, I propose to define "pattern-spotting questions from other sources" as off-topic, and close those questions using the reason mentioned above.

$\endgroup$
  • 3
    $\begingroup$ Very related, from way back in 2014. Personally, I do agree that many of these questions are of rather poor quality, but I don't see a reason why they should be considered off-topic outright. Most of the time, these questions have one justifiably correct answer (key word being "justifiably"), and the proper citation and speculative close reasons can handle the ones that are truly up in the air. But if the community thinks we shouldn't allow them, then I won't argue too much about it $\endgroup$ – HTM Nov 17 '20 at 8:50
  • $\begingroup$ @HTM I decided to post this (after this rant) to see how others think about this. Interestingly, at least to me, the answer at the link seems to support my argument: "The problem is whether an answer can be determined or proven to be correct". The questions considered here fail to meet the condition, at least if we interpret "correct" as "matching the intent of the original creator". $\endgroup$ – Bubbler Nov 17 '20 at 9:22
  • $\begingroup$ See also: Banning "IQ test" questions $\endgroup$ – Rubio♦ Nov 28 '20 at 17:30
  • $\begingroup$ Any updates on this? Is there enough consensus to declare these puzzles off-topic? $\endgroup$ – bobble Dec 5 '20 at 19:10
7
$\begingroup$

I'm not sure that the possibility of multiple equally likely answers is enough to declare a question definitively off-topic. There are sometimes answers that are very clearly correct -- in questions that do satisfy this, there's no reason to declare them off-topic a priori.

But... I do see some problems with them that make me believe: yes, we should declare IQ-test-style "find the next entry in this sequence" questions as off-topic, at least those from other sources.

Do new IQ-test-style questions add value?

Let's start with the obvious. I do think that they are consistently some of the lowest-quality questions we get, even when they are technically on-topic. In our site's chat, there are almost always groans whenever one of them appears in the question feed.

Not allowing future questions would significantly boost question quality overall.

But there's another issue that I believe we haven't considered:

Do old IQ-test-style questions add value?

Part of our site's general ethos -- the way I see it, at least, is to be an archive of both puzzles and questions about puzzles, in hopes that they will be helpful to other people.

The puzzle questions "help" in a different way than much of the SE network: puzzle questions serve more as examples for puzzle design and explanations of how solvers approach things, rather than direct answers to questions.

But I still believe that old puzzles like this can be useful for puzzle-crafters looking to learn how to design more effectively, or solvers looking to learn the ins and outs of specific puzzle types. I've recommended old and puzzles here to novice solvers before, and I've even used questions here to practice certain genres myself.

Plus, the site has been useful for people looking to understand the explanations to puzzles they found somewhere else. We have lots of old, well-known questions that get a lot of visitors looking to understand answers - particularly the detailed explanations of the Blue Eyes problem and the "64=65" rectangle dissection.

IQ test questions do not serve this purpose:

  • They frequently cannot be answered with certainty, making them useless for solvers.
  • The thought process required is almost always "guess what random manipulations the author is thinking of", making them useless for creators.
  • They are practically impossible to search for, making them useless for people trying to find answers. (I have seen multiple instances of IQ-test-type questions that were definitely duplicates, but I had absolutely no luck finding the originals. Even if the source is given, you still have to search through every single post with that source, because there's no nice way to search for the images used.)

Because of this, these questions have no lasting value after they are answered.

Learning from other sites

So, I looked on other sites to see if there was any precedent for something like this. And it turns out there is: several sites based around fiction have had problems with "identify this book/movie/game" questions. These seem to me to be a very good parallel to our IQ test questions: they are common, often low-quality, and provide very little lasting value after they're answered.

So, how did other sites on the network deal with these?

Arqade has strict rules for game ID questions. In particular, they need an "artifact" - some piece of tangible media that would let them confirm, without any doubt, that the answer they give is correct. Otherwise, it is off-topic.

Anime & Manga has also had to implement tight restrictions on ID questions.

Movies & TV has banned ID questions entirely, after a lot of debate. This appears to be a controversial decision, but several of the answers to the original discussion post detail some of the reasons they have hurt the site -- in particular, Napoleon Wilson's answer goes into great detail on this harm, with some helpful quotes from SE employees.¹

Literature seems to be allowing them. Rand al'Thor explains that this is (in part) because book ID questions let the askers provide lots of details. He also notes that Sci-Fi & Fantasy allows ID questions, and they get a lot of comments from people looking for the same thing. In my memory, we have never had someone comment that they were looking for the same answer to an IQ test question; I did a brief search modifying Rand's search mentioned in that post, and it gives no results.

We cannot get 100% correct verification on whether reasoning for an IQ test question from another source is correct, so the 'escape hatch' Arqade allowed would be impossible for us. Because of this precedent and the lack of usefulness to later visitors, I believe that we should close all pattern questions from other sources as off-topic.

Footnotes:

[1] Some quotes from SE employees given in this answer:

Identification [IQ test] questions are like... The cabbage, radishes and shredded carrots that some sushi shops put on the platter before stacking up the meat & rice. They make the plate look nice and full, and they're something to chew on once you've eaten all the meat... But you still gotta have plenty of meat ...If this site gets a tiny handful of ID [IQ test] questions, just enough to fill in the gaps between more interesting and useful questions, they'll probably be fine. But if a majority of questions fall into this category, it becomes quite unlikely that the site will survive. And, let's be honest: if the site takes off, y'all will want to get rid of ID [IQ test] questions anyway simply because when there's plenty of meat they're a distraction, keeping them around doesn't help anyone, and the folks who ask them don't stick around to help anyone else.

-Shog9, lightly edited by me

The problem is they are easy to ask, but they ultimately help exactly one person, and then they're useless. It gets tiresome, and drives away avid users who drive this site. They will continue to pervade the question space. And then they wear down a community. ...It's really hard to articulate as to why these questions are not good for this site; but suffice it to say that, once they permeate the front page, a few users will work long and hard to try and get rid of them. But the masses will keep them coming, inexorably...In a not-too-distant future, most of those questions will be asked by hit-and-run users who will never return to this site. And you'll get bored having to tease out a decent question and provide answers to a post that will not add one lick of value or interest to this site.

-Robert Cartaino

$\endgroup$
  • $\begingroup$ How would you propose implementing this? A new custom close reason? Would you want to apply the policy retroactively? $\endgroup$ – bobble Nov 26 '20 at 4:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .