Currently the main site is used mostly by community who create/post puzzles and write answers to them. It is possible to use the site just to solve puzzles and to check solutions against existing answers, and some users including me are trying to use it this way. I believe great many users could potentially use it this way, much more users that currently using this site. However for the most of those potential users the puzzles are too difficult, so if they try, they are likely to give it up soon. Also the existing community fray upon easy puzzles. Also there were several suggestions on this meta to mark the puzzles by difficulty, which would benefit those potential users, but the suggestions were rejected by the existing community, because "difficulty is subjective" and "no one needs it".

I suggest one of three possible alternative solutions addressing the problem of difficulty. The first is better, the second is easier to implement, the 3rd is very easy to implement.

  1. Each answer contains two fields instead of one: "formal answer" and "full answer". Formal answer is for future computer-checking. Formal answer would normally be short for example "102.564" or "6 7 8" or "elephant". Another user who tries answering the puzzle later could test his guess of formal answer against the existing ones - there would be a box under the puzzle to do just that. A user could try multiple guesses. There also may to a button show the answers. After clicking show the answers one would not be able to try more guesses of the the formal answer. Hidden texts in the existing answers would be shown only after a correct guess of the formal answer or after clicking show the answers. All guesses would be anonymously recorded and statistics from that could be used to objectively calculate difficulty (% of people whose first guess was the correct one)

  2. We add one checkbox beside every existing answer [x] this was my first answer as well. Any user would be able to check this checkbox. Other users will not see, what checkbox this particular user checked, however the statistics would be available for everyone. The statistics would be used to estimate the difficulty of the question (% of people whose first answer was the correct one).

  3. Just introduce tags "easy", "intermediate", "hard" etc.

  • $\begingroup$ Related $\endgroup$ – HTM Oct 22 '19 at 3:45

There are several reasons I see that this is not likely to happen, and not worth the time and effort it would take to implement.

Technical problem

There's one main technical problem: We, the users [including the moderators], can't easily add features to the site.

Puzzling is part of the Stack Exchange network. We individually do not have control over the site design or functionality. Adding extra fields, buttons, or checkboxes is not likely to happen ever, considering that the people who would be in charge of adding those features are also in charge of adding features to over 170 other sites. Features that could ever only apply to one site are not likely to be considered -- at the very least, they would be very low priority.

So it immediately seems like none of these can work except the last one. But there are still problems, even if we could get around all the technical issues:

Practical problems

There are a bunch of practical problems with this idea too, though. The obvious one:

(a) Difficulty is subjective.

How do you know whether a puzzle is "easy", "medium", or "hard"? Puzzles can have vastly different difficulty levels for different people. I have a lot of experience with cryptic crosswords, so I would likely give some of my variety cryptics as "intermediate" -- though I'm sure other people would disagree! And I'm not very good at loop logic puzzles like Slitherlink, so I would overvalue the difficulty there.

This isn't even useful on a type-by-type basis: I have a lot more experience with a different loop logic puzzle genre called Cave. So I would still probably under-rank the difficulty of any Cave puzzles I made.

(b) Auto-calculating difficulty falls to selection bias.

Even if we could come up with some nice cutoffs for percentages, we wouldn't be able to figure out how many people tried the puzzle. A major problem with your methods 2 and 3 is selection bias: say there's a puzzle that is very intimidating -- to the point where only one person tries it, and gets the right answer. Does this mean that the puzzle is easy, since 100% of people got it?

You might instead propose that difficulty be calculated based off of number of people who viewed the question. This is a problem too: questions with more clickbait-y titles will have their difficulty rankings massively inflated.

(c) Many puzzles don't have formal answers.

What is the formal answer to the prisoners' names in boxes puzzle, or the blue eyes problem? These are both classic puzzles, and there are many like them, but there's no automated way to check an answer.

(d) "First guess is correct" is not a good way to measure difficulty.

Some puzzles are very self-confirming, to the point where someone's first guess is very likely correct. For example, cryptic crossword clues each have a built-in confirmation mechanism (since they have two routes to the solution, by design). So, every first guess should be the correct answer. Does this mean that the clues are easy? No, cryptic clues are notoriously difficult, especially for beginners! It just means that when you've solved one, you know you've solved it. (And this is a principle that all good puzzles follow: when you've solved a puzzle, you should be 100% sure that you have the solution.)

(e) Tags would not be applied retroactively.

If your third choice was implemented, the vast majority of puzzles would be untagged. We would either have to go back and tag a large number of puzzles for this to be worthwhile, or the un-difficulty-tagged puzzles would vastly outweight the difficulty-tagged ones for at least two or three years.

So, I'm sorry to say, this idea is both difficult to implement technically, and likely not useful even if we somehow got around the technical issues.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .