# Okay, the sandbox didn't work. We still have a quality problem; let's figure out how to address it

As of today, the riddle sandbox is no longer mandatory. The close reason related to it has been deactivated, and the requirement has been removed from the sandbox text. It was an experiment to see if it would be sustainable and effective going forward; for a variety of reasons many of you have pointed out, it didn't work. (We also screwed up at judging the amount of support an idea needs before it has consensus, and offering enough time for a thorough discussion of the merits of ideas.)

But that's fine; everything is reversible, and it gives us all a better sense of the capabilities of the system going forward.

The sandbox itself is probably still worth keeping around, though without the mandatory requirement, because as a tool it could still prove to be valuable to anyone who decides to use it. Even if it's never touched again, it does no harm just sitting there. Unless someone thinks of a good reason it shouldn't be there at all, this is probably what's going to happen.

However, this still leaves us with a quality problem. The idea of a sandbox requirement came about in response to a growing sense that riddle quality is dropping on the site, and that it's becoming more heavily populated with low quality content. (Number sequence and cipher puzzles were even tacked onto the proposal's discussion for similar problems.)

Most of what I said regarding quality in the riddle sandbox proposal I still stand by:

Riddle quality is dropping. I think most of us have seen it lately: there’s been a slow slide in effort and energy put into riddles, and it’s starting to seriously hurt the site. On Stack Exchange, our goal is to optimize for pearls, not sand, and right now, we’re very much not doing this. If we were, it would not only push the quality of the site up, but also drive us to advance the state of the art.

Nowhere else that I know of on the internet do people collaboratively come together to develop new puzzles - including riddles - and that’s not something we want to stop. However, we need to do something to sort out what makes a riddle high quality for this site, and set better quality standards.

So it’s time for us to set aside some energy and effort to sort this out, and start over with a better structure in place to support riddles.

This is the discussion that I want to see continue. I think most of us recognize that there is a problem, and Hugh Meyers even offered insight into why the proposal might have gotten the sort of initial support that it did, even the idea wasn't ultimately very good:

The massive support the proposal had in its first day shows the widespread recognition of the situation and a strong desire for some sort of a solution.

I don't want to belabor the point, rehashing stuff you've probably already read and read again. Instead, it's time for you all to drive the quality discussion on meta.

What we've been doing, bringing these proposals to meta, is partially intended to try and drive discussion. This is where these problems are solved, and there's definitely a solution out there; we just need to find it. (And yes, perhaps more carefully consider its implementation and subsequent effects before diving in head-first.)

So please, please propose and discuss ideas on meta. Don't let this issue stagnate; mods are three of hundreds of community members, and we're all going to get the site that we fight for together.

• Is there anyway to flag a question for low quality? – Areeb Aug 30 '16 at 19:18
• @Areeb You can flag a question as "very low quality" provided it's negatively scored. Better to flag it for closure though (or vote to close, if you have >3k rep). – Rand al'Thor Aug 30 '16 at 19:33
• Actually, I would like a non-mandatory sandbox for all types of puzzles - so that, if one is unsure whether the puzzle is good to post it there and receive feedback on what could be improved. – Maria Deleva Aug 30 '16 at 22:27
• It seems a bit early to declare that the sandbox didn't work. My feeling is that it did lead to an increase in riddle quality. (But that doesn't mean removing the requirement for its use is a bad idea; most of the objections people raised weren't really elaborations of "it doesn't work" or "it won't work".) – Gareth McCaughan Aug 30 '16 at 22:28
• Incidentally, do we get Hugh back now? – Gareth McCaughan Aug 30 '16 at 22:29
• @Gareth We don't. People come and go. I'm sorry to see any user leave, especially one like Hugh Meyers, but we can't decide for them whether they're going to stick around. – Aza Aug 31 '16 at 2:41
• (It sounds as if I gave the impression that I think the mods can somehow force Hugh to change his mind. Of course I don't think any such thing.) – Gareth McCaughan Aug 31 '16 at 9:15
• I really appreciate that nobody is stubborn here to stick with an idea that didn't work. One big thumbs up from me. I like to think that my protest had a little impact in this decision. But even if it did not, please let me think that. :) – Marius Aug 31 '16 at 12:18
• riddle VLQs again :(... – EKons Sep 1 '16 at 23:45
• I thought it was working very well, and +25/0 seemed like plenty of consensus to me. – A E Sep 2 '16 at 21:17
• Are you proposing a puzzle on how to filter better puzzles? How meta! – David Starkey Sep 6 '16 at 17:24
• Possible duplicate of Is it time for us to disallow challenge-only questions? – Gilles 'SO- stop being evil' Oct 1 '16 at 22:24
• @Gilles Um, no. This question is about what we can do now to improve puzzle/riddle quality; it's not a duplicate of a long-dead failed attempt to turn the site into something other than what it is today. – Rand al'Thor Oct 1 '16 at 22:28
• @randal'thor I know correlation isn't causation, but the quality problem described in this question started when this site became dominated by challenge questions (with riddles being among the worst). I've yet to see a better proposal than that one, answers here included. – Gilles 'SO- stop being evil' Oct 1 '16 at 22:32
• @Gilles You're entitled to your opinion, of course, but it does seem rather fruitless to try to change the scope of a thriving graduated site to the extent of declaring 99% of its questions off-topic. For better or worse (better IMO, worse IYO), this is mainly a site of puzzles now, rather than a collection of questions about puzzling - surely it's more constructive/useful to deal with it as it is than to pine for a rejected alternative? – Rand al'Thor Oct 1 '16 at 22:41

Totally agree with both of Rand's answers, however, I think we need to hone in a little on what the new close reason should be, so I thought I'd add my thoughts...

### New close reason

See Rand's answer for reasoning/precedent, however, I believe the text should be a little different. The primary aim should be to catch/close low quality content, which can be redirected to a sandbox where it can potentially be improved, so I propose the following close reason:

Low Quality1 - In it's current form, this puzzle does not meet Puzzling's [community quality standards]2 and needs refinement. Please take the time to read the guidelines and improve your puzzle. If you would like assistance, you can post your puzzle to the [Puzzling Sandbox]3 to get guidance from the community.

1 - Could possibly be renamed to Needs Refinement or something equivalent, to make it slightly less inflammatory/accusatory - we don't want to scare off innocent newbies.
2 - Would link to the meta post described in point 3, below.
3 - Would link to the either a new general purpose sandbox for all puzzle types, or could possibly be reworded to say "...post your puzzle to the [Riddle Sandbox], [Cipher Sandbox] or [General Puzzle Sandbox]...", if we wanted things split up into chunks.

### Community Quality Guides

To keep the close criteria as objective and consistent as possible, we could create an tagged meta post, with individual answers per "major" tag (i.e. the big ones, that are currently problematic/controversial like , , , etc), plus a "general puzzle" catch-all, to help define exactly what constitutes the quality standard minimum, in as clear and unambiguous terms as possible.

We would obviously need individual posts to gather consensus on criteria for each puzzle type above, but there's already some great content here on meta that we could draw from (eg. for ciphers, mathematics, etc). To give you an idea though, I imagine each "answer" would look something like the example below.

Indicative example of a Community Quality Guide, take with a grain of salt... the important stuff of this post is above.

Riddle Quality Criteria

To meet minimum quality standards for Puzzling, a must:

• Have a single "obviously correct" answer
• Be more than just a straight description of the solution's features
• Describe a common everyday object/concept (or be tagged with an appropriate secondary tag, such as , , , etc to identify that specific domain knowledge is required)

Additionally, your riddle must meet at least three of the following criteria:

• Use well structured meter and rhyme
• Be concise and well written
• Use a creative/unique structure or presentation
• Employ letter/wordplay
• Make use of metaphor/polysemy/turns of phrase
• I like this close reason. – IAmInPLS Aug 31 '16 at 9:02
• Yes, +1 for your suggested close reason (though I do agree with the first footnote - we need to think about how these things look to someone who's never been on SE before) and for the idea of a faq post with separate answers for each tag. I bet it'll take a looooong time to get consensus on a proper definition for "low quality" though :-) Especially if you want to define criteria for low-quality puzzles in general, not just tag by tag - that one might actually be impossible. – Rand al'Thor Aug 31 '16 at 13:55
• @randal'thor - thing is we don't need 100% consensus, just a minimum bar. I think using a "must meet $x$ of $y$ criteria" gives flexibility so that the definition need not be absolute. And I think you'd get pretty unanimous agreement on a minimum bar. For example I doubt people would complain about the "must" list in my riddle example, above. The "at least $x$" list is probably reasonably ok already, even if you reduced the $x$ to two... wouldn't be as tight a close reason, but would still be better than currently. – Alconja Aug 31 '16 at 22:34
• @Alconja Sure, we don't need 100% consensus, but we also have to be careful not to rush into anything too quickly, as just happened with the sandbox. (There are several good posts here on PSE about how to write good riddles; I'll gather them together some time and maybe put a list of links in a comment here.) – Rand al'Thor Aug 31 '16 at 22:38
• @randal'thor - totally agree with that sentiment, which is why i was suggesting a discussion post for each tag to decide on what the minimum bar should entail. (hell, possibly even a meta post, with a chat room, then a second meta post with the proposed aggregated wording). – Alconja Aug 31 '16 at 22:48
• How about putting quality guidelines in large print at the top of the "Ask a Question" page? Or at least something brief on "Rules For Riddles" and a link. Make people look at it before they post. – Hugh Meyers Sep 2 '16 at 14:54
• with regards to the single "obviously correct" answer, I humbly request that we, as a community, please try to be respectful toward people who aren't experts in all fields. Someone might not know that their riddle also has a different answer when interpreted as a riddle in the topic of Eastern European Aeronautical Engineering, or Baroque Music, or Programming in FORTRAN, or Russian Ethnic Minority Comic Books of the 1960's, or... – Kingrames Sep 8 '16 at 14:54
• @Kingrames - that's why it says obviously correct (though perhaps it needs more clarity). Even if there turns out to be multiple technically correct solutions, it should be obvious which one was the intended one. Eg. Even though "a table that gets broken, and then partially repaired" technically fits the riddle of the sphinx, "man" is still clearly/obviously the better (and intended) solution. – Alconja Sep 8 '16 at 20:51
• Yeah, I wouldn't have added that except that there are a lot of people that come up with those kinds of answers and then insist that it works, so the question is wrong. Last time I was seriously active on this site, every single riddle question ended up with a flood of answers like it. – Kingrames Sep 13 '16 at 22:39

This may not make a huge difference, but since new users are often high on the blame list for the poor-quality stuff, we should

# change the Puzzling Tour to actually reflect this site.

I think this is probably a low-effort but some-reward step we could take, so why not do it. If you don't think it needs to be changed, go take the Puzzling Tour right now and try not to furrow your brow.

At least some of the recent poor quality questions (example, example, example) come from users who do have the Informed badge. That means at least there's one potential point of intervention to communicate something to new users. Right now, nothing Puzzling-specific is on that Tour, and in fact you could argue some of the stuff is Puzzling-detrimental.

And I think Emrakul indicated (as a comment on this post) that we do have some control over this. So...why not?

• That's weird. I'm sure the Tour page used to have an actual Puzzling question cited as its example question (specifically, this one, which explained why I kept getting drive-by upvotes on it) rather than that unicorn/daisies crap. To find out which questions are potential candidates, see this main meta post. Sadly a lot of great Puzzling posts would be disqualified because of spoilertags. – Rand al'Thor Aug 31 '16 at 17:11
• Aha, and now I realise that the question which used to be on the Tour became an invalid example question on 1st June because someone added spoilertags to the accepted answer! – Rand al'Thor Aug 31 '16 at 17:13
• Is there a reason that question looks like it was stolen from Worldbuilding? – Kevin Sep 21 '16 at 5:42
• @Kevin If it is, time travel was involved. – Gilles 'SO- stop being evil' Oct 1 '16 at 22:18

## A chatroom for detecting and discussing low-quality puzzles.

Whether or not we implement a new custom close reason for 'bad' puzzles as proposed in my other answer here, we need people to be aware of it and of any new puzzles coming in that should be closed using it, otherwise there's no point. Emrakul mentioned in comments on another meta post that part of the reason why moderators have been closing too many questions unilaterally is because the community haven't been active enough in closing questions which should be closed.

I propose creating a dedicated chatroom for crap-catching. It would have a feed to post new questions into the room, and in it people would discuss the possible closure of particular questions. This would:

• be easier to keep a constant eye on than the Close Votes review queue, since people could hang out and chat idly in between discussing questions to be closed
• encourage active community discussion rather than just hitting the "Close" or "Leave Open" button, which would be helpful in shaping policies for the future
• draw people's attention to questions in need of closing quicker than the review queue does, since no initial VTC would be needed - all new questions would appear on the room feed.

For the first while, when not many people frequent the chatroom, we might find that more or less the same bunch of users are closing many of the questions, but as we attract more and more participants to the room and the project, that should change. Note that users with <3k rep would also be welcome to join in: even though they can't actually vote to close questions, they can still flag them for closure and take part in the discussions around them.

Of course we'd have to publicise this room as much as possible to make people aware of its existence. Ways to do this might include a featured meta post linking to it and superpings from any chat mods involved to bring active community members into the room.

• Another way to advertise it would be to make a puzzle with it as the answer :P – dcfyj Sep 1 '16 at 18:05
• @dcfyj Yesssss! – Rand al'Thor Sep 1 '16 at 18:09

## A custom close reason for 'bad' riddles/puzzles.

Normally, the way to deal with 'bad' questions on Stack Exchange is simply to downvote them, while closing is reserved for questions which are off-topic, opinion-based, or otherwise inappropriate for the site. However, certain sites which have a consistent problem with floods of low-quality questions have developed a custom close reason to use for such questions, in order to get them shut down quickly before they can gather multiple answers and reach HNQs.

• Movies & TV is plagued with low-quality ID questions: the OP remembers some details about a film they once saw and asks if anyone can identify it. While ID questions as a whole are on-topic, the M&TV community have implemented a close reason which reads as follows:

Identification questions must contain sufficient detail to meet the site's quality standards and should not be about a commercial or music video. For help writing a good identification question, see: Identify-This-X Questions.

Thus, 'good' ID questions which include sufficient details about the film in question are fine, but 'bad' ones which say little more than "I saw a film when I was younger, it had an alien and a dog in it, please help me find it" can be quickly closed by the community.

• English Language & Usage is plagued with questions that are easily answerable by simply using a dictionary. In order to discourage such 'trivial' questions and focus on questions which require some expertise to answer, the ELU community have implemented the following close reason:

Please include the research you've done, or consider if your question suits our English Language Learners site better. Questions that can be answered using commonly-available references are off-topic.

Again, questions closed with this reason would be considered on-topic by network-wide standards: they're about English language and usage, they're objectively answerable, but they're just 'bad'. (For an example of the kind of question which would be shut down instantly today, see this historically locked post.)

We can do something like this at Puzzling. Technically, since our site scope covers puzzles posted as questions, we shouldn't close anything as off-topic if it's a puzzle? Not necessarily! If we get enough community consensus to do so, it will be possible to create a new custom close reason for 'bad' or low-quality riddles, provided we can define some sort of criteria for when it should be used.

My initial proposal for a criterion for a 'bad' puzzle would be that the intended solution fits the puzzle no better than one or more other possible solutions. This would cover "guess what I'm thinking" type puzzles, but I'm aware that there may be other categories of 'bad' puzzles which don't come under this header. Everyone, please do feel free to make amendments to this criterion, either by leaving comments here or by coming to discuss it in chat.

Before anyone points out that a couple of existing close reasons could be used for this: yes, I realise that puzzles of the type I'm talking about would also fall under "too broad", or under this existing close reason. The advantages of making a new one are:

• we can try to pinpoint as accurately as possible exactly what the problem is with the puzzles we consider 'bad', less generically than the existing close reasons
• we can make our close reasons more up-to-date, according to what the Puzzling site is today rather than what it was 2 years ago at the time of the meta post I linked above for the current close reason
• we can and should include a link to the sandbox in the close-reason text, so that people whose puzzles are closed for being 'bad' can go directly to a place where they can get helpful feedback, improve their puzzle, and eventually perhaps get it reopened.

Of course, we still need plenty of discussion before deciding exactly what such a close reason should say, and then we need time to get community consensus before actually implementing it. But in the long run, I think this may be the best solution.

• This is post number 5373 on meta.PSE, and my user ID is 5373. I now declare this post my defining moment on the site :-P – Rand al'Thor Aug 30 '16 at 20:13
• A new close reason is a great idea, although as I mentioned in chat, I don't think "the intended solution fits the puzzle no better than one or more other possible solutions" is sufficient, because, A) it's (almost) synonymous with "too broad", and B) it allows crappy puzzles through that do have specific, obviously correct answers. Personally, I think people are already doing a reasonable job of closing crap as "too broad", but there's lots of stuff in category B (especially sequence and cipher tags) that can't be closed validly. I'll post a separate answer with an alternative... – Alconja Aug 31 '16 at 6:41

I mentioned this in a comment but I think it ought to go in as an answer. If we have requirements and standards (as in the some of the suggestions for the tutorial and help center) they really ought to go in a prominent place on the "Ask Question" page right above where the Title box is. The yellow rectangle on the right is not good enough. That's where annoying banner advertising goes on most sites and my eyes automatically screen that out most of the time.

Here's what we say (in the yellow box):

We prefer questions that can be answered, not just discussed.

visit the help center » asking help »

Our top problem (I believe) is not that people are trying to start discussions, or that they are posting on topics other than the creation and solving of puzzles. We are concerned about puzzle quality. We should say so. If we have rules concerning the quality or nature of riddles, we should say so and we should say so here. This is the last stop before the submission goes onto the site. After it is posted then it needs to be read, discussed, voted on, and closed. Surely it is much better to do what we can to make sure that unacceptable content is not posted in the first place.

If the problem is over-eager newcomers posting unacceptable content, the help center and tutorial will reach only a small fraction of this segment. Our best chance is to put our message in big letters where they can't help but see it before they post.

RULES FOR RIDDLES

or

MINIMUM POSTING STANDARDS

followed by a short, readable list with links. In short we should do all we can to make the poster the crap filter rather than the already burdened mods and the community at large.

I there are a lot of other good suggestions here. My point is just that if the problem is a lot of crap on the site we should at a minimum be telling people in a loud voice not to post crap and exactly what we consider to be crap.

• Have you seen this question I posted a couple of days ago? – Rand al'Thor Sep 6 '16 at 17:12
• An even better way to achieve what you're looking for, specific to the riddle tag, would be to get an automatic popup reminding riddle posters about the quality rules for riddles (see here for what this would look like). This would probably be more likely to actually be read than the more static "How to Ask" box or help centre. – Rand al'Thor Sep 6 '16 at 17:15
• @randal'thor I agree the popup is better. I saw the other question and thought about posting there but decided that the quality issues are more top of mind and better addressed here. I agree the yellow box text should be changed but I don't believe it shouts loudly enough to make a difference to question quality. – Hugh Meyers Sep 6 '16 at 18:37

## Text building blocks

How about making a common list of responses that can be used to improve posted riddles? Mostly the riddles aren't bad, just need some more work. Instead of closing them immediately or discouraging to post, let's help the users to improve the question posting a comment. The text building blocks will be there to help with it.

I'm thinking of a list of customisable comments similar to this one in TeX StackExchange.

This doesn't exclude the proposal in Hugh Meyers' answer and is even complementary to it, engaging the problem in similar positive manner.

I've kind of brought this up before, but speaking as someone who is a content creator in the puzzle world, I am actively disincentivized when it comes to posting here. I think it must be some larger site policy for stack exchange, but if I post something here it legally means I give up ownership of it. I'm trying to start a puzzle magazine but I can't even show you guys samples here because it actually hurts me to do so. Now I don't claim to be the best puzzler of all time, but I try to make unique games and hire artists to flesh everything out and bring them on par with newspaper puzzles so it's not just some chunk of text. In my opinion, this is the sort of stuff that should be making its way here to you guys. If you want to create an influx of decent to good quality stuff, you need to create an environment conducive to bringing in people who are trying to make a living off of puzzling.

• To be clear, you don't give up ownership of your content. It's still yours, however you grant SE (and by extension everyone else) a license to the content under the Creative Commons Attribution Share Alike license. – Alconja Sep 1 '16 at 13:05
• ...and yes, CC BY-SA is core to the philosophy of Stack Exchange, and I highly doubt it'll be changing any time soon. – Alconja Sep 1 '16 at 13:10
• I don't give up ownership of the specific content, but I give up ownership of the IP which is something I am simply not prepared to do right now. – Topple Sep 1 '16 at 13:12
• Again, you don't give up ownership of anything. You can still do whatever you want with it, commercial or otherwise and it will always have your name on it. You just can't stop others sharing/using it too (while giving you credit for it). That being said, I do understand what you're saying, and the breadth of the license is, for practical purposes, pretty close to giving up ownership. – Alconja Sep 1 '16 at 13:19
• Yes, it depends on what exactly your concerns are. If you want to publish your puzzles in your magazine, you may still do so, even after posting them to this site. If you're concerned that you will no longer be able to publish your magazine, that's not a worry. But if you are concerned that others will also be allowed to publish your puzzle, that's where posting here doesn't make sense, because after you've posted it here, theoretically anyone can publish your puzzle in a magazine, as long as they attribute you. – GentlePurpleRain Sep 1 '16 at 14:17
• The CC-by-SA licence has three nice and friendly terms in its name ("creative", "commons" and "share") but it's like a dream come true for companies that want to make money out of other people's work without paying them. – h34 Sep 2 '16 at 3:04
• After posting it here, about the only thing that the author of a puzzle is allowed to do with it that everyone else isn't allowed to do with it - or in other words, about the only right reserved to the owner - is to publish it elsewhere without attributing it to himself! @Topple - good luck with your magazine! – h34 Sep 2 '16 at 3:09
• Thanks h34! To my understanding I can put an ad up for it through approved channels here and so you guys will be the first to know when it launches. – Topple Sep 2 '16 at 11:57
• Maybe puzzles may be first published in some magazine using more restrictive (but non-exclusive) licence, then (say, after 1 year) published here under CC-BY-SA. – Vi. Sep 10 '16 at 16:39

# Riddle Restrictions

This is just an idea and I'm not sure if it will be possible or will work, but its worth bringing up the idea.

Users on Stack Exchange can only use 40 votes in a day, after that it restricts you.

So can we do the same for riddles? Once a user has posted a question tagged , they can't post another one for 24 hours.

This may require code changes to the site so may not be possible.

# New or Edited Help Center Page

This will probably work better. The help center contains the question

Maybe it would be a good idea to add there something about riddle and give a few guidelines.

Alternatively we could create a new help center page specifically for riddles. The question could be something along the lines of:

• How do I ask riddles?

• How can I write a good riddle?

• Should I post riddles here?

• What sort of riddles can I post?

The page can give advice on riddles, this will hopefully raise riddle quality to an acceptable level

• Re restrictions: this would be hard to enforce (without code changes which SE probably wouldn't want to do) and easy to circumvent (just post a riddle without the riddle tag). Re editing the help centre: this is definitely a good idea, and easy to do: mods can edit certain parts of the help centre on their own, once consensus has been reached on meta. So +1 for the second half of your answer. – Rand al'Thor Sep 3 '16 at 13:42

I had a crazy idea that, I hope, might inspire someone to propose a less crazy idea. I propose a "workshopping" tag (or perhaps a "sandbox" tag).

If a puzzle is tagged "workshopping", it means that it is a work in progress. People won't (or shouldn't) get mad if the question changes. I have seen users post questions that have problems and then get flayed for "totally changing" (i.e. attempting to fix) the question in response to criticism. To me, this feels much more friendly than closing or not allowing people to post unless certain criteria are met. I know that I have a few puzzles (I'm thinking of Alice In Cryptic in particular) that I would like to fix but I don't know how to do it without partially plagiarizing myself or changing a question with an accepted answer.

This tag would be automatically applied under appropriate conditions. Maybe it is applied to every question by default. Maybe it is applied by default if your reputation is below a certain level. I believe some sort of automatic application is important because a lot of problem content is posted by people who are not familiar with the way the site works, the available tags, and posting standards. My suggestion would be to let users remove it if they want to but to include it by default to get them to stop and think.

If people don't want to see "the chaff" then they can exclude that tag. Maybe it is excluded from common searches by default.

A question with the "workshopping" tag would be automatically deleted after some time period under the assumption that the poster must have given up on it. I believe there is a lot of "fire and forget" low-quality content so an automatic deletion process has a lot of appeal. I don't believe we can actually stop bad content from happening but perhaps we can make it ignorable and transient.

Maybe it would be nice if there was an option to reset votes to zero (and hence erase rep loss) when the tag comes off. (Probably not practical and I'm not even 100% sure it's a good idea.)

How is this different from the sandbox? Well, chiefly, it feels different to me. We are not saying "your content is not welcome on our site." We are giving people a chance to post their content in a safe way. You aren't trying to earn your way onto the main site. There is no "dual posting" with the problems associated with users seeing the question on meta before it goes to the main site. My hope is that when good questions are posted, the users will take the tag off fairly quickly and that bad questions will be abandoned and deleted. Along the way, maybe we can salvage some questions that had problems and alienate fewer novice puzzle makers.

There are a number of problems with this idea. We would somehow have to let new users know what this tag signifies. We would need to communicate to existing users the etiquette regarding "workshopping". I don't have anything clever to say about what happens when people answer "workshopping" questions. If you accept an answer, does that mean that the question is no longer in workshop mode? It is also a meta-tag and, as such, explicitly discouraged for good reason. I would say in defense that puzzling se is for the creation and solving of puzzles. This tag says that the poster is (for the moment anyway) interested in puzzle creation and is asking for feedback.

Again, I am mostly hoping that this might spark someone to have a better idea.

• I think this could happen without a new tag. We already have puzzle-creation, and all a user would have to do is phrase their question as a question about a puzzle, instead of just a puzzle. e.g. "I created this riddle, but I think it might be too easy. How can I obfuscate the solution a little more?" (or any manner of other things) – GentlePurpleRain Sep 1 '16 at 14:13
• @GentlePurpleRain Sure, that works. I realized I had mostly edited out an important part which was that the tag would be applied automatically (but would be removable by the poster) because I believe that in many cases the problem is posters who would not know to search for such a tag and might not think off the bat that their puzzle might have a problem. – Hugh Meyers Sep 1 '16 at 14:32

Any time I read "standards are dropping", I hear "in my day, we had to walk ten miles to school, uphill both ways...". If you've heard most classical riddles and become accustomed to riddler thinking, then it's very likely that you will feel the standards have dropped from when you were new, and the riddles seemed fresh, new and exciting.

Simply put, I don't believe your claims. You should not be doing anything at all (you should not even have made this post!) until you can prove that something needs to be done. Opinion is simply not sufficient grounds for the formation of new policies.

What statistics have been gathered to support the argument that riddle quality is dropping?

What metrics could be used?

As an example metric, has the average ratio of views to upvotes risen or fallen over time?

TL;DR: You seem to be tilting at a windmills. Show us the data.

My suggestion:

Allow a user to post only one riddle per fortnight*, unless their previous riddle has been solved and received a vote score of at least 15* (in which case they can post them earlier). If the rule is violated, place the new puzzle(s) on hold for the rest of the period.
(* Exact period and score values TBD.)

I think a major source of the problem is new enthusiastic users who get carried away.

1. This approach does not directly stop them from posting, nor does it directly describe their content as not being good enough for the site. I think it more objectively reflects the intentions of the community and hints at the problems that are being caused by casual puzzle creation.
2. The rule encourages questioners to prefer quality over quantity. A person with 3 half-baked ideas will now be inclined to stop and think about which their best idea is first, and try to make it better so that they have more of a chance to post their next one soon. In the process, they automatically start thinking about how to polish their act.
3. If they fail, they can get feedback in comments and/or opt for the sandbox before their next attempt. Alternatively, they can start thinking of posting a different type of puzzle.
4. A low-quality riddle can still be posted every fortnight, but the perception (hopefully) changes from "I'm not being given a chance" to "I need to can do better than this".
5. This can be applied to any dominating / mass-producible type of puzzle that seems to be overrunning the site at any time, and not necessarily just to riddles.

This doesn't require site changes or moderator involvement to work: community members can VTC errant puzzles and that too in a completely objective manner. The subjective feedback is left to the general PSE community and is not restricted to the Meta community or specific reviewers. Proactive community members can give a heads-up and advice to new users in the comments of their first puzzle as soon as it is posted.

I know this still leaves open holes for a few things like sock-puppets, but I feel that a large portion of the users causing this problem are really just honest and enthusiastic but casual puzzle creators - who are either really not good enough and will get restrained, or those that can do much better once they realize that they ought to.

• -1. This also restricts users who want to post hard riddles, and makes easy riddles more likely to go through. We want to optimize for quality, not easiness. – Deusovi Sep 5 '16 at 0:17
• @Deusovi: I don't quite understand. Why do we not also want to restrict hard riddles if they are not being well-received? The restriction is lifted after a fortnight irrespective of how the puzzle fared and even if it wasn't solved. I don't see the link to easiness instead of quality (assuming the votes reflect quality). Would it be different if the requirement for the riddle to be solved was removed, or the period shortened to, say, a week? – KeyboardWielder Sep 5 '16 at 7:12
• -1: this restricts quantity without any guarantee of affecting quality. Ideally we want a huge flood of really good riddles, not a slow trickle of the same old crap. – Rand al'Thor Sep 6 '16 at 17:10
• @randal'thor: I doubt that any solution exists that would give us a "huge flood" of good riddles. More realistically, we can try to optimize the ratio of good:bad. This suggestion attempts to do that by restricting the quantity of only poorly-received puzzles, not well-received ones. Also see points 2-4 above. But yes, no guarantee. :) – KeyboardWielder Sep 6 '16 at 19:31
• @KeyboardWielder If we find a solution for weeding out bad riddles and then encourage traffic to increase, we should eventually end up with a flood of good ones. Your suggestion here involves reducing traffic, which (IMO) would be shooting ourselves in the foot. – Rand al'Thor Sep 6 '16 at 19:34

Can't you form a committee of riddle experts on Puzzling StackExchange - maybe chosen 5-10-20 people, who review every riddle and vote/discuss whether it is good enough for the website? You may argue that the opinion of so few users may not be representative for the mass, but isn't that the main issue here - the mass is often voting up simple, lower quality puzzles.

The riddles which do not pass the selection process can be still kept in the sandbox (or other similar place), along with explanation what is wrong and what can be improved on them.

• I think this kind of oligarchical review system would be against the principles of SE. – Rand al'Thor Sep 3 '16 at 14:49
• @randal'thor I think it will let more inexperienced riddle makers learn from the more advanced puzzle creators, and this would help them improve their skills. I agree the system wouldn't work well for most of other types of puzzles on Puzzling, but believe it will be good for riddles particularly. At the end of the day, it is mostly 30 people here who express unhappiness with the quality, the rest seem satisfied, judging by the votes. – Puzzle Prime Sep 3 '16 at 15:38
• As with the sandbox itself, I'm not particularly against the notion of a committee of 'experts' discussing each new riddle. What I really don't like, in both cases, is to make the process mandatory before a riddle can be posted on the main site. Many (albeit not all) of the objections I raised here would still apply to this proposal. – Rand al'Thor Sep 3 '16 at 16:09
• Know what? I actually don't think this is a bad idea. $(+1)$ – Mr Pie Sep 13 '18 at 4:52