10
$\begingroup$

I'm concerned by what seems to be a rise in low-quality puzzles posted in the last month or so.

Here's a graph of the percent of questions whose net score is negative or 0, by week, since Puzzling went public in May 2014:

enter image description here

(Thanks to Emrakul and Gilles for the data.)

Note the upward spike all the way on the right side, representing roughly the last month. Recently, more than 1/3 of questions have scores ≤0. That's the highest it's been except for a week in Sep 2014, and it comes after a period of relative quality.

This could also be a fluctuation on the historical scale, or reflect a change in voting. But I had myself gotten the impression of way more poor puzzles being posted than before. And I do think the downvoting is helping; it's the underlying problem that concerns me, not the symptom of downvotes.

What is the reason for all these low-voted posts? What can we do about it?

$\endgroup$
4
  • 2
    $\begingroup$ I am curious how this count shifts if rebus puzzles are excluded. $\endgroup$
    – user20
    Mar 8, 2015 at 9:19
  • 2
    $\begingroup$ I don't see a problem here - I'm expecting it to self-correct back towards the mean pretty soon. If it doesn't, then we have a problem. $\endgroup$
    – A E
    Mar 9, 2015 at 16:51
  • 1
    $\begingroup$ A factor you aren't taking into account is that older negative-scoring questions may have been deleted. But my subjective evaluation agrees with you: I did feel a marked decline in quality around October 2014, for reasons that haven't been addressed (questions challenging people to guess what the asker was thinking). $\endgroup$
    – Gilles 'SO- stop being evil'
    Mar 9, 2015 at 22:23
  • $\begingroup$ Good news! As of March 24, the fraction of questions scored ≤0 has dropped to about 20%, which is only a bit higher than usual/ $\endgroup$
    – xnor
    Mar 24, 2015 at 22:37

3 Answers 3

Reset to default
13
$\begingroup$

What I think we're looking at is a tendency toward a desire for higher originality. Take a look at rebus puzzles sorted by age, then scroll to the bottom of the page. As you scroll up, watch the scores, and you'll see what I mean. Initially, they scored highly, then slid down quickly.

Let me ask this question: how many triangles are in this pentagram shape?

enter image description here

If this were the first puzzle of its kind that you'd ever seen, then I imagine you'd probably upvote it: It's interesting the first time. The second time? How many triangles are in this figure?

enter image description here

By now, you're bored. Even if it's the first time you've ever seen a puzzle like this, you're still probably thinking "another one? Really?" Thinking back on the first puzzle, the votes are now inconsistent. Why upvote one and not the other?

People begin holding puzzles to higher standards of originality. We seem to have a bias toward new types of puzzles, even when these new shiny puzzles aren't all that high quality in the context of others.

In the long run, it's probably alright to let this bias run its course. For each type of puzzle we see, we'll come to expect originality and high quality. I think our standards for these puzzles will converge and be consistent long-term.

As a case in point, I'm going to take rebus puzzles as an example, as linked above. Many of the first rebus puzzles posted were upvoted a ton. Theeeeen after about seven days, we hit a stack of negatively-scored posts. Why? If I had to guess, I'd say people got sick of dumping rebus puzzles. When it comes down to it, a lot of them are really mostly the same.

I'm confident, though, that if someone were to post an intricate rebus puzzle, or one which contained some unusual or nonstandard variation, it would receive upvotes.

Do we need to do anything about this bias? I'm not sure. Let's split it up by new users and regular users:

  • For regulars, I'm not sure it's a serious issue. Regular users know when they're being original and when they're not. We know when a puzzle we're posting has new value, an unusual twist, or odd surprise - and when it's just a pulp paperback puzzle. If you'd see it, and can imagine yourself going "another one? Seriously?" then it's probably not original.

  • For new users, it gets sort of tricky, and I'm not totally sure what to do about it. We definitely want new users to come and stick around a while, and that usually means having a constructive experience on the site. It can be hard to get "constructive" and "downvotes" to mix well.

    Here's what I'd recommend: if you see a lackluster puzzle, and it's coming from a new user, leave a friendly comment. Give 'em a welcome, and let them know that we see a lot of these types of puzzles, but if they have other new or interesting content they'd like to post that they should feel free to.

I'm not sure this covers everything, but it's a place to start, and I think this is the pattern we're seeing.

$\endgroup$
6
  • 1
    $\begingroup$ I think this is an excellent summary of the situation. Rebus puzzles are the new spaghetti! OTOH, maybe the main problem is not that the puzzles are inherently 'boring' but that people downvote too quickly. Your two triangle puzzles are equally valid; would it be right for the second one to be downvoted just because it was posted second? IMO we should judge puzzles on their own merit, regardless of what other puzzles have been posted before. $\endgroup$
    – Rand al'Thor
    Mar 8, 2015 at 11:07
  • $\begingroup$ @rand I agree that there should ideally be more objective criterion by which we vote, and I don't think I'd encourage this type of voting. Still, it seems like it's the trend. I'm just observing here, though. $\endgroup$
    – user20
    Mar 8, 2015 at 11:11
  • $\begingroup$ I think you're implicitly encouraging it by the language you use. "If you see what you think is a stock, boring puzzle" might be more neutral. $\endgroup$
    – Rand al'Thor
    Mar 8, 2015 at 11:17
  • $\begingroup$ While I agree that people got tired of rebus puzzles, I think part of it is that the rebuses generally got worse. And that's prone to happen when a puzzle type gets popular enough that newbies to that puzzle type try their hand at making them. It's good for puzzle writers to stretch their boundaries, but not to this level of bandwagoning. $\endgroup$
    – xnor
    Mar 8, 2015 at 18:00
  • $\begingroup$ @xnor That may be a side effect of the original upvotes. People see highly-scored posts, and want to post similar ones, but don't put in as much effort. Could be wrong, though. $\endgroup$
    – user20
    Mar 9, 2015 at 3:06
  • 3
    $\begingroup$ I don't think it's wrong at all. This is exactly what happened with code trolling back at PPCG, and with spaghetti on this site earlier on. $\endgroup$
    – user88
    Mar 10, 2015 at 2:03
4
$\begingroup$

On the whole Emrakul's excellent answer sums up the situation nicely, but I have enough extra points to make that they won't all fit in comments.

Firstly, let's not assume that all multi-downvoted puzzles are necessarily bad. People downvote for all sorts of reasons, many of them subjective. As Emrakul said, regular users have now seen a lot of rebus puzzles and started getting bored of them. Maybe some people downvote every new rebus puzzle just because it's a rebus puzzle - this is bad. We should objectively examine each new puzzle and judge it on its merits, not on whether it's similar to lots of other puzzles we've seen. What if a real gem comes up as the 20th rebus posted in a day? Would it get upvotes or downvotes?

From Emrakul:

Here's what I'd recommend: if you see a stock, boring puzzle, and it's coming from a new user, leave a comment. Give 'em a welcome, and let them know that we see a lot of these types of puzzles, but if they have other new or interesting content they'd like to post that they should feel free to.

I agree with this, but I think the tone of such a comment should be carefully crafted. A lot of people come to SE, post one or two questions, get a negative response, and then leave. Users who've been around for a long time can be trigger-happy with the downvote button and forget how it feels to be a new user hoping for approval. That's why I've never quite liked remarks along the lines of "Puzzling is going really well: we're getting lots of downvotes now!" Of course some questions really are cr*p and should be downvoted or even closed, but we should be careful not to go too far. Some of our top users started off posting bad questions (often as near-copies of existing questions) and then stayed, learned from their mistakes, and ended up with a very high-quality output. Here's my suggestion for a comment of the type Emrakul mentions:

Welcome to Puzzling SE, and thanks for your [type X] puzzle! Don't be too discouraged if you get downvoted; there have been a lot of [type X] puzzles lately and people are getting bored of them. Please feel free to contribute more puzzles to the site; non-[type X] puzzles will probably have a better reception.

Let's be positive, people! :-)

$\endgroup$
2
  • $\begingroup$ A good attitude?! :O $\endgroup$
    – d'alar'cop
    Mar 8, 2015 at 11:58
  • 1
    $\begingroup$ @d'alar'cop ??? $\endgroup$
    – Rand al'Thor
    Mar 8, 2015 at 12:02
3
$\begingroup$

I agree with your perspective. Without an analysis of the low-voted posts, I am guessing that two significant issues were a) questions from new users and b) Rebus puzzles.

It seems that types of puzzles go in cycles. Before February, there had only been 4 Rebus puzzles. During February, there were 40 and almost half received zero votes or less. It seems like it is easy to post a poor quality Rebus and difficult to make a good one.

Regarding new users, I think that it is not easy for them to understand what is a good question unless they spend time going through past questions and meta discussion. In my opinion, the most significant need on puzzling.se is the development of concise guidelines about what is good and what is not. I do understand that this would take a significant effort by several dedicated people as well as some time to build consensus.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .