13
$\begingroup$

We all love Puzzling Stack Exchange, but there is a whole world of people out there who need answers to their questions and don't even know that this site exists. When they arrive from Google, what will their first impression be? Let's try to look at this site through the eyes of someone who's never seen it before, and see how we stack up against the rest of the 'Net.

The Site Self-Evaluation review queue is open and populated with 10 questions that were asked and answered in the last quarter. Run a few Google searches to see how easy they are to find and compare the answers we have with the information available on other sites.

Rating the questions is only a part of the puzzle, though. Do you see a pattern of questions that should have been closed but are not? Questions or answers that could use an edit? Anything that's going really well? Post an answer below to share your thoughts and discuss these questions and the site's health with your fellow users!

$\endgroup$
  • 5
    $\begingroup$ Some questions cannot be Googled because they are extremely individual (eg 'my friend sent me this coded message'). What do we do for cases like this? $\endgroup$ – user2174 Nov 24 '14 at 7:53
  • 5
    $\begingroup$ @MathiasFoster If the question cannot be googled, it's not interesting to many people, so “satisfactory” if it is interesting and has good answers and “needs improvement” otherwise. $\endgroup$ – Gilles 'SO- stop being evil' Nov 24 '14 at 8:34
  • $\begingroup$ @MathiasFoster, perhaps (in some cases) we could add keywords relating to the type of solution or the method of getting it? $\endgroup$ – A E Nov 24 '14 at 11:51
  • 3
    $\begingroup$ @Gilles and MathiasFoster, anything that's not generally useful should be marked as 'needs improvement' officially. Just saying. And when they can't be found in the first two pages of Google then all the more so (per last line of needs improvement). $\endgroup$ – David Mulder Nov 24 '14 at 20:20
8
$\begingroup$

My thoughts on the questions:

How many Queens on a board

This is a variation of the eight queens problem, but there's no way to make that connection without trying to search on Google. The linked question doesn't point out the original puzzle either. That's a problem for two reasons:

  1. Experts tend to care about the history of their craft. This is an interesting variation on the original problem, to be sure. But the puzzle is more interesting when you know about the source. This version seems a bit arbitrary.

  2. There's no way to find this question if you are looking for variations on the 8-queen problem.

The second consideration makes me doubt the self-evaluation results:

Net Score: 14 (Excellent: 21, Satisfactory: 24, Needs Improvement: 7)

I'm curious what search terms people used to come up with an Excellent evaluation. Most of my searches actually turned up this Stack Overflow question (which isn't exactly spectacular either). Only when searching the exact wording of the question, did I find this question. And even then, it was below the Wikipedia article I linked to above. Needs Improvement


The Old Millionaire

Doing a quick search of the title, I found this droll cartoon:

The crabbed millionaire's puzzle

That's not exactly the same puzzle though. Thankfully, the OP commented:

When knights and knaves meet the Monty hall... – warspyking Oct 21 at 18:08

If you are looking for a mashup of the Monty Hall and knights and knaves puzzles, this question shows up in Google near the top result. With that little bit of background, this becomes a very interesting (and findable) puzzle. It also seems novel. My vote: Excellent

Net Score: 14 (Excellent: 21, Satisfactory: 22, Needs Improvement: 7)


3D wooden puzzle

The answer helpfully notes that this is called the Burr puzzle. Unsurprisingly, this question doesn't show up in Google searches. My guess is that with an edit of the title it would. Needs Improvement

By the way, this is a fine example of an identification question. While some sites encourage these questions, they have limited value.

Net Score: 14 (Excellent: 20, Satisfactory: 29, Needs Improvement: 6)


Crack the Code #2

Very generic title; I didn't bother with a search. Needs Improvement

Net Score: 6 (Excellent: 15, Satisfactory: 30, Needs Improvement: 9)


Ship and stone puzzle

I immediately found other versions online: The Brick, The Boat and The Lake and Will there be any change in water level/height of floating part of the boat if stones are dropped into the pond from a floating boat?. The Puzzling.SE version is perhaps a bit cleaner, which is good. Typically, this would be an Excellent in my book, but I have a nagging suspicion that one of the other variations should have been credited as the inspiration

Net Score: 3 (Excellent: 20, Satisfactory: 20, Needs Improvement: 17)


Crack the Code #4

Ok. Another in the series I mentioned before. Ho hum. Needs Improvement

Net Score: -2 (Excellent: 10, Satisfactory: 18, Needs Improvement: 12)


Knights and knaves in a foreign language

The answer identifies this as The Hardest Logic Puzzle Ever, so I'd call this a chestnut. A better question would cite the original formulation and find some way to add value to what you can already read on Wikipedia. Failing that: Needs Improvement

Net Score: -3 (Excellent: 11, Satisfactory: 22, Needs Improvement: 14)


Who is the killer

Ugh. Needs Improvement

(As an aside, "abcdefghijkmnopqrstuvwxyz" is a surprisingly popular title for YouTube videos.)

Net Score: -4 (Excellent: 14, Satisfactory: 20, Needs Improvement: 18)


What is the number 4.9292683…?

Well, this question turns up right away when you search for the number. But so does the Inverse Symbolic Calculator Plus entry, which is certainly the more useful result. All in all, I'm going to go with Satisfactory since it does provide a general solution to this sort of problem.

Net Score: -8 (Excellent: 14, Satisfactory: 20, Needs Improvement: 22)


Puzzle of Random Numbers

While it's damned easy to find this question if you were presented with the same message, the odds are you won't find it if you are faced with a similar cypher. Needs Improvement

Net Score: -15 (Excellent: 10, Satisfactory: 19, Needs Improvement: 25)


Overall, my evaluation is much lower on most questions than the community's view. Part of the issue might be that I'm weighing this bit of criteria:

Run comparative Google searches on these questions and see if the content is better or worse than what is already out there on the internet. Are the answers correct, clear, useful and informative?

More heavily than this bit:

Would the question and answer be interesting to the kind of user this site is trying to attract?

In my judgement, based on this sample 90% of the questions are disposable. What I mean is that they will provide some entertainment when they are first posted and after a day or two be lost deep in the question list never to be read again. It's a fine line between long-tail questions and no-tail ones.

One of the ways we measure that difference is by looking at how much traffic comes via search engine. We are looking for something like 80% on a healthy site. At the moment Puzzling is at 18%. Now that should increase over time as more and more questions are asked in indexed by Google. But I'm concerned that this site has become too dependent on flash-in-the-pan questions that pop up in the hot questions sidebar on other sites for traffic.

I'm also concerned that so few questions acknowledge the debt they owe to existing work, titles are generally poor, and so many questions are minor variations of previous questions. I'd love to see more people take ownership of the content on this site by improving it when possible and downvoting it when not.

$\endgroup$
  • 1
    $\begingroup$ I agree that many questions are disposable - this probably deserves a meta q? I would also say that most of these questions were asked by new users (i.e. users who were new at the time) $\endgroup$ – d'alar'cop Dec 4 '14 at 22:32
  • 4
    $\begingroup$ @JonEricson Given the recognition that Puzzling is unusual in that challenges are not meant to solve a real-world issue, can you explain the role of being easy to find on Google? Where do original puzzles fit in? More generally, how would SE judge the health of a challenge-based site? $\endgroup$ – xnor Dec 5 '14 at 4:49
  • 1
    $\begingroup$ I actually found the burr puzzle question in a single search. It was on the first page, too. $\endgroup$ – Aza Dec 6 '14 at 1:47
  • $\begingroup$ @JonEricon I think you use a different angle when looking at the site than many others. As such, your posting is extremely valuable in my eyes. +1 I think xnor's comment above needs to be taken into account, when evaluating the site as a whole - and Google does play a smaller role on Puzzling.SE than on other expert sites - , but your recommendations on improvements are surely valid. In particular as far as titles and references are concerned for those puzzles, which do life 'outside' their own posting. $\endgroup$ – BmyGuest Dec 16 '14 at 17:24
  • $\begingroup$ @Emrakul: Using an incognito window, I was able to find a question about the burr puzzle. Unfortunately, it wasn't the one from this evaluation period, but this question. And the link was near the bottom of the second page of results. $\endgroup$ – Jon Ericson Dec 17 '14 at 23:01
  • $\begingroup$ @JonEricson Interesting. In incognito, I remember finding it there, but I can't seem to do it anymore. I don't remember what I searched - should have made a note of it. Oh well. I'll go with your result! $\endgroup$ – Aza Dec 17 '14 at 23:03
5
$\begingroup$

I may stand alone, but I feel there are a lot of "loopholes" used in both highly voted questions as well as answers.

It's hard for me to create an exact list of loopholes but I know them when I see them.

For example Looking at a die's second highest answer at this moment is that the die is clear and thus from almost any angle you can see all 6 sides. The question had to be edited to specifically say it was an opaque die. An answer like that makes the question trivial. It also implies that the question isn't giving you enough detail and you have to fabricate your own details in order to provide an answer. Logic didn't lead a person to believe "because of fact A it must be translucent" instead the solver simply decided "I'll make this easy on myself and change something very fundamental to the question."

I feel this isn't in the spirit of a puzzle. You need wit to solve the problem, not to undermine the problem.

$\endgroup$
4
$\begingroup$

Final Results

Net Score: 14 (Excellent: 21, Satisfactory: 24, Needs Improvement: 7)


Net Score: 14 (Excellent: 21, Satisfactory: 22, Needs Improvement: 7)


Net Score: 14 (Excellent: 20, Satisfactory: 29, Needs Improvement: 6)


Net Score: 6 (Excellent: 15, Satisfactory: 30, Needs Improvement: 9)


Net Score: 3 (Excellent: 20, Satisfactory: 20, Needs Improvement: 17)


Net Score: -2 (Excellent: 10, Satisfactory: 18, Needs Improvement: 12)


Net Score: -3 (Excellent: 11, Satisfactory: 22, Needs Improvement: 14)


Net Score: -4 (Excellent: 14, Satisfactory: 20, Needs Improvement: 18)


Net Score: -8 (Excellent: 14, Satisfactory: 20, Needs Improvement: 22)


Net Score: -15 (Excellent: 10, Satisfactory: 19, Needs Improvement: 25)


$\endgroup$
  • 2
    $\begingroup$ The scores seem to correlate quite well with how rigorously the puzzle can be stated. $\endgroup$ – xnor Dec 2 '14 at 4:31
  • $\begingroup$ the very top scored question here... (which I asked btw)... was identified as low quality at some point in the past... what should be extrapolated from this? $\endgroup$ – d'alar'cop Dec 2 '14 at 9:17
  • $\begingroup$ @d'alar'cop what do you mean by "identified as low quality"? That question has not gone through the low-quality review queue (or any queue for that matter), has never been flagged, and has no deleted comments. $\endgroup$ – Kevin Dec 3 '14 at 6:40
  • $\begingroup$ @Kevin fair enough. someone had pointed to it in a meta post. it is also, to be fair, an instance of a mass-producible puzzle $\endgroup$ – d'alar'cop Dec 3 '14 at 6:42
  • $\begingroup$ @Kevin oh btw, it is a Challenge question... and actually at least 4 of the 5 with positive score are Challenge questions $\endgroup$ – d'alar'cop Dec 3 '14 at 12:35
2
$\begingroup$

Honestly, I have to say, this site doesn't seem to exist to attract users.
It's the only puzzle website with a variety of puzzles; most puzzles sites are jigsaw or strategy games. I mean, just look at the tags, yeah, they're all over the place. If this site is made to attract people, it's doing a pretty bad job.
Honestly, I think that this site is more like a break from all the schoolwork, a place to try new things. Stack overflow, Ubuntu, those will attract users because you don't try new things. I think it should stay; it really emits the time, work, and ingenuity that goes into these puzzles.

$\endgroup$
1
$\begingroup$

The initial visiblity of answers makes the site dangerous to browse just for the sake of browsing and looking for interesting puzzles to solve. Even just seeing out of the corner of your eye that the accepted solution is one line gives you data on how to solve the puzzle.

We already have people who come here and post new puzzles: these clearly are not within the remit of building a repository of canonical puzzles with canonical answers that's searchable from the outside.

Hence my feature request that answers be hidden on the site for each question you visit until a button is clicked to show them. The Chess guys get their animated chess move software, why can't the Puzzling site get this feature?

$\endgroup$
  • 2
    $\begingroup$ We have Empuzzler. Maybe we should try to have it integrated officially. Nice idea: meta.puzzling.stackexchange.com/questions/1316/… $\endgroup$ – d'alar'cop Nov 25 '14 at 23:53
  • 4
    $\begingroup$ You might like to ask a separate meta question out of this answer. $\endgroup$ – d'alar'cop Nov 26 '14 at 0:19
  • 5
    $\begingroup$ @d'alar'cop I already did, and got a bunch of answers that said "everything is fine the way it is" $\endgroup$ – Almo Nov 26 '14 at 3:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .