I see a concerning trend where puzzles get upvoted and downvoted on first impressions before any solution attempt is posted. And, if a solution is posted that makes the votes seem unwarranted, there's not enough votes after that to swing it the other way, and by then it's too late to for the votes to have an impact.
It's often hard to tell if a puzzle is good from its statement because so much depends on the solution. It might be clever or tedious, well-fitting or arbitrary, satisfying or disappointing. A good puzzle is enjoyable to solve, not just to read. By the very nature of puzzles, the crucial element, the solution, is hidden. Yes, it is important that the puzzle be intriguing and be well-written and seem like it would have an interesting solution, but voting only on that is judging a book by its cover.
This is a particular problem for enigmatic or cryptic puzzles, whose surface content is liable to look like gibberish.
I believe this causes a number of problems:
Searchability: It's harder for future readers to decide if a puzzle is probably good. The long-term goal of the site is to make a well-organized repository of high-quality puzzles, so it's important that well-received puzzles stand out.
Feedback: Puzzle-writers get worse feedback via votes, making it harder to judge what people liked. A new-ish user who posts a great puzzle that looks like nonsense might get discouraged with the flood of rash downvotes and leave. A writer who writes a flawed puzzle that looks pretty might not get the message that it's a poor puzzle because of all the upvotes.
Reward: Votes give encouragement for puzzle-writers to make good puzzles. But, skewing the reward towards surface appearances might condition puzzle-writers to post shallow but immediately-appealing puzzles over deep but satisfying ones.
(A related problem is that votes are largely proportional to views, and a well-received +10/-0 looks just like a poorly-received +25/-15 one to anyone without 750 rep.)
In part this is a technological problem: once you vote, you can't change it unless the puzzle is edited. So, those whose minds were changed by a solution can't change their votes. I sometimes refrain from vote to wait for a solution, but then I forget the post and never vote.
Is this indeed a problem? Is there something we can do about it?
