You should:
- Upvote puzzles that you deem upvote-worthy, for whatever reason.
- Downvote puzzles that you deem downvote-worthy, for whatever reason.
- Closevote puzzles that you deem closevote-worthy, for whatever reason.
And completely independently, you should:
- Upvote answers that you deem upvote-worthy, for whatever reason.
- Downvote answers that you deem downvote-worthy, for whatever reason.
Because:
- A great answer is not made less great because the puzzle was poor.
- In no SE universe is a great answer made less great because the question was poor.
- Addressing a possible concern of yours: Just because the puzzle is easy doesn't mean the answerer doesn't deserve the points. I mean, they could have wrote a crappy answer, but instead they wrote a great one, and that deserves some respect regardless of the puzzle quality. At the end of the day, in the bigger picture over a long period of time with a lot of answers, reputation points will begin to reflect both a given user's positive involvement in the site as well as (ideally) their ability to handle the moderation privileges given to them, and it does not matter what quality some of the puzzles they answered were.
- If a user gives a great answer, regardless of the puzzle, then philosophically / socially of course they gain a better reputation.
And most importantly:
- You don't need to set a guideline for this, because over time the voting patterns will organically reflect the community's collective consciousness, which is what is supposed to happen. If voting patterns become poor on a grand overall scale it is a sign of a changing and/or struggling community, and so bigger problems would have to be addressed.
You get to decide what those "whatever reason"s are when voting, because it is in your power to choose how you vote, and you are part of the community, and the community is made of a lot of you's, and so collectively the votes will reflect what the community naturally wants, which is what these sites are about.
Don't think of the puzzling site as an exception to SE sites. Think of it this way: Stop focusing on the precise number of points somebody has. Instead think of the end effect of having those points and how it reflects that person's activity on and dedication to the site, for example, fuzzier ways of thinking about rep are:
- Hey this person has a ton of points, they must be really active here. Doesn't matter how many points.
- Hey this person has been active enough here that they now have the privilege of being part of the review team. Yay. Doesn't matter how many points.
- Hey this person has very few points but posted a great answer, let's take this opportunity to welcome them to the community (not by rep, but by e.g. friendly interactions).
- Etc.
All of those work out in the long run, +/- a few hundred or thousand points doesn't matter. If somebody comes in, leaves a massively upvoted answer to a really poor puzzle, then leaves, it doesn't matter, because their reputation points won't continue to grow like the active users do, that type of person probably won't use any privileged tools they acquire, and it doesn't hurt anything.
Instead you should focus on higher level meta discussions, such as what makes a good puzzle and what makes a good answer. Leave discussion of how to vote out of it, and let folks decide on their own based on these conversations how they want to vote, without pressure or forced obligation to click a certain vote button. Don't ask "should we upvote good answers to puzzles which we've downvoted", instead ask "what makes a good answer to a poor puzzle".
In my opinion: There's actually a bit of a social danger risk by defining answer voting guidelines, too. If you start doing that, then you start having this community where votes are rather robotic, and even worse, you start to see comments like "please do not upvote this answer, it does not meet our guidelines". Comments like that isolate the voter from the folks who define "our guidelines" (ours vs. yours has now been established), isolate and depersonalize new users, gradually establishes a bad preach-y attitude, and generally runs a risk of reducing cohesion and passion in a community. Call me a pessimist but I have dealt with similar things in real life communities at least, and they become extremely hard problems to solve.
