There's obviously been some great discussion recently around possible platform changes that could help bring more attention to this site's best content, and we already have the quarterly "Best of" series... However, I've been trying to think of other ways to highlight worthy content, especially content that may have gone by unnoticed for a lot of people.
The problems I'm aiming to overcome are:
- Quality content slips off the front page too quickly, and while well received, don't get the views they deserve and end up getting missed by people who would otherwise be interested
- Due to the way the Hot Network Questions algorithm works, lower quality questions often get vastly more views*, and thus get more votes (purely from this additional exposure), meaning that raw votes are unfortunately not particularly representative of the quality of a post, which makes things like the Hottest Questions This Month tab fairly useless (case in point, 2 of the top 4 posts in that list are on hold/closed)
- The "Best of" posts, while awesome, are necessarily limited by the fact that they're only run quarterly and because it requires individual users both remembering specific puzzles and choosing to participate, by nominating them, meaning we're only highlighting a handful of posts each year
Broadly, what I'm proposing is monthly posts here on meta listing collections of high quality content, in a similar way to the existing "Best of" posts, except making them:
- Fully automated
- More frequent
- Highlight significantly more content
With that in mind, I've been playing around with the puzzling data on SEDE trying find ways of identifying posts worthy of bringing additional attention to. As noted above, looking at pure vote or answer counts is not a good indicator, but I've observed that looking at "votes per view" starts to not only filter out some of the chaff, but also uncover some pearls with low view counts.
Obviously just directly sorting posts by votes/views doesn't work particularly well, since you end up with outliers having very low views and one or two votes overwhelming the results. However, using that as the basis, I think it should be possible to find an algorithm to automatically list out "high quality" content on a monthly basis in a reasonably fair and accurate manner.
I've come up with a couple of potential algorithms that appear to be suitable, but have different strengths/weaknesses. I've posted more detail on each as individual answers below, so as not to further clog this post and so that they can be discussed/voted on in their own right.
Firstly, let me be clear that I see this as a complementary series of posts to the "Best of" quarterlies, with those being the "people's choice" awards, so to speak, and these being purely automated. With that in mind, some topics for discussion:
- Is this generally something worth pursuing?
- Which of the algorithms below are most useful/suitable? Or is there an alternative that's better? (noting that I'm far from a statistician, so it's entirely possible that there's issues with my algorithms)
- How frequent should these posts be? (I think monthly is probably a good balance)
- How many posts should we "highlight"? Top 10? Top 25? All above a given threshold? Top N posts from multiple algorithms?
- How should the posts be structured?
- Single mega post with a new "answer" each month, containing that month's list
- Post per month with a single "answer", containing that month's list
- Post per month with several "answers", each containing a list from a different algorithm
- Post per month with individual "answers" for each puzzle in the list (so that they can be voted on and "sorted" by the community)
I've posted a bunch of answers showing a few example algorithms and their results, purely for the benefit of gathering vote based feedback, but feel free to post other answers just adding to the discussion, or proposing alternative algorithms.
* The issue with HNQs from PSE's perspective is that the algorithm is based off a question receiving many upvoted answers in a short time, so a low-ish quality, broadly worded riddle/lateral-thinking puzzle will naturally be ranked as "hotter" than a puzzle which sits unsolved for a while, or gets a single correct response and is "done". Then, once in the HNQ list, it gets more views, some proportion of which naturally being upvoters and more answerers, further boosting the "hotness", and skewing the validity of raw vote scores as a measure of quality.