As many others have said, having the hottest questions be those with lots of low quality answers is problematic. I wager the network wide aggregation of hotness (the calculation used to make the actual hot questions list) normalizes all the site specific values and performs its own calculations. Likely this includes number of answers already, so we should factor this out of our calculation. I could be wrong about the way this works, but I'm too lazy to dig through MSE to figure it out, so I'm just going to assume it.
The variables we have to work with, from the linked MSE post, to which I've added abbreviations:
- QS - Votes [aka Score]
- V - Views
- N - [number of] Answers
- AS - Answer Votes [aka Score]
- A - Whether the question has an accepted answer or not
- To - Time question was originally asked
- Tr - Time of last activity on question
- Rq - Reputation of asker
- Ra - Reputation(s) of answerers
Factors we want:
- QS - Question score
- AS - Answer scores
- V - Views
- Tr - Time of recent activity
Factors we don't want:
- Answer count. Maybe this can be useful indirectly, but questions experiencing willy-nilly answer accretion aren't what we want to show.
- Reputation. Some of the best content I've seen (admittedly I haven't been active on here long, so this is going to be biased in a few ways) has come from new or low rep users. Whether or not someone can write a great post has nothing to do with their rep. And high visibility for low rep but high quality contributors will create a positive feedback loop (and similarly a negative one for low quality contributors), resulting in the creation of more good content.
I'm undecided on whether or not to include "has an accepted answer" in the calculation. If so, weight the formula towards not having one. A puzzle which is hot but unsolved is an interesting one.
Calculated metrics I think are valuable:
- AAS - Average answer score. This is a good way to correct for lots of low quality answers. But, if there are one or two really great answers among the low quality ones, then this will redeem the hotness score somewhat. Which is good, because it means that the puzzle was really tricky, or someone came up with a really great answer; these are the sorts of things that make a question interesting.
- TA - Length of time active: Tr - To. This needs to be normalized somehow, since it's an interval of time.
- VD - View density: V / TA. A measure of baseline interest in the question (or at least its title and tags). This corrects for question age, since older questions will naturally have a lot more views.
- SF - Score frequency: QS / TA. Questions which get more upvotes quickly are objectively hotter.
- SD - Score density: QS / V. A question with 5 upvotes and 100 views is probably a better question than one with score 10 and 10000 views.
I'm intentionally not supplying an actual formula. I'll leave that discussion (and the inevitable bikeshedding) to others. But personally, I think something that is dominated by Tr and the four meaningfully qualitative calculated metrics (TA doesn't tell us much about the actual content) is the right way to go.