12
$\begingroup$

Context

We can now get an anti-AI-use banner in the answer box. It would look like this:

Image showing the answer field, in focus, with a banner saying "Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more"

Puzzling SE's current AI-answer policy is, in brief, no using AI to generate answers except for limited you'll-know-it-if-you-see-it exceptions. Note that knowing when a situation falls into an exception is something we probably trust experienced users to do more than new/under-informed users. It's the new/under-informed users who are the target of the banner.

Details

There are two options for text:

  • **Reminder**: Answers generated by Artificial Intelligence tools are not allowed on [Site Name]. Learn more

  • **Reminder**: Answers generated by Artificial Intelligence tools must be cited on [Site Name]. Learn more

What would the downsides of the banner be? We'd lose the "new contributor" banner (example), for one. Also, neither text option fits our policy exactly.

How do we get the banner?

  1. We'd need to collectively decide that we want the banner, which means coming to a consensus in this meta discussion.
  2. We'd need to come up with text for a new Help Center article (see Stack Overflow's AI policy as an example). This article would be linked by the "Learn more". We could discuss in answers to this question, set up a separate meta question, take the existing Puzzling Meta answer (perhaps slightly modified), etc.

Question

Do we want this banner? If so, what should our Help Center article say and/or how should we decide that?

$\endgroup$

2 Answers 2

Reset to default
13
$\begingroup$

I can't speak for others, but to me, AI-generated content does not seem like a huge problem on this site. There certainly has been some, but it doesn't seem to be as harmful here as it is on other sites.

I see three potential scenarios here:

  1. Someone posts a large number of incorrect AI-generated answers to puzzles. I believe this is a lot less harmful here as compared to other Stack Exchange sites. The thing that sets this site apart from most of the others is that here, the poster generally already knows the correct answer. There's no danger of anyone being misled by false information. Yes, low-effort wrong answers are certainly unwanted clutter, but they aren't the major threat that they are on fact-based question and answer sites like Stack Overflow.
  2. Someone posts a large number of correct AI-generated answers to puzzles (thus stealing the green check mark from those who have earned it). Time will tell, but this seems wildly unlikely to me, at least for puzzles that are original and have some thought behind them. AIs like ChatGPT are embarrassingly bad at this sort of thing. I could see an AI getting the right answer to a question that has appeared many times on the internet (in other words, in the AIs training data), but do we really want questions like that on this site anyway?
  3. Someone posts a large number of AI-generated questions (puzzles) on the site This could theoretically be a big problem if the AI generates something that looks like a puzzle but has no answer. In practice, I'm not too worried. As an experiment, I tried my best to get ChatGPT to generate a set of convincing-looking puzzles, and it mostly generated superficially puzzle-like garbage that wouldn't fool anybody. It knows what a puzzle looks like, but it doesn't understand how they work. Just one of them looked OK, but it was a well-known riddle that would have been quickly closed for lack of attribution. Again, this is not something the current generation of AIs are good at. Even if this was an issue, the banner wouldn't help much because (if I understand correctly) it only shows up on answers, not questions.

I personally do not think we need the AI banner on this site, especially if it's a choice between that or the new contributor banner (which, IMHO, is fairly important).

$\endgroup$
3
  • $\begingroup$ Accepting this for now, it's what I expected! Good that we now have something to point to for the decision, though. $\endgroup$
    – bobble
    Commented Jan 12 at 4:32
  • $\begingroup$ Re: 3. the banner can only be activated for the answer box, AI-generated questions are very rare across the network when compared to generated answers. $\endgroup$
    – Glorfindel
    Commented Jan 15 at 21:11
  • $\begingroup$ @Glorfindel - Thanks, that confirms what I was thinking. I included this scenario here because I could have sworn that, a while ago, I saw a few weird half-formed questions that looked suspiciously like someone had dumped some unmodified AI-generated fake puzzles on the site (for who-knows-what reason). I can't be 100% certain about that though. It was just my impression. It doesn't really matter in any case, because, as you point out, the banner is for answers only. $\endgroup$
    – plasticinsect
    Commented Jan 16 at 17:28
5
$\begingroup$

Wow. Not a whole lot of engagement on this question...

I haven't been around enough recently to know how much of an issue AI-generated answers have been on Puzzling, but I think that should inform the answer.

  • If there has been a lot of AI-spamming, then probably it's worth adding the banner, even if it doesn't exactly reflect the policy (the "Learn more" could clarify).

  • If AI answers have not been a big issue, then there doesn't seem to be much point in adding the banner.

The question could always be revisited in the future if circumstances change.

$\endgroup$
1
  • $\begingroup$ There have been several that I recall - they tend to come in waves, one user posting a bunch at once (why, I couldn't tell you), no more than 1 a week on average. $\endgroup$
    – bobble
    Commented Jan 9 at 0:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .