Reddit Help-Seeking Didn’t Drop After ChatGPT—Here’s What Changed

Reddit Help-Seeking Didn’t Drop After ChatGPT—Here’s What Changed

Introduction

For years, people have used the internet—often Reddit—to ask strangers for help with everything from “what’s wrong with my PC?” to “how do I fill out this form?” The big question after generative AI like ChatGPT arrived is simple: did we stop asking humans because machines could just answer instead?

New research based on the original paper suggests the answer is: not in the way you’d expect. Hazem Ibrahim and Yasir Zaki study Reddit informational communities around ChatGPT’s release (30 Nov 2022) and find that informational help-seeking didn’t decline. Even better (or more frustrating, depending on your viewpoint), the study explains why earlier results were all over the place.

But the paper doesn’t stop at “post counts stayed steady.” It also digs into whether AI-written content might have quietly replaced human posts—so the totals could look unchanged even if real participation dropped. Spoiler: detector-based checks don’t show a big enough AI “replacement” to hide the declines previously claimed by other studies.

Why This Matters

This matters right now because we’re at the stage where AI isn’t a novelty anymore—it’s becoming a default option. That changes user behavior in two competing ways:

1. Direct substitution: “Why ask Reddit when I can ask a chatbot?”
2. Workflow reshaping: “I’ll still ask, but maybe I’ll ask better—or ask later—after using AI for drafting, troubleshooting, or filtering out the obvious.”

The Ibrahim & Zaki results support the second path more than the first. People didn’t stop posting in the most AI-substitutable spaces on Reddit. That’s a big deal for anyone building communities, support platforms, or tools for troubleshooting: the instinct to “replace” humans may be the wrong instinct. Instead, AI might be acting more like a drafting assistant or a way to sharpen the question so the remaining human questions are worth answering.

A real-world scenario you can apply today: imagine a “help forum” for a specific industry (say, home brewing, small business taxes, or DIY troubleshooting). If leadership assumes AI will “cannibalize” help requests, they might divert community moderators and support resources away. This research suggests that at least on Reddit-style public help channels, users can keep asking humans even when AI answers exist—meaning community management and knowledge-sharing incentives still matter.

It also builds on earlier AI impact research by attacking a methodological problem: many prior studies compared a community to itself (before vs. after) or compared to groups that weren’t a fair substitute test. This paper adds a contemporaneous control group on the same platform and also repeats the analysis at 66 earlier “placebo” dates—a move that reveals how easy it is for natural differences between community types to masquerade as a ChatGPT effect.

How the study answers “Did people stop asking?”

The researchers’ core idea is that if AI is truly substituting for human help, then the communities most likely to be replaced should show a clear decline after ChatGPT’s launch. But to avoid being fooled by Reddit’s messy reality, they compare informational communities to other communities on Reddit that AI can’t easily replace.

The community types they track

They build four groups of subreddits, chosen to represent different “reasons to ask”:

• Group I (Informational advice): places where a model can give a “correct enough” answer—facts, procedures, diagnoses
• Group H (Human-anchored support): people want someone’s perspective/witness; the value is tied to who replies
• Group Q (Low-stakes curiosity): people ask for fun or curiosity, often the kind of question AI could answer easily
• Group C (Controls / Hobby communities): hobby spaces where posts are mostly projects/photos/community talk—not questions a model could answer like a substitute

Instead of just watching the informational subreddits, they also watch these controls so anything that affected Reddit overall (seasonality, site-wide changes, general trends) hits both groups similarly.

What they compare (and why it’s more reliable than “before/after”)

They estimate the change in monthly post counts using a difference-in-differences approach with:

• community fixed effects (controls for persistent differences between subreddits)
• month fixed effects (controls for platform-wide shifts)
• a control group (accounts for general Reddit-wide drift)
• a design assumption tested using placebo dates

For the main window, they compare six calendar months before ChatGPT with the same six months one year later, but only after aligning months to remove seasonal weirdness. The window is:

• Pre: Dec 2021 to May 2022
• Post: Dec 2022 to May 2023
• (and they drop the months in between)

They also run a longer window later (more on that below), but this primary window is designed to capture early adoption while minimizing confounds like Reddit blackouts.

Placebo dates: the “pretend launch” test

This is one of the paper’s most important contributions. They rerun the exact same method at 66 earlier dates where ChatGPT couldn’t possibly have caused anything. If the method “detects” a decline anyway, that’s evidence the design itself may be picking up pre-existing drift between community types.

And it does find drift—often large enough that without controls, earlier studies could easily mistake it for a ChatGPT effect.

The headline result: informational help-seeking didn’t decline

Here’s what they find when looking purely at monthly post counts.

Main estimate after ChatGPT: Group I vs. controls

In the primary 6-month window around ChatGPT, informational communities don’t show the decline the substitution theory would predict.

So instead of a drop, the point estimate is slightly up (+5.1%), and the uncertainty still rules out declines larger than 3.4%.

How this compares to prior published estimates

Prior work on Reddit and other platforms reported declines ranging from “maybe a little” to “pretty big.” Ibrahim & Zaki compare their results to the published numbers they discuss in the paper. Their main interval excludes every decline in that prior list except one smaller estimate.

In particular, their results rule out declines like:

• Stack Overflow ~25% (a different platform, and a different comparison design)
• Reddit advice communities ~8.3% (Gao & Hahn’s estimate)
• Reported declines on the order of ~14% and up

The only one their interval can’t rule out is the smaller ~2.6% estimate from an earlier conference version of one study.

A subtle risk: “stable post counts” could still hide replacement

Even if total posts don’t drop, it’s possible humans left and bots filled the gap. That’s why the paper does the next—and more interesting—test: they check whether AI-like text increased enough to explain away a human decline.

Did bots replace humans with AI-written posts?

This study uses detector-based methods to estimate whether AI-written content accumulated faster in informational communities than in hobby controls.

The basic problem with AI detectors

AI-text detectors can be imperfect—they might flag human writing as AI (false positives) or miss some AI text. If you used raw “how many posts are AI?” you’d risk detector bias.

So instead of labeling documents as AI/human, they use a clever difference approach:

• Score posts with a detector
• Compare how the score shifts between periods
• Subtract the corresponding shift in control communities

That way, detector biases that are stable over time cancel out.

How they score AI-likeness

They use a detector called Fast-DetectGPT on 274,411 sampled posts, and they also run a comments check with Binoculars because Fast-DetectGPT wasn’t reliable enough on shorter text like comments.

They calibrate the detectors using text generated by multiple LLMs (three models widely used during the observation window plus a later model as a stress test). The goal isn’t to claim “this exact post is AI”—it’s to translate detector score changes into an estimated change in AI-share.

Posts: AI-written content didn’t rise enough to hide big declines

They find that AI-written posts rose only 2–3 percentage points more in informational communities than in controls.

They also translate that into a threshold question: would that rise be big enough to “hide” the ~8.3% posting decline claimed in prior Reddit work?

Their result says no. The AI share increase is short of the ~5.1 percentage points needed for the detector-driven “replacement” story to rescue an 8.3% decline.

Comments: no evidence of a similar shift

Finally, they check the responses people receive (comments under posts). If AI were replacing participation, you might see more AI-like comments even when questions remain human.

But the comments don’t show that pattern. Depending on detector calibration, they find either:

• no meaningful rise in AI-like comments, or
• at most very small shifts (and not consistent “pile-up” in informational communities)

So both the “AI posts replaced human posts” and “AI replies replaced human replies” stories don’t get strong support from the detector evidence.

Why earlier studies found “declines” (and this one doesn’t)

The paper’s placebo analysis is basically a warning label for all “before vs after ChatGPT” studies.

Placebos show the method would report effects even with no AI event

At the placebo dates, the researchers still find changes between community types. The magnitude is sometimes large—comparable to what prior papers reported as “ChatGPT effects.”

For example, human-anchored communities (Group H) show gains relative to controls, while curiosity communities (Group Q) show losses relative to controls, even when the “pretend launch” is months or years earlier than any real AI release.

The paper describes drift between informational/human-anchored patterns as large enough that a design lacking proper controls could easily misattribute that drift to generative AI.

A specific cautionary example: low-stakes curiosity communities

If you look at curiosity communities (Group Q), you do see a decline after ChatGPT’s launch—and it looks tempting to call it substitution.

The paper argues the opposite: curiosity communities were already shrinking at about 18% per year against controls before ChatGPT existed. In the study’s largest estimated decline for that group, the placebo trend is almost the same size.

So the “effect” was basically already in motion.

This supports the broader conclusion: without contemporaneous controls and placebo checks, you can get the right-looking direction for the wrong reason.

The longer window (2024–25): not a clean “ChatGPT-only” estimate

The primary window measures early adoption and avoids some major intervening events, but it ends by mid-2023. The authors also run a longer window that covers up to late 2025 to see what happens later.

What the longer window finds

Over the longer window, informational communities show an increase in post volume relative to controls.

Why the authors don’t treat it as “proof of AI helping”

They explicitly caution that the long period contains other big factors that affect Reddit visibility and traffic, including:

• Google’s licensing deal with Reddit (Feb 2024)
• rollout of Google’s AI Overviews (May 2024)

These could shift who finds Reddit threads and how often. Also, by this later period, more posts might be machine-generated, and the count data alone can’t separate demand shifts from platform distribution changes.

Still, when you combine this with their detector findings (“AI-like posts didn’t rise enough to cover big human declines”), the longer window makes it harder to argue that a large substitution-driven drop happened later.

Practical implications: what should community builders and researchers do now?

This study isn’t just “AI didn’t win.” It’s also a guide for how to measure platform-scale change without misleading yourself.

If you run a community or help service

If you’re planning staffing or expect user behavior to rapidly move to chatbots, this research suggests you should be careful. At least in Reddit-like public informational forums:

• users may still come to humans for answers
• social context, accountability, and “who answered me?” can matter
• AI may change how people ask, not whether they ask

A practical move is to treat AI as a moderator assistant or a question-improvement tool, while still investing in human reply capacity for the questions people actually post.

If you’re doing AI-impact research

The paper recommends two “cheap but powerful” safeguards for future studies measuring AI effects:

1. Use contemporaneous control communities on the same platform with low AI exposure (or low substitution likelihood)
2. Run placebo tests at earlier dates where the AI effect should be zero

This paper basically shows that without those, you can accidentally measure pre-existing drift and call it an intervention effect.

And because counts alone can hide replacement, they also recommend testing whether AI-written text is rising faster in the relevant group.

Key Takeaways

• Informational help-seeking on Reddit didn’t decline after ChatGPT. In the primary launch-adjacent window, informational communities show an estimated +5.1% change and the study rules out declines larger than 3.4%.
• Detector checks don’t support the “bots replaced humans” story at large scales. AI-like post text rose only 2–3 percentage points more in informational communities than controls—below the amount needed to hide an 8.3% decline.
• Comments people receive also show no meaningful AI pile-up in informational communities relative to controls, which undercuts replacement-by-AI for answers too.
• Why prior studies disagreed: pre-existing drift between different subreddit types can be large enough that “before vs after” designs (or weak comparisons) can mistakenly interpret drift as a ChatGPT effect. The study’s 66 placebo runs demonstrate this clearly.
• Longer-window results are suggestive but confounded. In 2024–25, informational posts rise, but the period includes other major changes affecting search/discovery (e.g., AI Overviews), so the authors don’t claim a pure ChatGPT-only effect.
• For future research and real-world planning: use controls + placebo tests, and don’t rely on post counts alone—AI-generated text can complicate interpretation.

If you want, I can also rewrite this as a shorter “executive summary” version (for sharing internally) or pull out the exact methodology assumptions and what would have to change for the conclusion to flip.