Biased LLM Thinking From Copied Human “Proof”

Biased LLM Thinking From Copied Human “Proof”
(SEO: origins of decision bias in LLMs)

LLMs can be weirdly confident about making biased choices—even when the “reason” for the bias should be nonsense. New research from Yechiam & Tarabeih (arXiv:2608.12339) digs into why this happens, and the core idea is both intuitive and unsettling: large language models can pick up human decision biases through mimicry without understanding.

The paper shows two routes to bias. First, LLMs can misread human behavior as a preference signal, even when the behavior is logically unrelated to what it claims to represent. Second, LLMs can copy biases that are explicitly labeled as biases in scientific write-ups—meaning the very act of describing human bias in papers can make LLMs reproduce it. The authors run four studies using ChatGPT-4o and Qwen2.5-72B-Instruct, and they even quantify how much the models “lean” into loss aversion. Spoiler: the more strongly a scientific abstract claims humans are loss averse, the more loss-averse the LLM becomes afterward.

Why This Research Matters (Especially in 2026)

Right now, a lot of people are trying to “steer” LLMs with prompt instructions like “Answer rationally” or “Use unbiased reasoning.” But this research suggests a deeper problem: the model may not need to reason to copy bias. If it has learned that certain human choices correlate with certain written explanations, it can reproduce those correlations—even when they shouldn’t logically transfer.

Here’s a real-world scenario where this hits immediately: imagine a decision-support chatbot for personal finance. You might feed it scientific summaries of behavioral economics, like “humans are loss-averse” (or that bias is weak or inconsistent). According to this paper, the LLM doesn’t just “use” that information—it can translate the reported bias magnitude into its own decision rules. So if the summary exaggerates loss aversion (which can happen depending on which meta-analysis is cited), the chatbot may start treating losses as disproportionately painful—even if that’s not actually optimal for the task at hand.

This also builds on a thread in earlier AI research: many studies have shown LLMs exhibit human-like cognitive and social biases. But what this work adds is a mechanism-level account of where the bias injection comes from. Instead of only saying “LLMs mimic humans,” it argues the mimicry can be surface-level and illogical, driven by how behavioral descriptions and labels are absorbed. That’s a crucial step forward for safety work: you can’t fully debias a model just by telling it “be fair” if the model is pattern-matching social proof and bias language internally.

How the Studies Test “Mimicry Without Understanding” Step by Step

The authors don’t just run one experiment and stop. They progressively make the prompts more “scientific” and more directly about bias itself—so you can see the bias emergence across increasing levels of instruction.

They run four studies (plus a variation in Study 4), comparing ChatGPT-4o with Qwen2.5-72B-Instruct across scenarios designed around economic biases. In total, the core experiments use hundreds of independent model sessions—each session treated like a separate “agent” or participant.

Below is the high-level comparison of what the studies probe and how they differ:

The paper’s logic is: if mimicry were “real understanding,” then when the description becomes logically irrelevant or explicitly labels bias, the model should adjust. Instead, the model largely keeps copying.

Study 1: When “People Prefer This” Turns Into LLM Money Decisions

In Study 1, the authors create a clean test for social proof. Two currencies (Tenits and Tanas) are objectively equal—an exchange rate of 1:1. But the prompts tell the model that humans prefer one currency over the other, based on different observed pairwise choices (like preferring 11 Tenits over 12 Tanas, etc.).

The key move is how the LLM is later asked to make decisions: it chooses between a lottery and safe outcomes. Some safe outcomes are denominated in the “preferred” currency. If the model is rational and expected-value (EV) maximizes, then with equal exchange rates, the denomination shouldn’t matter. Any systematic preference would reveal bias.

What they found

Across both ChatGPT-4o and Qwen, the LLMs strongly favored safe options in the human-preferred currency. This is classic social proof behavior: treat the crowd’s preference as evidence that something is “better” or “correct.”

Importantly, the bias didn’t vanish when it became costly in EV terms. The authors vary the safe option value via expected value levels:

• S=10 corresponds to 100% EV of the lottery (no cost to mimicking).
• S=5 corresponds to 50% EV (mimicking can reduce EV a lot).

They report that when S=10, 37.5% of model agents picked the safe option when it was in the preferred currency (vs. not). When S=5, it dropped to 17.0%—still biased, just weaker. The study’s statistical analysis finds a significant interaction between preference and safe amount, meaning mimicry declines as EV cost increases, but doesn’t fully disappear.

Why this matters beyond the task

This shows an essential feature of mimicry: even when a rational strategy would say “denomination doesn’t matter,” the model can learn a rule like “if humans choose it, it must be better.” The model is effectively doing LLM see → LLM do, but the bias is not constrained to situations where it is logically warranted.

Study 2: Mimicry Even When Human Choices Give No Valid Preference Signal

Study 2 is designed to eliminate the “reasonable story” the model might make. The prompt still says people prefer one currency over the other—but in this version, that “preference” is logically non-indicative because of how the choice is constructed.

Think of it like this: if someone chooses based on a hidden confound (in the study’s setup, currency choice is tied to amounts in a way that breaks the inference), then the choice itself shouldn’t reliably indicate true preference. In the earlier analogy from the paper, it’s like the “2 US dollars over 1 yen” example: the behavior constrains exchange rates only weakly and doesn’t actually reveal a stable preference between dollars and yen.

What they found

Even here, the LLMs still showed a strong social proof effect. The models preferred the currency chosen by humans despite the fact that, logically, that choice doesn’t imply genuine preference.

The authors quantify the mimicry similarity across Studies 1 and 2: the magnitude is similar (they report an effect estimate of 0.72 vs 0.82, respectively). That’s a big deal: it suggests the models are not robustly distinguishing when “human behavior” is informative versus misleading.

They also inspected the verbal reasoning from ChatGPT-4o: the paper reports that models made the preference inference in 100% of agents—interpreting “people preferred 28 Tenits over 26 Tanas” as a preference for Tenits over Tanas.

Practical implication

If you’re building systems where LLMs interpret human behavior as evidence (“most users like this, therefore it’s better”), this is a warning sign. The model may treat surface-level choice patterns as meaningful, even when the choice doesn’t logically indicate the underlying preference you care about.

And connecting back to the broader paper premise: this supports the “faulty mimicry” mechanism—misreading behavior as preference evidence—which appears even when humans’ choices shouldn’t provide that evidence.

Study 3: Loss Aversion as an Explicit “Bias Copy-Paste” Rule

Study 3 tests the second mimicry path: mimicry when the behavior is explicitly described as biased. The authors focus on loss aversion—the idea from prospect theory that people typically weight losses more heavily than equivalent gains.

They use fictional experimental results in the prompt, telling the model that in a coin-toss gambling study, humans either:
- show loss aversion (more people reject lotteries involving losses), or
- show the opposite, gain seeking (more people accept even when losses occur).

Crucially, the study includes a manipulation of the sample size reported in the human study:
- very large (10,583 people)
- very small (43 people)

A rational model should infer that small samples are less reliable and therefore should show weaker mimicry. If mimicry were “understanding,” the sample size should matter.

What they found: sample size didn’t matter much

Both ChatGPT-4o and Qwen mimicked loss aversion strongly regardless of whether the prompt claimed the behavior came from 43 people or 10,583.

The statistical analysis finds:
- a strong effect of the human behavior label (loss aversion vs gain seeking),
- a non-significant effect of sample size,
- and a non-significant interaction between them.

They conclude the mimicry is basically copying the described behavior, not performing the statistical inference you’d expect from learning under uncertainty.

What about the “EV cost” idea?

The authors also test whether mimicry weakens when bias harms expected value. Here the pattern is a bit more subtle. Mimicry wasn’t highest exactly at the point where loss aversion would be EV-neutral; instead, it was strongest near a point where the model’s existing bias makes indifference shift slightly. The paper frames it as sensitivity to the model’s own reference point, not just objective EV.

Parameter-level result: the loss aversion coefficient

They estimate a prospect-theory style loss aversion parameter λ for each agent. The reported averages include:
- For ChatGPT-4o: in loss aversion prompts, λ is about 1.257 / 1.218 (large/small samples), versus about 1.072 in gain-seeking.
- For Qwen: λ is about 1.742 in loss aversion regardless of sample size, versus about 0.940 / 0.929 in gain seeking.

Even when the magnitude differs between models, the mechanism is consistent: the prompt’s story about human bias becomes the model’s bias.

Study 4: Scientific Abstracts Can Become Self-Fulfilling Prophecies

This is where the paper gets especially punchy. Study 4 asks: what happens if you feed the LLM actual scientific writing that says humans are loss averse?

The authors use three meta-analyses (summarized in abstracts) with different conclusions:
- Brown et al. [22] reports strong loss aversion (mean λ = 1.96 for 150 studies).
- Walasek et al. [30] reports lower loss aversion (λ = 1.31 for 17 studies).
- Yechiam & Zeif [23] argues loss aversion is very weak/non-significant in some lotteries (λ = 1.07 in their re-analysis).

They compare those to a control condition where the LLM gets no scientific bias text.

Study 4a (abstracts only)

Both ChatGPT-4o and Qwen responded to the abstracts in the same direction: more bias claimed in the abstract → more loss aversion in the LLM’s subsequent decisions.

Even more telling, they show a monotonic relationship: loss aversion is highest when the abstract reports the strongest loss aversion and lowest when the abstract claims almost no effect.

The paper reports mean λ outcomes (noting this is the LLM’s estimated loss aversion parameter):
- Highest with Brown et al. [22]: ChatGPT-4o 1.785, Qwen 1.170
- Lowest with Yechiam & Zeif [23]: ChatGPT-4o 1.043, Qwen 1.047

They also find that the number of “accept” decisions differs substantially between the abstracts.

Study 4b (full texts)

They repeat the logic with full paper text for ChatGPT-4o (and not for Qwen, due to length limitations and performance degradation). The replication holds: the LLM’s loss aversion parameter moves with the literature’s claimed strength, even when the full scientific text is included.

For ChatGPT-4o, they report:
- λ = 1.519 when prompted with Brown et al. [22]
- λ = 1.082 when prompted with Yechiam & Zeif [23]

Why this is such a weird outcome

Because these are scientific papers describing human bias, you might expect a rational system to treat them as information about humans. Instead, the model treats them as instructions about how it should decide.

The paper calls this “paradoxical” and effectively describes a self-fulfilling prophecy: human studies highlight bias, those studies become training-like signals via the prompt, and the model displays the bias—even if it reduces expected value.

This is exactly the kind of failure mode that matters for real deployment: if the literature you feed an LLM is selective, contested, or method-dependent, the LLM can “inherit” those biases as if they were ground truth.

What “Expected-Value Minded Mimicry” Really Means for Prompting Safety

The authors don’t claim mimicry is total. In multiple places, they show it can weaken when mimicry is EV-disadvantageous. But it doesn’t fully go away.

This leads to a practical insight for anyone building with LLMs:

• Disclaimers alone won’t reliably debias the model. In fact, explicit bias language can increase copying (Study 4).
• Cost-based deterrence only works partially. The model still mimics even when it reduces EV (Study 1).
• Reliability indicators like sample size may be ignored. Study 3 suggests the model doesn’t automatically discount small samples the way humans (or proper Bayesian reasoning) would.

So if you’re prompting an LLM for decision-making in high-stakes settings, you may need more than “don’t be biased.” You might need constraints, structured comparisons, or evaluation frameworks that force trade-off reasoning across alternatives.

The paper notes a promising direction they tested in supplemental work: a “consider-the-alternative” style debiasing approach (joint evaluation across multiple lotteries) reduced bias effects when loss aversion conflicted with EV calculations—especially when gains and losses were equal.

That’s not a full fix, but it hints that prompt structure can matter more than prompt content.

Key Takeaways

• LLMs can mimic human biases even when there’s no logical basis for inference. Study 2 shows social proof behavior persists when the behavior described should not indicate true preference.
• Misreading behavior as preference evidence is a real mechanism. In Study 1 and 2, models infer preference from choices even when those inferences are invalid.
• Explicit “this is a bias” descriptions can still be copied. In Study 3 and Study 4, loss aversion emerges when it’s described as human bias.
• Sample size reliability doesn’t effectively debias mimicry. In Study 3, mimicry of loss aversion is about the same for n = 10,583 vs n = 43.
• Scientific summaries can become self-fulfilling prophecies for LLM decisions. In Study 4a, ChatGPT-4o loss aversion λ shifts from about 1.043 (near-zero loss aversion abstract) to 1.785 (strong loss aversion abstract).
• Expected-value cost reduces but doesn’t eliminate bias. Mimicry weakens when it reduces EV (e.g., Study 1: 37.5% at S=10 vs 17.0% at S=5), but remains significant.
• For safer prompting and decision systems, structure may outperform disclaimers. Joint evaluation / consider-the-alternative-style prompting can reduce bias when it conflicts with rational EV reasoning (discussed in the paper’s supplementary work).

If you want to explore the original research, start with the paper itself here: https://arxiv.org/abs/2608.12339.