Sex Bias in AI Clinician Reasoning: How Large Language Models Mirror Medical Stereotypes

Table of Contents

• Introduction
• Why This Matters
• Main Content: What the Study Found
  • Sex Bias Across LLMs and Temperatures
  • Abstention as Guardrail—and Its Limits
  • Downstream Diagnostic Reasoning: How Sex Shifts Top Diagnoses
  • Why This Happens: Architecture, Data, and Stereotype Completion
• Practical Implications: Safer Deployment in Healthcare
• Key Takeaways
• Sources & Further Reading

Introduction

Biased data begets biased outputs. That’s a core worry when large language models (LLMs) start assisting in health care—everything from patient notes to decision support. A new line of research digs into a crucial question: do contemporary general-purpose LLMs carry sex-based biases into clinical reasoning, and if so, under what conditions do those biases show up? The study, titled Evaluating the Presence of Sex Bias in Clinical Reasoning by Large Language Models, puts modern AI to the test with clinician-authored clinical vignettes, examining how models assign sex, when they abstain, and how sex differences ripple into the final differential diagnoses. This work, conducted by researchers at Oxford and Australian institutions, provides a timely look at whether AI tools in health care might inadvertently reinforce stereotypes or drift toward biased clinical inferences. For a full dive, you can check the original paper here: Evaluating the Presence of Sex Bias in Clinical Reasoning by Large Language Models.

In short, the researchers asked four widely used general-purpose LLMs to work on sex-neutral clinical scenarios and then on male- versus female-version vignettes. They looked at three experimental angles: default sex assignment when sex isn’t stated, the option to abstain from sex inference, and how the presence of male vs. female sex information would shape the top five differential diagnoses. The findings are striking: model architecture and the way these systems were trained appear to encode sex-based priors that show up as bias in clinical reasoning—often in ways that mirror long-standing medical stereotypes.

Why This Matters

This topic lands at the intersection of AI safety, health equity, and practical clinical care. Why now? Because AI tools are getting woven into everyday health workflows, from drafting notes to offering diagnostic suggestions. If these tools reproduce or amplify sex-based biases that already show up in human health care, they risk influencing real patient care decisions. And because LLMs are trained on vast, mixed-language data from the public web and professional text, they can reflect and magnify society’s biases unless we actively design around them.

A real-world scenario helps ground this: imagine an AI-assisted triage system in an emergency department that uses a general-purpose LLM to generate a short list of possible diagnoses. If the model tends to assign “female” labels more readily in certain specialties (say psychiatry or endocrinology) and “male” labels in others (cardiology, urology), the triage outputs could subtly steer clinicians toward certain pathways or distract from others. Even when prompts explicitly avoid sex cues, downstream reasoning still exhibits sex-contingent differences, underscoring a stubborn, model-internal bias rather than a superficial prompt artifact.

How does this fit with prior AI research? It starts where many bias studies in AI end: performance benchmarks can look solid even when the system leans on biased associations. Earlier work has shown that ChatGPT-like models can pass medical licensure-style tasks, but successful benchmarking doesn’t guarantee fair or safe clinical reasoning. This study adds a layer—showing that biases aren’t just about accuracy; they’re baked into the reasoning process and can persist across models, prompts, and even abstention safeguards.

If you want to explore the study in detail, the original paper is a must-read: Evaluating the Presence of Sex Bias in Clinical Reasoning by Large Language Models.

Main Content: What the Study Found

Sex Bias Across LLMs and Temperatures

The study ran three experiments across four general-purpose LLMs (ChatGPT, Claude, Gemini, and DeepSeek) using 50 clinician-authored vignettes spanning 44 specialties. The key aim was to test sex as a non-informative variable in the initial diagnostic pathway and observe if and how models label sex and generate differential diagnoses.

• Binary sex assignment on neutral vignettes (no sex cues) varied significantly by model:
  • At temperature 0.5, ChatGPT assigned female sex in about 70% of cases; DeepSeek ≈ 61%; Claude ≈ 59%; Gemini showed the opposite pattern with a male skew (female assignments ≈ 36%).
• Temperature effects were surprisingly limited on the overall sex assignment pattern, but interactions between model and temperature were statistically significant. In other words, temperature didn’t flip the bias direction on its own, but it did shape how strongly each model expressed its biases.
• Specialty context mattered a lot. Across models, psychiatry, rheumatology, and hematology were labeled female in nearly all cases, while cardiology and urology were labeled male in all cases. Pulmonology leaned male in most models too, though Claude displayed a more balanced pattern.

This pattern—model-specific bias directions coupled with strong specialty hotspots—suggests that LLMs don’t just reflect a neutral statistical world; they inherit and reproduce gendered patterns embedded in medical discourse and training data. It’s not that they’re randomly guessing; they’re showing structured priors that line up with real-world stereotypes and representation gaps in medicine.

Abstention as Guardrail—and Its Limits

The second experiment introduced an abstain option on the same neutral vignettes. The idea is straightforward: if the model isn’t confident about inferring a patient’s sex, it can abstain rather than guess.

• Abstention had a dramatic surface effect. ChatGPT abstained in 100% of cases across temperatures. Claude and Gemini abstained in 84% and 80% respectively at 0.5, and DeepSeek abstained in about 58%. So, when given a chance to opt out, many models prefer not to reveal a sex label.
• Yet abstention did not erase downstream bias. Even with abstentions, the models still produced sex-contingent diagnostic differences when asked to generate differential diagnoses for male vs. female vignettes.
• The takeaway? Abstention is a useful guardrail to reduce explicit demographic labeling, but it’s not a cure for implicit bias embedded in internal reasoning pathways. Prompt design can reduce surface labels, but it cannot fully neutralize the deeper associations the model has learned.

This has practical implications for deployment: if you want to minimize explicit demographic leakage, enabling abstention helps, but you also need downstream safeguards and auditing to ensure clinical outputs don’t drift toward biased inferences.

Downstream Diagnostic Reasoning: How Sex Shifts Top Diagnoses

Experiment three looked at how explicit sex information affected the five-diagnosis lists models produced, ranking those diagnoses by likelihood.

• All models produced divergent outputs for male vs. female vignettes. At temperature 0.5:
  • Claude showed that about two-thirds of its outputs differed in content and order between the two sexes, with roughly two-thirds of results diverging in both what diagnoses were listed and in which order.
  • ChatGPT also showed substantial divergence (about two-thirds of outputs differed in content and order), but its lists were more often completely or partially different than the other models.
  • DeepSeek and Gemini tended to produce similar lists more often, with a substantial share of identical content or only reordered items.
• Similarity metrics tell a clear story. Across models, the Jaccard similarity (overlap of diagnoses) hovered around the high 0.6s to 0.8s at lower temperatures, but dropped as temperature rose, indicating that higher sampling variability amplified sex-contingent differences.
• Notably, the degree of diagnostic divergence between male and female lists varied not only by model but also by specialty, reinforcing the idea that these biases align with existing stereotypes in medicine, rather than reflecting a universal, neutral diagnostic process.

In essence, even when a model is prompted to treat male and female vignettes the same, the internalized priors and pattern recognition learned during training push the model toward different differential diagnoses. The variance across models underscores that bias is not a single, intrinsic property of “AI,” but a product of training data, alignment choices, and architecture.

For readers who want to see the nitty-gritty, the study provides a spectrum of metrics: Jaccard similarity, item-level agreement, cumulative match characteristics, and Kendall’s tau—each capturing a facet of how closely the two sex-based lists align. Linkage to the original work is here for the curious: Evaluating the Presence of Sex Bias in Clinical Reasoning by Large Language Models.

Why This Happens: Architecture, Data, and Stereotype Completion

The authors propose a threefold explanation for the observed pattern:

• Model architecture and training signals dominate bias direction. Different commercial models encode distinct sex priors based on how they were trained, what data they were exposed to, and how they were aligned. This explains why ChatGPT, Claude, and DeepSeek leaned toward female labeling, while Gemini consistently leaned male.
• Specialty-level skew mirrors entrenched medical stereotypes. When models operate in fields like psychiatry or cardiology, long-standing societal narratives about gender roles and disease prevalence appear to seep into the outputs. The study notes that some specialties show near-complete sex labeling bias across models, suggesting a deep resonance with biased discourse rather than a purely data-driven neutral reasoning.
• The bias is not simply prevalence-based. While some conditions do show sex-based prevalence differences in epidemiology (e.g., higher rates of certain mood disorders in women), the near-complete, invariant bias across models and prompts cannot be explained by base rate alone. Instead, the phenomenon looks more like stereotype completion embedded in linguistic patterns the models learned from text, rather than a faithful reflection of population statistics.

The takeaway for developers and researchers is clear: if you want to reduce bias, you’re not just tuning a temperature or tweaking a prompt—you’re tackling training data, alignment objectives, and the ways models internalize domain knowledge. This aligns with broader calls in AI fairness to audit models at the specialty level and to design systems that explicitly penalize stereotype completion when clinical cues don’t justify it.

Practical Implications: Safer Deployment in Healthcare

What should clinicians, health systems, and AI developers take away from these findings? A few practical takeaways can guide safer, more transparent deployments of AI tools in care settings.

• Prefer domain-specific auditing and governance. The study shows that bias hotspots cluster around certain specialties. Health systems should implement specialty-level fairness audits and maintain clear documentation of model defaults, versions, and abstention behavior. This isn’t just about ethics; it’s about reproducibility and patient safety.
• Standardize prompts and guardrails. Prompt design matters. Consistent prompts, explicit limitations, and transparent uncertainty statements can help reduce the chance that a model’s outputs are misinterpreted as clinical advice. When abstention is allowed, it should be clearly surfaced and accompanied by human-oversight pathways for uncertain cases.
• Use abstention strategically, not as a substitute for human oversight. While abstaining reduces overt demographic labeling, it does not neutralize downstream reasoning differences. Clinical workflows should route high-risk queries to human clinicians, especially when outputs would influence diagnostic pathways or treatment decisions.
• Balance model choice with safety objectives. The research shows model architecture largely drives bias direction. If the goal is safer, more neutral assistance, teams should evaluate multiple models, compare bias patterns, and consider domain-adapted variants or models tuned for clinical use that minimize stereotype-driven inferences.
• Open transparency with users. Clinicians and patients deserve clarity about model limitations, default behaviors, and how outputs should be interpreted. Providing links to evidence-based resources and publishing uncertainty can help prevent overreliance on AI-generated suggestions.

If you want to see more about the broader research context and the methods used, the original study is a rich resource: Evaluating the Presence of Sex Bias in Clinical Reasoning by Large Language Models.

Key Takeaways

• Four widely used general-purpose LLMs show sex-based bias in clinical reasoning when asked to analyze sex-neutral vignettes, with model-specific directions (e.g., female-biased labeling by ChatGPT, Claude, and DeepSeek; male bias by Gemini).
• The direction of bias is highly model-dependent and is amplified in certain medical specialties, particularly psychiatry, rheumatology, hematology (female labeling) and cardiology, urology (male labeling); pulmonology tended to be male across several models.
• Allowing abstention reduces explicit sex labeling but does not eliminate downstream differences in diagnostic reasoning. Abstention is a helpful guardrail, not a cure.
• When asked to generate top-five differential diagnoses for male vs. female vignettes, outputs diverged in content and ranking for a majority of cases, with similarity metrics decreasing as sampling temperature increased.
• Temperature mainly increased variability rather than flipping bias direction; model architecture and training data drive bias more than sampling randomness.
• The findings argue against using general-purpose LLMs to guide diagnoses or treatment decisions in clinical settings. They call for safer configuration, explicit uncertainty, and continuous human oversight—along with broader human- and domain-specific auditing.
• Practically, this work reinforces the push for domain-tuned, bias-aware AI in health care and highlights the need for transparent governance, benchmarking, and ongoing monitoring as AI becomes more embedded in patient care.

Sources & Further Reading

• Original Research Paper: Evaluating the Presence of Sex Bias in Clinical Reasoning by Large Language Models
• Authors:
  • Isabel Tsintsiper
  • Sheng Wong
  • Beth Albert
  • Shaun P Brennecke
  • Gabriel Davis Jones

For readers who want the precise numbers, methodological details, and the full suite of analyses (including tables and figures), the paper provides a comprehensive appendix and data resources. If you’re a healthcare professional, AI researcher, or policy maker, this work is a timely reminder that the tools we rely on in patient care carry the weight of human biases—biases that we must actively address through design, governance, and vigilant clinical practice.