Demographic Signals Survive Anonymization in German LLM Résumés

Demographic Signals Survive Anonymization in German LLM Résumés

Hiring is getting a lot more automated—especially in the résumé stage. And while many teams focus on removing names and obvious demographic markers, new research from Leininger et al. suggests something uncomfortable: even “anonymized” German résumé text generated by LLMs can still leak gender information through subtle word choices.

This study (based on the paper above) runs a two-stage audit on German-language résumés created with major LLMs. First, the researchers generate résumés from real, already-anonymized job-matching profiles—but they vary the names to represent different genders and ethnicities. Then they simulate a downstream screening pipeline that tries to anonymize and “gender-neutralize” the generated text again, before training classifiers to see whether demographic signals are still recoverable.

The result: gender leakage remains strongly detectable across models—even after anonymization and gender-neutralization steps. Ethnicity-related leakage, in contrast, is comparatively weak. The big twist is why this happens: the leakage isn’t mostly from overtly gendered wording. It comes from small, German-specific lexical preferences between seemingly equivalent terms.

Why “Just Anonymize It” Fails Fast in Real Hiring Workflows

This research matters right now because hiring pipelines are moving from “AI as a suggestion” to “AI as a gate.” Once an LLM helps structure, summarize, or rewrite a résumé, the output isn’t just text—it becomes an input to ranking, screening, or feature extraction. Regulators and companies often respond to fairness concerns with anonymization, but anonymization is usually treated like a switch: remove identifiers → demographic influence is gone.

This study challenges that assumption—specifically for German. In German, grammar and vocabulary can carry signals even when you remove explicit markers. Think of it like removing a person’s name from a voicemail: you might still recognize who it is from cadence, vocabulary, and phrasing. Here, the “voice” is the LLM’s language choices.

A concrete today scenario: imagine an HR vendor offers an applicant-facing tool like “Generate a professional résumé from your answers.” Applicants paste their structured profile, and the tool outputs a résumé in perfect German. Later, the employer runs an anonymization step before screening (names removed, text processed), expecting fairness. This paper suggests that the employer may still unintentionally screen through latent gender cues embedded in the generated wording.

And importantly, the work builds on earlier AI fairness research by shifting the target from the decision model to the generated document itself. Prior studies often analyze bias in rankings or predictions. This audit asks: what if the résumé text already contains demographic information that downstream models can recover—regardless of what the screening system claims to remove?

How the Researchers Built a Controlled Test of Résumé Demographic Leakage

The audit design is careful because it isolates demographic effects from everything else. The researchers didn’t hand-write resumes; they used LLMs to generate résumé text from real data—then controlled the demographic “variable” through names only.

Real anonymized profiles, demographic variation through names only

The résumé generation input comes from German job-matching profiles provided by Chemistree GmbH. These profiles are anonymized and contain no personal identifiers such as names, age, gender, or ethnicity. The dataset contains 5,932 profiles, of which 3,542 are fully completed. For the experiment, the researchers sampled 30 profiles (Np = 30) for the controlled generation setup.

To systematically add demographic variation, they created an external set of 40 names (Nn = 40) and paired each profile with each name, creating 1,200 profile–name combinations per model (30 × 40 = 1,200). In other words: the qualifications and profile answers stay fixed; only the demographic-associated name changes.

Gender was treated as binary and inferred from names (male vs. female). Ethnicity was binary as well (German vs. Turkish-associated names), focusing particularly on Turkish-associated names because they represent a large migrant-origin population in Germany and previous labor market studies report persistent discrimination.

The LLM lineup: proprietary + open-weight, multiple scales

They ran the first stage using:
- ChatGPT (GPT-4o-mini)
- Gemini 2.5 Flash-Lite
- Qwen 3 open-weight models at 4B, 8B, and 14B

They kept generation settings mostly consistent. Temperature was fixed at 1 for all models. For sampling behavior, Gemini used a fixed top_k = 64. For Qwen 3, they tested both top_k = 20 and top_k = 64 to check whether leakage is robust to decoding choices.

Each LLM was prompted to output a realistic German résumé in a structured schema (sections like competencies, work experience, education, languages, interests, plus a target position and a short cover-letter snippet).

You can think of this stage as a “résumé factory” where the inputs are constant except for one dial: the name. If the name is associated with gender (or ethnicity), does the output résumé still carry that signal—especially after later attempts to erase it?

What the Downstream Screening Audit Simulated (and Why It’s the Right Stress Test)

Stage two of the audit is where the fairness question gets real.

Instead of modeling a complete hiring pipeline, the researchers simulate a common fairness intervention: an anonymization + normalization step before screening. Then they test whether demographic information is still recoverable from the processed résumés.

Step 1: Remove obvious identity fields and names

They remove the top-level personal data fields from the generated JSON—things like names, contact details, email addresses, and LinkedIn profiles. They also remove names wherever they appear in other sections.

This step handles the “easy” demographic signals. But the paper’s point is that even when you wipe out explicit identifiers, indirect signals can remain.

Step 2: Gender-neutralize German lexical gender markers

German contains grammatical gender and gendered variants of job nouns (for example, feminine forms ending in -in). To prevent a classifier from cheating using overt grammar cues, the researchers apply a rule-based gender-neutralization procedure.

Examples include:
- Mapping feminine suffix forms like -in to a neutral/canonical counterpart (e.g., Beraterin → Berater)
- Collapsing gender-marked lexical pairs into a shared representation (e.g., Kaufmann/Kauffrau and Fachmann/Fachfrau become normalized forms)

After that, they do standard text preprocessing:
- Remove German stop words
- Lemmatize using spaCy German models
- Restrict features to unigram tokens
- Discard rare terms (keeping only words appearing in at least 10 résumés)

Step 3: Train TF–IDF logistic regression classifiers to “recover” demographics

Now for the audit test: the researchers train classifiers to predict demographic attributes from the processed résumé text.

They represent résumés using TF–IDF features (unigrams), chosen partly because it’s easier to trace which words are driving predictions.

The classifier is L1-regularized logistic regression (sparse feature selection, also helping identify the most consistent leakage indicators). They evaluate performance using:
- Accuracy (ACC)
- Area under ROC curve (AUC)

In this setup, ACC = 0.50 and AUC = 0.50 mean random guessing—i.e., no detectable demographic leakage.

They also use grouped splits so that résumés derived from the same original matching profile don’t leak into both train and test sets. That’s important: otherwise, the model could learn profile-specific quirks instead of demographic patterns.

Why this approach is a strong fairness audit

This is essentially a “can someone infer it anyway?” test. If classifiers can reliably recover gender (or ethnicity) from text after anonymization, then anonymization alone didn’t remove the underlying signal—only the explicit labels.

That aligns directly with the fairness logic behind the EU AI Act’s high-risk classification for employment-related systems (as discussed in the paper): transparency and safeguards matter, but you also need to validate that interventions actually reduce sensitive information access.

The Core Findings: Gender Leakage Is Robust, Ethnicity Leakage Is Weak

Here’s the headline result from the paper’s screening audit: gender information remains highly recoverable after anonymization and gender-neutralization, while ethnicity-related leakage stays near chance.

Gender leakage: high and consistent, driven by lexical choice (not obvious gender words)

Across all model families, gender leakage is strongly detectable. The paper reports mean classifier performance for gender recovery, with values substantially above random guessing for all models—especially ChatGPT and Gemini.

Below is a compact view of the performance described in the results:

Even more telling is what words the classifier relies on. The leakage isn’t mostly coming from explicit gendered wording, because the researchers neutralized gender-marked forms first. Instead, the classifiers latch onto subtle differences between semantically equivalent terms.

A standout example is the normalized term:

• Fachkraft (gender-neutral specialist/general workforce term in German)

The paper reports that Fachkraft is selected in all 30 runs and is consistently associated with female-associated résumés across model families.

Meanwhile, male-associated résumés tend to include alternative lexical choices, such as:
- Fach-mann/-frau emerging in all 30 ChatGPT runs (a gendered variant label collapsing into a normalized representation)
- Profi (short for professional) appearing in Gemini outputs

Importantly: these selected terms are either already gender-neutral after preprocessing, or they reflect the model’s preference among overlapping German alternatives, not simple leftover explicit gender grammar.

Ethnicity leakage: comparatively weak and less stable

Ethnicity-related leakage is much weaker. Classifier performance stays close to random guessing for most models, with only small deviations.

The paper reports:
- ChatGPT: essentially chance-level (ACC and AUC close to 0.50)
- Gemini: small deviation above random (ACC = 0.521, AUC = 0.540)
- Qwen 8B: also close to chance

Selected ethnicity-associated terms appear less stable than gender terms and have smaller coefficient magnitudes. The paper highlights examples such as:
- Muttersprache (mother tongue), especially in phrases like Türkisch (Muttersprache) for ChatGPT
- Austausch (exchange), often in contexts like interkultureller/internationaler Austausch for Gemini
- kulturell (cultural) for Qwen 8B, but only in fewer than half the runs

So even when ethnicity signals are present, they seem less reliably encoded and less consistently recoverable.

Why gender is easier to leak than ethnicity in this setup

This “gender leaks, ethnicity barely leaks” pattern makes sense in light of the experimental design and German language behavior:

• Gender was varied through names and is likely to influence LLM phrasing habits very directly during generation.
• Ethnicity signals might be present only in more context-dependent ways (e.g., language, “mother tongue,” cultural exchange), which the models may not reproduce as consistently—especially when the structured profile answers are the same.

The paper explicitly frames the leakage as driven by German-specific lexical alternatives, which is a plausible mechanism for gender recovery even after gender-neutralization rules.

Does Model Size Change the Leakage? (Sometimes, but not in a simple way)

The researchers also investigated how leakage behaves across Qwen 3 model sizes and decoding settings.

Key pattern: gender leakage is detectable across sizes; ethnicity stays near chance

They report that:
- Gender leakage remains detectable across Qwen 3 sizes and top_k settings.
- Ethnicity prediction stays largely near chance level.

The non-monotonic behavior is the interesting part

For gender, the relationship with model size isn’t monotonic. Accuracy increases with model size, but AUC peaks at 8B and slightly declines at 14B.

They also note that increasing top_k generally reduces both accuracy and AUC for gender leakage. For ethnicity, there’s no consistent scaling pattern; the 14B model shows the strongest deviation described, with ACC = 0.566 and AUC = 0.579 at top_k = 20.

So scale isn’t a simple “bigger model = worse fairness” slider. Leakage seems tied to how specific lexical preferences form in each model, not just raw capability.

Practical Implications: What Hiring Teams, Vendors, and Applicants Should Do Next

This is the part that matters for anyone building or using AI résumé tools.

If you rely on anonymization as a fairness fix, you may need a second layer

The paper’s conclusion is blunt: anonymization-based interventions may not fully prevent demographic leakage in multilingual AI hiring pipelines.

That means HR vendors and platform builders can’t stop at “remove names.” They may need:
- Detect-and-neutralize gender-associated lexical variation (German-specific)
- Evaluate fairness interventions using downstream recovery tests, not just manual spot checks
- Treat LLM résumé generation as part of the “model system,” not just a formatting step

A specific engineering direction: test “leakage recoverability,” not just demographic parity claims

One practical takeaway from the audit design is the evaluation method itself: train a simple recovery model (like TF–IDF logistic regression) on anonymized outputs and measure how close performance stays to random guessing.

This kind of stress test can catch cases where “neutralized text” still encodes protected attributes via word choice.

For applicants using AI résumé assistants

The paper suggests a simple mitigation: omitting demographic-related names and other direct demographic markers from prompts may reduce the risk of unintended demographic encoding in the output.

In other words, even if the user intends to be fair, feeding gender-coded names into an LLM may cause the model to write “different but subtly equivalent” résumé phrasing. If your goal is privacy/fairness, that upstream input matters.

Important nuance: “leakage” is a prerequisite, not proof of discrimination

The paper is careful about interpretation: recoverable demographic information doesn’t automatically mean the hiring system will discriminate. It means a downstream system could use those signals.

Whether hiring decisions actually become biased depends on:
- what features the screening system extracts,
- how it’s trained,
- and how prediction is converted into rankings or outcomes.

Still, if the leakage is strong (like gender here), it’s a real risk factor worth addressing.

Key Takeaways

• Gender demographic leakage persists even after anonymization and gender-neutralization of German LLM-generated résumés.
• In the screening audit, gender recovery stays well above random guessing across multiple models (especially ChatGPT and Gemini), while ethnicity leakage remains much closer to chance.
• The leakage is not mainly driven by overt gendered wording. It comes from subtle lexical preferences among semantically overlapping German terms (e.g., strong associations involving Fachkraft).
• German language specifics matter: grammatical gender and gender-marked lexical alternatives create additional pathways for indirect demographic encoding.
• Model size and decoding settings don’t create a simple trend, but gender leakage remains detectable across Qwen 3 sizes and top_k values (with non-monotonic AUC behavior).
• For hiring platforms and vendors: anonymization alone isn’t enough—you need validation that demographic signals can’t be recovered from generated text.
• For applicants: avoid including gender-coded names in prompts if you want to reduce the chance of the résumé assistant encoding demographic cues.