Anthralytic’s Substack
Sign in

How do we research when AI strengthens qualitative interpretation, and when it reproduces, amplifies, or obscures human biases?

Six pieces in, the question still isn’t answered. This is what would answer it.

Anthralytic's avatar
Anthralytic
Nov 14

Part 6 of 6

This is the final piece in a six-part series on AI and qualitative interpretation. Part 1 asked the overarching question. Part 2 asked what a model matching human coders shows. Part 3 asked when working with AI exposes the assumptions a team shares. Part 4 followed the material from recording to finding. Part 5 asked how humans and AI might check each other. This piece brings the five questions together into an agenda for research, with a reading guide at the end.

Thanks for reading Anthralytic’s Substack! Subscribe for free to receive new posts and support my work.


Five pieces ago I asked a question I knew I couldn’t answer: when does AI strengthen qualitative interpretation, and when does it reproduce, amplify, or obscure human biases?

I still can’t answer it. What’s changed is that I can say precisely why not, and what it would take.

The four pieces in between took that question apart. Each pulled on a different part of it: what stronger interpretation would even mean and who gets to judge, whether working with a model exposes a team’s assumptions or buries them, whose meanings survive the journey from recording to finding, and whether humans and models can catch each other’s mistakes. None of those is a side question. Each has to have an answer before the main one can.

What the four subquestions turned up

Each question, and what came back.

What should count as stronger qualitative interpretation, and who gets to judge?

Agreement measures whether a framework was applied consistently. It can’t show the framework was right. And the reference the model is measured against is itself a set of judgments made by particular people on a particular day. When Liu and colleagues put human and model coding in front of a blind expert, preferences split almost evenly, and on several codes the authors found human consensus encoding a shared bias the verifier rejected. Three kinds of judge appear across these studies, the original coders, outside experts, and participants, and each has limits the others don’t fix.

When does working with AI expose shared blind spots, and when does it reinforce them?

In CoAIcoder, pairs sharing an AI model reached agreement faster and produced fewer distinct codes. Reflexis was built to push the other way, and its evaluation ran twelve researchers through a single session with no comparison condition. The problem underneath both: convergence that comes from genuine clarification and convergence that comes from absorbing someone else’s framing produce the same output. You can’t tell them apart from the result.

Whose meanings are preserved, changed, or lost as material becomes codes, themes, and findings?

Ashwin and colleagues found model coding errors that tracked the characteristics of the people being interviewed. Marston and colleagues found physical safety the hardest category for thirteen of the fourteen models they examined closely, and several models recoding discrimination as whatever service was denied. Nurkic and Ropp named documentation bias, where thin reporting makes an activity look like one where less happened. The losses aren’t random, and they fall hardest on accounts expressed indirectly.

Under what conditions can humans and AI identify and correct problems in each other’s qualitative interpretations?

Human review is named as the safeguard in nearly every practice account and specified in almost none. The IFAD protocol that made review possible, by tying every point to a timestamped quote, is the same protocol that narrowed what review could notice. At CGIAR, verification absorbed much of the time the tools saved, and junior staff did most of it. The reverse direction, AI finding problems in human work, has barely been tried.

Read together, the four answers point back at the same gap in the main question. Each describes a place where interpretation could go wrong without anyone noticing.

The thing I didn’t expect

One pattern turned up in all four subquestions, and I hadn’t anticipated it.

Nearly every instrument this field uses is built to detect the same thing: the model departing from human judgment. Agreement scores measure how often the two differ. Expert preference studies ask a judge which reading is better. Human review asks whether a person finds the model’s output acceptable. Audit trails confirm the model didn’t invent a quote. Every one of those fires when the model goes somewhere the humans wouldn’t. None of them fires when the model and the humans land in the same wrong place.

Fung’s evaluators caught the frequency problem because the output clashed with their sense of what mattered strategically. Nobody has shown what happens when it doesn’t clash, and part of the reason is that nothing has been built that could show it.

I don’t think this answers the overarching question. It’s a reason the question stays open. The tools we would reach for to answer it are mostly pointed somewhere else.

What the evidence establishes, and what kind of evidence it is

Three kinds of source have appeared in this series, and they carry different weight.

Four studies measure something. Ashwin and colleagues used real interviews with real stakes and found systematic error. Parfenova and colleagues showed that agreement with a reference and expert preference can point in opposite directions, on fifteen segments with three experts. Liu and colleagues ran the blind verification, with one verifier. Marston and colleagues benchmarked forty-six models, on synthetic transcripts. Two of the four are preprints.

Three studies examine interaction rather than output. CoAIcoder used a classifier trained on coding histories, not a generative model, with novice participants. Reflexis ran one short session on synthetic data without a baseline. Wheeler’s comparison across three software environments is a situated reflexive account by one researcher, which is what it claims to be.

The rest are practice accounts, nine of them from From Algorithms to Evidence. They describe what teams did and what the teams noticed. They don’t measure whether it worked, and they don’t claim to. That distinction gets blurred constantly in how these cases are cited, and it’s worth keeping sharp: an account of a careful workflow is evidence that careful workflows are possible, not evidence that the output was good.

There’s a structural limitation worth saying once, plainly. All four measurement studies are comparison studies. They compare a model’s output to a human reference. That design can’t detect shared error, because the reference is the thing in question. This isn’t a flaw in any individual study. It’s what you get when the only available benchmark is the judgment you’re trying to check.

And this literature is young. The core studies date from 2025 and 2026, the models they tested are already superseded, and the practice accounts describe workflows from 2024 and 2025. That argues for repeating the work rather than discounting it.

The study I want someone to run

So here is the study I’d like to see, in enough detail that someone could cost it.

Start with a completed evaluation and its real interview transcripts. Not synthetic ones. Synthetic transcripts can establish whether a model applies a codebook consistently, and they can’t establish whether it handles what people actually say, in the languages they say it in, with the indirection and hesitation real accounts carry.

Before any reviewer sees it, a panel of adjudicators who had no hand in the original analysis works through the findings and marks a set of claims, some well supported and some not. The errors need to vary in difficulty rather than all contradicting the source outright, because catching an obvious contradiction says little about catching a plausible overstatement. Where the adjudicators disagree with each other, that gets recorded rather than resolved away. A claim two experienced people read differently is a different object from one they both call wrong, and the difference matters.

Then recruit sixty or so qualitative researchers and evaluators and assign them at random across review arrangements: reviewing the analysis on their own, reviewing it after a model has flagged what it reads as unsupported, reviewing a model’s analysis of the same material, and an iterative version where each responds to the other. Equal time in every arm. Equal authority to change the finding, because a reviewer who can flag a problem but not fix it has produced a note rather than a correction.

Collect the process, not just the verdicts. Each reviewer’s initial reading before they see anything else. Every revision with the reason given for it. What they considered and rejected.

Score it blind, and count a correction only when it cites evidence from the interviews. That keeps the measure on whether the reviewer found something rather than whether they objected. Track what each arrangement caught, what it missed, what it flagged that was fine, and where reviewers declined to decide rather than deciding wrongly. Then follow the corrected and uncorrected findings into whatever the evaluation fed, because a missed claim that changes nothing matters less than one that redirects a program.

Two honest limits. This design suits codebook work and it’s the wrong frame for reflexive analysis, where there’s no single correct reading for the arms to converge on; that tradition needs a different question about whether the analyst engaged more deeply or less. And the study wouldn’t settle the overarching question. It would answer one part of it properly: whether a model in the loop helps a reviewer catch what they’d otherwise miss, or whether it mostly produces agreement faster.

Who decides, and whether they can

Every arrangement above assumes someone can act on what they find. That assumption deserves scrutiny.

Someone sets the analytical framework. Someone supplies the source material and decides what’s in the corpus. Someone can challenge an interpretation, and someone else can authorize the change. In the CGIAR account those roles came apart: the people doing verification were the most junior in the room, checking output a senior colleague had already found convincing.

The person best placed to notice an error may be the person least able to say so. That’s not a technical problem and no tool fixes it, but it determines whether any of the review arrangements above would work in a real organization.

What to do before the evidence arrives

None of this helps someone with a deadline next month. What follows isn’t a validity checklist, and I want to be clear that doing all of it wouldn’t make an analysis sound. It would make the analysis checkable, which is a lower bar and one most work currently misses.

Record the model and its version, with dates. Models change behavior under the same name. Keep the prompts. Say what the corpus contains and what it excludes. Flag where documentation was thin, so nobody mistakes an absence of reporting for an absence of activity. Where the method calls for independent readings, preserve them before anything is shared. Log revisions and the reasons for them. And say who reviewed what, with what access to the sources and what authority to change the result.

A year from now, that record is what lets anyone, including you, work out whether the analysis held up.

Back where we started

This series opened with a model that mistook how often something was said for how much it mattered, and with evaluators who corrected it. I said then that I couldn’t tell you how we’d know the evaluators were right. Five pieces later I still can’t.

The study above would start to tell us. I’d like to see it run, by someone with the transcripts and the reviewers to do it properly, because the alternative is what we have now: asking a model and a team whether they agree, and learning only that they agree.

Leave a comment

Share

Anthralytic builds evaluation tools for organizations that can’t afford an evaluator on staff. Evaluability checks, theory of change builders, and planning assistants, designed so the people doing the work can do their own evaluation, with an evaluator in the loop when it counts.


A reading guide

Organized by what each source helps you investigate rather than by type.

Whether a model’s errors are random

Ashwin, Chhabra, and Rao (2025), Using large language models for qualitative analysis can introduce serious bias. Real interviews with Rohingya refugees and their Bangladeshi host community; coding errors tracked speaker characteristics. Open access. Limit: measured against human codes, which can’t show how biased the humans were.

What agreement actually measures

Parfenova, Marfurt, Pfeffer, and Denzler (2025), Text annotation via inductive coding. Reference agreement and expert preference diverge. Limit: fifteen segments, three experts.

O’Connor and Joffe (2020), Intercoder reliability in qualitative research. The case for reliability, made by people who also name its limits, including interpretive convergence. Open access.

Whether the human reference is sound

Liu and colleagues (2026), Agreement is not quality. Blind expert verification of human and model coding. The one study that puts human consensus on trial. Limit: one verifier, preprint.

What models miss, theme by theme

Marston and colleagues (2026), Can large language models reliably code qualitative humanitarian data?. Forty-six models; physical safety and indirectly expressed needs were the hardest. Limit: synthetic transcripts, preprint.

Whether agreement is the right test at all

Braun and Clarke (2021), One size fits all?. Why coding reliability makes no sense in reflexive thematic analysis. Author copy.

How teams surface their own assumptions

MacQueen, McLellan, Kay, and Milstein (1998), Codebook development for team-based qualitative analysis. The structured version.

Barry, Britten, Barber, Bradley, and Stevenson (1999), Using reflexivity to optimize teamwork. A team writing and sharing reflexive accounts.

Olmos-Vega, Stalmeijer, Varpio, and Kahlke (2023), A practical guide to reflexivity. The four dimensions. Open access.

How the collaboration arrangement changes the result

Gao and colleagues (2024), CoAIcoder. Shared AI support sped agreement and narrowed code diversity. Limit: a classifier, not a generative model; novice participants.

Ye and colleagues (2026), Reflexis. Designed to support deliberation rather than suggest codes. Limit: one session, synthetic data, no baseline.

How the software shapes what you can see

Wheeler (2026), Technological reflexivity in practice. MAXQDA, NVivo, and ChatGPT on the same data. Open access. A situated account, not a controlled comparison.

Why models agree with you

Sharma and colleagues (2023), Towards understanding sycophancy in language models. Where the tendency comes from and why training reinforces it.

Practice accounts, all in From Algorithms to Evidence

These describe what teams did, not whether it worked.

Fung, on frequency mistaken for significance. Nielsen and colleagues, on a bounded labelling task where agreement was the right test. Damme, on a rubric for assessing whether a tool supports or erodes critical thinking. Beyene, on a solo evaluator using a model as a thinking partner. Nurkic and Ropp, on documentation bias and an Accept/Edit/Discuss review protocol. Smith and Kraft, on the tension between letting clients speak and producing actionable insight. Naidoo, Sharma, Jonckheere, and den Boer, on protocol-driven extraction at IFAD and what the protocol cost. Corsetti, Asare, and Cekova, on verification labor at CGIAR. And my own chapter, on AI-assisted evaluation planning with a grassroots nonprofit.

orange and blue We Can Do It poster
Photo by Mariana Villanueva on Unsplash

Discussion about this post

User's avatar

Keep reading

Anthralytic’s Substack

Anthralytic Substack

  • About
  • Archive
  • Recommendations
  • Sitemap
© 2026 Anthralytic · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for writing, podcasting, and media founders.