Five Ways AI Breaks in Social Programs
And how we can stop making the same mistakes—just faster.
AI is already shaping how social impact organizations do their work.
It’s deciding who gets food, housing, healthcare, and legal aid. It powers tools that predict need, flag risks, and distribute scarce resources. Refugee families are assigned aid through algorithms. Students are routed by scoring systems. Clinics use predictive models to plan services.
But here’s the problem: a lot of these systems just aren’t safe.
Not just in the “someone hacked our data” kind of way—though that’s part of it. These systems are vulnerable in ways specific to social programs. They’re shaped by real-world dynamics: families doing what they need to survive, staff stretched thin and working around rigid systems, and policies nudged by politics, not just data.
This isn’t a hypothetical risk. It’s already happening.
I remember doing a site visit for a grant program where we’d selected participants based on standard scoring criteria—data points like income, narrative strength, and fit. One participant looked like a great choice on paper. But when I got there, I found a multi-story building under construction. It turns out they had more wealth than me and most of the people I worked with in my global development organization.
This wasn’t AI. It was just the same kind of data we’ve always used. But it missed the context. The risk with AI is that we take those same blind spots and scale them—faster, further, and with less room to course-correct.
This Is Already Happening
By 2024, more than three-quarters of organizations, including those in international development, reported using AI in at least one function1. Governments are deploying algorithms in everything from welfare to criminal justice. The “AI for Good” market is growing fast.
But most of these systems wouldn’t survive in the private sector:
Child welfare tools flag too many false positives2
Education platforms encode bias into how students are tracked3
Health allocation models look great in pilots but fall apart in practice4
In tech, that kind of failure gets flagged.
In social impact, it gets normalized.
So it’s no surprise that some organizations are responding by banning AI altogether. And I get the impulse—when the tools are risky and the stakes are human lives, “no thanks” can feel like the safest option.
But that avoidance has its own cost.
We need smarter tools to manage complexity, reach underserved communities, and make better decisions.
Shutting AI out completely means giving up on progress we actually need.
The goal isn’t to reject AI or blindly adopt it.
It’s to build systems that actually work—and don’t collapse under pressure.
What This Series Covers
This is the first in a five-part series on how AI breaks in social programs—and what we can do about it.
Subscribe to follow along as we unpack the real-world risks (and design opportunities) across:
Data Vulnerabilities – how bad data (or bad incentives) lead to bad decisions
Infrastructure Gaps – what happens when the context doesn’t support the tech
Human Factors – how people override, misinterpret, or work around AI systems
Regulatory Confusion – who owns the data, who sets the rules, and what gets lost in translation
Systemic Design Flaws – why the models fail to hold up—and how to make them antifragile
We’ve been making high-stakes decisions with imperfect data for years. AI just scales the risk—and speeds up the failure.
Core series and related posts
Five Ways AI Breaks in Social Impact (Intro to the Series)
1 of 5: Social Impact Data Vulnerability in Three Acts - (Article)
2 of 5: How AI digital infrastructure breaks in social impact - Is your New $15,000 Data System a Glorified Clipboard? (article)
3 of 5: How People Break AI in Social Impact - Gaming, Bias, and the Human Element(Article)
4 of 5: Data Privacy, Sovereignty, and Surveillance in Social Impact - Avoid the AI Regulatory Trap: A Quick Cheat Sheet for Privacy, Sovereignty & Surveillance Risks
Pods
Social Impact Data Vulnerability in Three Acts (Podcast version)
How fragile data, weak governance, and good intentions collide—and how to stop AI from multiplying the risk.
Is Your $15,000 Data System a Glorified Clipboard? (Podcast version)
Why infrastructure assumptions (power, sync, support) turn “smart” systems into expensive notebooks—and how to design for the field, not the demo.
How People Break AI in Social Impact (Podcast)
Why gaming and “helpful” tweaks tank models—and how to close the feedback loop without breaking trust.
Side Quests
Side Quest #1: Choose Your AI Tool Wisely (article)
Pick tools that fit your context, not a vendor deck.
Side Quest #2: The Sync Loop of Doom (article)
Side Quest #2: The Sync Loop of Doom (podcast)
Why online-only workflows crumble in low connectivity—and how offline-first plus staged sync keeps programs running.
Side Quest #3: GPT OSS Breaking News Edition
Side Quest #3: GPT OSS Breaking News Edition (podcast)
What open-source LLMs mean for privacy, control, and on-device AI in low-resource settings.
Side Quest #4: The Side Quest that Changes the Game (podcast)
Side Quest #4: The Side Quest that Changes the Game (article)
Tech Tyrant, Human Wildcard, Context Colossus—and how to survive them together.
Anthralytic is a strategy and evaluation firm using human-centered design and AI tools to help mission-driven organizations make smarter, safer decisions.
We help nonprofits, governments, and funders move from overwhelmed to evidence-informed—with tools that actually work in the real world.
https://hai.stanford.edu/ai-index/2025-ai-index-report?utm_source=chatgpt.com
https://childwelfaremonitor.org/2023/04/05/using-algorithms-in-child-welfare-promise-confusion-and-controversy/?utm_source=chatgpt.com
https://www.businessinsider.com/nonprofits-ai-tools-edtech-global-inequities-2025-7?utm_source=chatgpt.com
https://inflecthealth.medium.com/the-hidden-pitfall-in-healthcare-innovation-why-brilliant-pilots-fail-to-scale-c98ae51b74f3