In February, Virtuous and Fundraising.AI released the 2026 Nonprofit AI Adoption Report. Gabe Cooper, the CEO of Virtuous, used the launch to call the question of whether nonprofits should use AI largely settled, and to say the real work now is rethinking workflows. In the same press release, the company’s chief AI officer, Nathan Chappell, described the organizations pulling ahead as the ones integrating AI into how decisions get made. That, he said, is where capacity begins to expand.

Two months later, the same Nathan Chappell told the Chronicle of Philanthropy that the productivity paradox is “one of the things I worry about the most with AI in our sector.” His worry was specific. A twenty-dollar tool that makes a worker twenty percent more productive looks like a free extra day of labor, and the temptation is to keep loading that worker until the job no longer fits one person. He named the risk by its real name. Retention.

Same person. Different room. The February version is the one that traveled. This piece starts from the April one.

Subscribe now

The numbers are not the surprising part

The report surveyed 346 nonprofits in late 2025. Ninety-two percent use AI. Seventy-nine percent report small to moderate efficiency gains. Seven percent report major improvements in organizational capability. Eighty-one percent use AI individually, without shared workflows. Forty-seven percent have no AI governance policy. The report calls the space between the ninety-two and the seven an efficiency plateau.

The dominant reading is that ninety-three percent of the sector has an integration problem. The seven percent did the hard work. They built cross-functional teams, wrote governance policies first, went slow before they went fast. Everyone else is one person on the side drafting an appeal.

I want to read the same numbers differently.

The seven percent are succeeding at acceleration, not transformation

The report defines impact as efficiency gain, fundraising velocity, and personalized donor outreach. By that definition, the seven percent are not doing something categorically different from the ninety-three percent. They are doing the same thing at scale, with the friction removed.

So the question worth holding is not whether the seven percent figured out integration. The question is who is better off when the workflow goes faster.

Three groups could plausibly be made better off by AI in this sector. The people the organizations exist to serve. The workers doing the serving. And the organizations themselves, as measured by their dashboards.

The report measures the third. It does not ask about the first two. What follows takes them in turn.

The people being served are not measured

A case manager who finishes intake forms faster has made a real gain. Intake is not the work anyone went into the field to do. The question is where the saved time goes, and whether the client is better off for it.

The nonprofit-specific evidence here is thin to the point of absence. A 2025 systematic review of AI-assisted case management in social work found a small set of empirical studies reporting effective outcomes, but the outcomes measured were things like risk-assessment accuracy and decision support. None of them measured whether case managers had more relational time, or whether clients felt more heard. The Virtuous report measures self-reported efficiency from staff. It does not ask the people the work is for.

If the saved time goes to a tenth intake of the day, the client is processed faster and heard no more. If it goes to a real conversation with one of the ten, the client is genuinely better off. If it goes to fixing what the AI summary got wrong, the client is now at risk of a decision made on bad data. We do not know which of these is happening at any scale, because the people producing the evidence base are not asking.

The extreme version made the news this spring. A Texas mental health nonprofit used an AI grant research tool that surfaced a foundation as a perfect match, presented years-old information as current, and missed that the foundation had changed its mandate long before. Three weeks of application work followed. The counseling program closed two months later. The story comes to us through a press release from a competing AI vendor, which is its own small lesson, but the shape of it is not in dispute. The communities served by that program are measurably worse off. The communities served by the case manager remain unmeasured.

The workers are not measured either

The worker question has the same shape. Where does the time go, and is the worker better off.

The cross-sector evidence is genuinely mixed. On the encouraging side, a Zoom survey early this year found most knowledge workers saving thirty minutes a day or more and spending it on breaks and life outside work, and Workday found that the workers who report good outcomes from AI tend to reinvest saved time in higher-value work. On the harder side, the Berkeley Haas eight-month ethnographic study published in Harvard Business Review in February found that AI did not free up time at all. It expanded what people were willing to take on. Workers moved faster, widened the scope of what counted as their job, and let work seep into lunch and evenings, often without anyone asking. A separate Workday survey of 3,200 employees found that close to forty percent of the time AI saves gets spent fixing what AI got wrong.

None of this is from the nonprofit sector. And nonprofit workers do not enter this period as neutral subjects. They enter it from inside the pattern I have written about before, the martyr effect, where sacrifice is the currency of belonging and giving up rest is the quiet price of staying.

The Berkeley Haas study describes a workforce that was already willing to absorb more. The nonprofit workforce is structurally more willing than that. The worker who says AI freed up an hour, so I am leaving at five today, is making a move the field has spent decades organized to punish.

Whether AI is making nonprofit workers better off or simply accelerating their existing pull toward self-extraction is an empirical question. The report did not ask it. The cross-sector evidence is too mixed to assume the answer. That gap is not incidental. It is a research demand.

Only the dashboard improved

What is left is the third group. The dashboard.

The CRM looks better. The reports look better. The numbers the funder sees look better. The report’s own exemplar is a development director who set aside six hours for an email campaign and finished it in twenty minutes with an AI fundraising tool. That is offered as proof of impact.

It is also exactly what an improved dashboard looks like. The hours are tracked. The output is counted. The conversion rate is measured. Whether the saved time went to the worker, or to a deeper conversation with a donor, or to a colleague who needed help, does not appear anywhere in the report.

The dashboard is better off. The dashboard is not the work.

This is the water the sector already swims in

The sector spent thirty years building its incentive architecture around what the dashboard could see. Donor outreach velocity. Output counts. Overhead ratios. Grant compliance cadence. The work that did not fit a CRM field, the judgment, the listening, the slow trust, the accountability owed to communities, was invisible because the reporting frame could not hold it. The architecture rewarded what it could count.

AI is now arriving inside that architecture and being judged against the same metrics. Of course it is succeeding. It was built to.

This is not a failure of integration. It is success at the wrong thing.

I am not outside this. I built monitoring systems that rewarded throughput. I wrote indicator frameworks that counted contacts and never the depth of what passed between them. The infrastructure now being accelerated is infrastructure I helped lay. The acceleration is not happening to the sector from outside. It is running through the architecture practitioners like me put in place.

Efficiency is the wrong word if the time goes nowhere

Efficiency is the word the report uses. It would be an honest word only if the time saved went somewhere that mattered. To the client, in the form of being more fully heard. To the worker, in the form of going home earlier or being paid for the work they actually do. To the mission, in the form of work the dashboard cannot see.

The time isn’t going there. The thing that wins is the dashboard.

So the real question is not whether AI works. AI works. The question is why we keep optimizing for the dashboard.

Forthcoming posts in this series:

Who is Better Off When AI Speeds up the Workflow?

Measuring the Wrong Thing Faster

Grading Their Own Homework

Three Moves to Tell if Nonprofit AI Works (and for whom)

Leave a comment

Share

Anthralytic is a strategy and evaluation studio for mission-driven organizations. If you make decisions about resources in the social sector, whether or not you call yourself an evaluator, this newsletter is for you.

The International Rescue Committee, with staff on the ground, has named the pattern directly. Years of underinvestment compounded by recent funding cuts have hollowed out eastern DRC’s surveillance capacity, and the outbreak escalated faster because the system that would have detected it earlier had been thinned.

That is the news. What follows is the structural story underneath it, read against the two pieces in which the Secretary of State has, over the past year, introduced and defended the new American aid architecture.

The mechanism was announced in plain English.

On July 1, 2025, the day USAID formally ceased implementing foreign assistance, Secretary of State Marco Rubio published Making Foreign Aid Great Again on the State Department’s Substack. The piece rested on a familiar move. USAID, in Rubio’s telling, had drifted into charity divorced from American interest, and the closure would restore foreign assistance to its proper role as an instrument of American foreign policy. The strawman is worth setting aside for a moment, because the operational promises that came alongside it are what merit re-reading now.

Rubio committed the State Department to “consolidating fragmented appropriations accounts” to create more flexible pools of funds. He committed it to new efficiency criteria that would “measure impact quantitatively.” He committed it to building a “fast feedback loop” through empowered diplomats in regional bureaus, so that programs would track to American interests and partner needs in real time.

Those commitments were not loose. The consolidated pools of funds became the $19.2 billion closeout pool that an April 20 congressional notification later set aside to wind down USAID. Rubio described the new architecture in plain operational terms and promised it would work better than what had come before.

Nine months later, the promises had disappeared.

On April 6, 2026, Rubio published another piece on the same Substack. The Face of America Abroad is a defense of the Foreign Service as the inheritor of a long American diplomatic tradition. The piece moves through Silas Deane’s secret 1776 mission to France, Benjamin Franklin’s negotiations in Paris, and the Treaty of Paris. It celebrates the State Department’s recruitment refresh, the removal of DEI from entrance exams, and a renewed focus on diplomatic history and tradecraft for new officers. It closes by holding up Franklin as the model for the modern Foreign Service Officer.

The word aid does not appear in the piece. There is no accountability report on the new efficiency criteria, no example of the feedback loop in operation, no description of what the consolidated appropriations pools have produced. Nine months after the closure announcement, the deliverable that was promised has dropped out of the rhetoric entirely. What remains is a discourse about American power and the people who project it, with the thing being delivered moved off-stage.

The redirect is doing two jobs at once.

On May 7, CNN reported that the administration plans to redirect roughly $3.2 billion that Congress appropriated for global health and foreign development assistance to pay for the closure costs of USAID. The redirect is one slice of the $19.2 billion closeout pool.

That pool is doing two things at once. It is paying for the institutional dismantling of USAID. It is also paying organizations the US government already owes for work performed, supplies procured, and contracts terminated mid-stream. Both jobs are being done by the same money, through the same mechanism, with funds Congress appropriated for neither.

The $19.2 billion breaks down across three pools. The largest, more than $15 billion, is money the US had committed to multi-year grant agreements with partner governments but had not paid out when those agreements were cancelled. About $625 million is unspent 2024 appropriations. The remaining $3.2 billion is unspent 2025 money from accounts Congress designated for global health and economic development.

The legal piece sits inside that pool. A June 2025 internal estimate reported by Bloomberg Government put litigation costs at roughly $344 million, much of it tied to lawsuits from implementing partners and USAID staff who contend they were terminated illegally. Chemonics International, DAI Global, and others have filed suit alleging hundreds of millions in unpaid invoices, including, in Chemonics’ case, $240 million in committed medicine and health supplies stranded mid-supply-chain.

For a decade I worked for one of those implementing partners. The organizations now negotiating settlements and pursuing lawsuits are former colleagues. This analysis comes from inside that field, not from outside it.

The line that catches outbreaks is the line being closed.

Inside the 2025 portion of that pool, $647 million is in reductions to global health security specifically. That line pays for outbreak surveillance, preparedness, and the laboratory infrastructure that catches a strain like Bundibugyo before it has been circulating for months. The funding that would have detected this outbreak earlier is the funding now paying to close the agency that ran the surveillance.

A former USAID official quoted in the CNN reporting described the agency as the glue. The role was to coordinate health officials, NGOs, and donors so that when an outbreak appeared, the experts arriving on the scene had something to attach to. Without the glue, expertise arrives but cannot pay health workers, cannot move supplies, cannot stand up the operational backbone that a response actually runs on.

This is the part that does not show up on a funding diagram. Surveillance is not a line item; it is a set of relationships that take years to build and minutes to dissolve. The line items describe the activity. The relationships are what make the activity possible.

The feedback loop Rubio promised in July, in plain operational terms, is precisely this kind of infrastructure. It is the surveillance and field-monitoring architecture that would have caught a Bundibugyo strain in eastern Congo before it had been moving through communities undetected for months. The loop he announced is the loop that just got defunded. The only loop running now is between the closure and itself.

I have written budget narratives for surveillance indicators. I have written into log frames the assumption that the institutional glue would be there when the indicator needed to be measured. The line items will reappear in some other agency’s budget eventually. The relationships will not.

The mission money inverted.

Set the closeout ledger against what the same pool of money was appropriated to do. Analysts projecting the impact of the $2 billion health reduction have estimated 121,000 preventable TB deaths, 47,600 preventable malaria deaths, nutritional aid lost for 23 million children, and safe childbirth access for 5.7 million women. Add the Ebola line on top of that ledger: more than 170 deaths so far, with WHO warning that the actual scale of the outbreak is much larger than has been detected.

The detection systems that would have caught the outbreak are the detection systems being paid to be turned off. The mission money is paying for the mission’s shutdown.

Aid was the interest.

The Secretary’s framing for closing USAID rests on a strawman. He describes a charity model that lost sight of American interest. That is not what USAID was.

The agency was created in 1961 under the Foreign Assistance Act and designed from the start to integrate development into foreign policy. For most of its history, US food aid had to be procured from American farmers and shipped on American flag vessels. Development contracts carried Buy American provisions. PEPFAR, the Millennium Challenge Corporation, Feed the Future, and the Development Finance Corporation that the first Trump administration created were all explicitly built around American strategic and economic interest. The interest was the architecture, not an afterthought to it.

I have written grant narratives that named US strategic interest in the same paragraph as community outcomes. The two were never severable in USAID work. That was the design, not the corruption.

A Bundibugyo strain not caught early in eastern Congo can land in an American hospital. Disease surveillance in DRC is one of the most American-interest-aligned line items in the federal budget. The $647 million in global health security reductions is the line item that catches what arrives.

The April piece talks about American power as if it has only ever been projected through diplomats. That has never been true. American power has been projected through aid as a deliberate instrument of foreign policy for more than six decades, and the architecture is being erased from the rhetoric at roughly the same speed it is being closed in the budget. The interest was always there. The closure is what removes it.

The counterfactual does not exist.

The closure mechanics are being measured. The USAID Inspector General has begun a series of audits on asset disposition at terminated missions, with the first published in March 2026. House Foreign Affairs leadership has formally asked GAO to review what the cancellations cost and what capacity remains at State.

What is not being measured is the programmatic counterfactual. The Ebola response will be assessed against what is happening on the ground now. What the response would have looked like with the surveillance intact is unknowable, because the surveillance was the way we would have known.

You can audit the closure of a measurement system. You cannot evaluate the impact of its absence using the system that is gone.

Three pieces, one mechanism.

Two Substack posts, nine months apart. The first announced the architecture and promised accountability through measurement. The second moved to Franklin and stopped mentioning aid altogether. The Ebola outbreak landed between them.

Collapsed. Redirected. Inverted. Indirect-cost-as-structural-failure in its purest form: the agency’s closing pays itself, in the currency of the missions it was supposed to deliver.

Anthralytic is a strategy and evaluation studio for mission-driven organizations. If you make decisions about resources in the social sector, whether or not you call yourself an evaluator, this newsletter is for you.

Leave a comment

Share

The first was an evaluation opportunity. As we talked through what the client was looking for, the shape of what they wanted became clear without anyone saying it out loud: they had a decision to make and they wanted a summative evaluation that arrived at that recommendation. They needed a third-party expert to put it in writing, and the signature was the unspoken deliverable.

I declined. The request did not meet ethical evaluation standards, which require findings to follow the evidence rather than the other way around.

The second was a conversation with a former colleague who now works in public health. She was describing a company trying to win a contract, and to win it the company needs to demonstrate specific statistical benchmarks. They are massaging the data to get there. The benchmarks were set up to standardize quality across providers, but in practice, the colleague said, everyone games them, and the actual quality across providers varies wildly.

By her account, the company in question is actually one of the good ones, with a service that an external evaluation found to exceed expectations. They provider is gaming the benchmark not because their work is poor but because the benchmark cannot capture what their work actually does. But the poor quality providers also do the same thing, so rather than focusing on improving actual quality, they focus on success in gaming the system.

Two different sectors, two different stories, and the same underlying problem sitting underneath both.

The contract refusal was a privilege

The privilege is worth naming out loud, because pretending otherwise is dishonest.

Evaluators have industry standards and ethical codes, but declining work depends on being able to, which depends on other work, on savings, on a household income that does not require this one contract. The evaluator who needs this engagement to pay rent is not always less ethical, but they are, by definition, more constrained, and they are also a more useful case study in what the system actually rewards.

When we talk about evaluator integrity, we tend to treat it as a stance you take. Sometimes it is. More often, it is a luxury good purchased with margin, and writing about ethics without naming that costs nothing and changes nothing.

The fish do not make the water

The public health organization is doing what the system asks of them. Their question is existential, and the intervention they offer might be one of the ones that actually works. If they do not produce the numbers in the right shape, the contract goes elsewhere, the program ends, staff lose jobs, and the people they serve lose the service.

In that frame, massaging data starts to look like keeping an effective program alive. One that they know is working. The data is now load-bearing for survival, not just for learning.

The temptation to round up is not exotic, and it is not a character flaw. It is the rational response of a fish trying to swim in the only water available.

This is a structural problem before it is a character problem. The fish are not the issue. The water is.

The dynamic has a name. Goodhart’s Law says that when a measure becomes a target, it ceases to be a good measure. Campbell’s Law, the social-science cousin, says that the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures. It is the same law in different language, and the field has had fifty years to know it, but keeps building the systems anyway.

I wrote about this dynamic for the AI context last August, in a piece on how people break AI in social impact. That piece looked at how the pattern plays out inside a single model. What follows extends the frame to three sectors I have either worked in or watched closely: international development, education, and healthcare. The public health story sits inside a much larger pattern, and the pattern is the point.

Bad data follows the money in international development

I will start with the field I know best. Justin Sandefur and Amanda Glassman wrote a paper for the Center for Global Development in 2014 called The Political Economy of Bad Data. Their finding, across multiple African countries, was that official statistics systematically exaggerate development progress, and they traced the exaggeration to two specific mechanisms.

The first mechanism is that governments overreport to foreign donors. Their lead example is a results-based aid program that paid for reported vaccination rates, and the reported rates exceeded what independent household surveys could find. The money was tied to the number, and the number obliged.

The second mechanism is that governments are themselves misled by frontline providers. Their example is primary school enrollment, where official numbers diverged sharply from survey estimates after funding shifted from user fees to per-pupil government grants. The schools had been counting students before, more or less honestly, and then the grant changed what the count was for.

USAID’s own Inspector General reached similar conclusions from the inside. Across 21 performance audits in Egypt, Jordan, and the West Bank and Gaza between 2011 and 2013, 71 percent found unreliable data, including overstated indicators and missing documentation. In one case from Ukraine, the implementer overstated an indicator by roughly 100,000 and attributed the overstatement to a typographical error. In Guatemala, another implementer overstated leveraged funds by $3.4 million.

The pattern is not occasional misconduct; it is the predictable output of a funding structure that pays for indicator achievement. Logframes do not cause this. What does is the cost-reimbursement contract, the indicator-tethered disbursement, and the political pressure to show progress, all running at the same time.

Andrew Natsios, who ran USAID from 2001 to 2006, named the dynamic from the inside. He called it “obsessive measurement disorder.” The programs that get measured most precisely, he argued, are the least transformational, while the programs that actually transform anything are the hardest to measure. The aid system, he said, had organized itself around the wrong half of that observation.

Education made evidence into a badge

The Every Student Succeeds Act of 2015 set up a four-tier system for evaluating evidence behind educational programs, with Tier 1 reserved for strong evidence from a randomized trial, Tier 2 for moderate evidence from a quasi-experiment, and the lower tiers for weaker designs. Federal money for certain program categories is tied to using interventions with stronger evidence.

Vendors self-select the tier they claim, and the incentive structure that results is striking. Once a vendor has a single Tier 1 study showing a statistically significant positive effect, additional studies can only weaken the claim. The rational move, then, is to fund one tightly controlled study with extensive implementation support, get the positive result, badge the product, and never study it again. The market fills with Tier 1 claims, and the actual quality among Tier 1 products varies enormously.

The framework was supposed to standardize quality. In practice, it standardized the gaming.

This is the story my colleague was telling me, told in a different sector. Replace ESSA with whatever the equivalent benchmark is for a state public health contract, and the structure is the same. There is a number, that number determines whether the program gets the money, and the shape of the company’s behavior follows.

The older and sharper version of this same pattern is the Atlanta Public Schools cheating scandal, which ended with multiple educators convicted of racketeering for systematically changing student answers on standardized tests. The high-stakes use of a single metric for teacher accountability did not produce better teaching. It produced erased answers.

In healthcare, the metric moves and the reality does not

The Affordable Care Act created the Hospital Readmissions Reduction Program in 2010, under which hospitals with higher-than-expected 30-day readmission rates for selected conditions get penalized through reduced Medicare payments. The program was credited with substantial reductions in readmissions, and for years it was held up as a success of pay-for-performance in healthcare.

In 2019, Ody and colleagues published a paper in Health Affairs showing that the credit was overstated. A coincident change in electronic transaction standards allowed hospitals to document more diagnoses per claim, which meant higher risk scores, which in turn meant lower risk-adjusted readmission rates. The patients were the same, the care was largely the same, and what actually changed was the paperwork. Accounting for that change cuts the apparent decline in risk-adjusted readmissions for targeted conditions by 48 percent, and the authors conclude that either the HRRP had no effect on readmissions, or its effect was roughly half what we thought.

The Veterans Affairs wait-times scandal is the cleanest version of the same dynamic. The VA set a 14-day target for new patient appointments and tied executive bonuses to performance on that measure. Across at least 93 VA facilities, schedulers and managers manipulated the data: they started the clock late, kept secret paper waiting lists, and cancelled and rescheduled appointments to reset the count. One scheduler wrote in an internal email, “Yes, it is gaming the system a bit. But you have to know the rules of the game you are playing, and when we exceed the 14-day measure, the front office gets very upset.”

The Inspector General documented the manipulation in 2014. A follow-up audit in 2022 found the practice continuing under different mechanisms, with a 66-day wait being logged as 43 because of how the start date was defined. A decade on, FOIA records show the gaming persists.

Gaming is the system working as designed

Three sectors, different actors, different stakes, and the same pattern emerges in each. This is not a moral failure of evaluators, vendors, hospitals, or aid implementers. It is the predictable result of designing accountability around single quantitative targets attached to financial consequences, while leaving the underlying conditions of the work unexamined. The field has not lacked for the analysis. It has lacked for any structural willingness to use it.

My refusal of the predetermined evaluation matters. It protects my integrity, and it may have spared a specific program from a third-party signature on a verdict already written. It is the right thing to do. It does not change the structure that produced the request.

For every evaluator who declines, there is another evaluator who needs the work, and the field selects, over time, for the people who will deliver what is asked. That is not because evaluators are weak. It is because the field is funded that way. The same is true of the public health vendor. They can decline to game, but the next vendor will not, and the next vendor will get the contract. The system is not waiting on individual ethics to fix itself.

This is the part that gets uncomfortable for those of us who think of ourselves as on the right side of the question. Our refusal is necessary, and our refusal is not enough.

Some interventions actually end the gaming

If individual integrity is not enough, the next question is what is.

The research and the field experience point at a few real candidates, and the two with the strongest track records are worth developing in some detail.

The first is closure paired with quality alternatives. Charter schools are an imperfect and sometimes maligned case, and the case needs to be told honestly. In states with strong authorizer accountability, low-performing charter schools do close, and at higher rates than equivalent traditional public schools. CREDO’s 2017 Lights Off study tracked 1,522 low-performing schools that closed between 2006 and 2013 across 26 states, and it found that the charter sector closed a higher share of its low-performing schools than the traditional sector did. That is a real structural intervention, and one that is rare in the rest of the social sector, where ineffective programs tend to keep running because no one has both the authority and the willingness to end them.

The same CREDO study, however, found that less than half of displaced students landed in better schools. Closure without somewhere good to go is not accountability; it is dispersal. The intervention is not closure alone. It is closure paired with quality alternatives, and paired with rules that prevent failing programs from authorizer-shopping their way back into the market.

The second is independent verification that the producer cannot prepare for. Sandefur and Glassman flag one working example at the end of the bad-data paper: the World Bank’s Health Results Innovation Trust Fund, which uses small unannounced household surveys to cross-check the administrative data participating facilities report, and applies penalties for over-reporting. In Cameroon, this dropped over-reporting of outpatient consultations by more than 90 percent in less than a year. The verification is built into the funding mechanism, and the people producing the numbers cannot study to the test, because they do not know which numbers will be checked or when the check will arrive.

Neither of these is exotic. Neither requires new theory. They require funders willing to design accountability that cannot be gamed by the people whose careers depend on it.

The harder thing this week was not declining the evaluation. The harder thing is staying clear-eyed about which parts of this system I am still inside even when I decline.

I take contracts, I produce indicators, and I write reports that get read by people making decisions about budgets. I am one of the fish. Refusal is honest. Refusal is partial. Building anything different requires naming the ecosystem the game runs on, not just naming the players in it.

Leave a comment

Share

Anthralytic is a strategy and evaluation studio for mission-driven organizations. This newsletter is a practitioner's thinking-out-loud about evaluation, AI governance, and the systems social impact work runs through. If you know someone who makes decisions about resources in the social sector, whether or not they call themselves an evaluator, this newsletter is for them.

That admission is unusual in print from that quarter of the field. It is also worth taking seriously, because the struggle they name is real and it is not only about discourse.

What the struggle actually is

The field is being asked to evaluate a transition that has not yet stabilized into something evaluable. The objects of evaluation are changing under our feet. In a piece last week, Who’s Left at the Table, I wrote about the early data on where new US foreign assistance dollars have actually gone: to a small handful of multilateral pooled funds, and to bilateral government-to-government arrangements. That shift is not only a change in who receives money. It is a change in what kind of work gets done, by whom, and inside what institutional culture.

When the work changes, evaluating it changes too.

What is happening to the craft

The US implementing partner ecosystem developed a particular set of evaluation conventions over several decades, including utilization-focused evaluation, theory-of-change-driven design, third-party mid-term and final evaluations, MEL teams housed inside implementing organizations, and Collaborating, Learning, and Adapting frameworks for ongoing learning. These conventions did not emerge from nowhere. They emerged from a delivery architecture that needed them and was, in imperfect but real ways, built to use them.

When the delivery architecture changes, the conventions either travel or they do not. They are not changing because the field decided to change them. They are changing because the institutional homes that supported them are getting smaller, fewer, or differently configured.

The cultures receiving the work

The United Nations Office for the Coordination of Humanitarian Affairs runs pooled funds with humanitarian monitoring and evaluation conventions optimized for speed and output reporting. They are designed for crisis response. They are not designed for the kind of longitudinal outcome learning that the US implementing partner ecosystem built over decades.

The Global Fund operates on a vertical-program model with disease-specific indicator frameworks and country reporting systems. Its evaluation culture is real and substantial. It is not the same culture as the broad-spectrum, mixed-methods, utilization-focused tradition.

Bilateral government-to-government arrangements typically have minimal independent third-party evaluation by design. Accountability runs between two states. The affected community is not the audience for the evaluation, if there is one.

None of this is a failure of the new institutional receivers. They have their own histories and conventions, which work for the problems they were built to solve. Those conventions are not, however, interchangeable with the ones the US implementing partner ecosystem developed.

Why the words are getting harder

When CGD writes that they have been struggling to talk about evidence and evaluation, part of what is happening is that those two words now carry different meanings inside the new architectures than they did inside the old one. Value for money looks different through a humanitarian pooled fund than through a USAID cooperative agreement. The same words, with audience-specific meanings, are sliding past each other in the field’s current conversation.

The discourse problem is not that the field forgot how to talk. The referents are moving. The struggle to talk is downstream of that.

A practitioner note

I have built monitoring and evaluation systems inside cooperative agreement structures for years. I have written learning agendas that depended on the continuity of the implementing partner. That continuity is gone in many of the cases I worked in. The systems I built are not as portable into the new architectures as I once assumed they were. The struggle CGD names is not external to me. I am inside it.

Three questions that travel

Three orientations work in any architecture. They are not new. They are the ones the field has been asked to take seriously for a long time and has not always wanted to.

The first is motive: why we are evaluating. Evaluation for control, for justification, and for learning are different practices that produce different power dynamics. The new architectures are arriving with their motives already attached, and naming our own, as evaluators, is now a more visible act than it used to be.

The second is ownership: who owns the knowledge. The implementing partner, the funder, the multilateral pooled fund, the partner government, and the affected community are all candidates. The answer matters because ownership of findings shapes whether findings get used, and by whom.

The third is authorship: who counts as an evidence-maker. Affected communities themselves can be evidence-makers, not only the people the funder hires to study them. Indigenous and participatory traditions have answered this question well for decades. It is harder to ignore in an architecture where the old implementing partner intermediaries are gone.

These orientations are practical, not framework-shaped. They work in cooperative agreements, in pooled funds, in bilateral arrangements, in domestic philanthropy. They survive the architecture shift because they are not architecture-dependent.

CGD’s admission is worth taking seriously. The struggle to talk is real, and it is also structural. Naming what is moving makes the words usable again. The questions that travel are the ones we have been asked to take seriously for a long time. The field’s reorganization may be the moment to actually take them seriously.

Share

Anthralytic is a strategy and evaluation studio that helps mission-driven organizations clarify and amplify their impact.

In late February of this year, in a five-day stretch, the US government signed three new foreign assistance awards that together moved more than two billion dollars. Two went to the United Nations Office for the Coordination of Humanitarian Affairs. One went to the Global Fund to Fight AIDS, Tuberculosis and Malaria. After almost a year of very few sizable new awards, the money started moving again, and it moved to a small number of places.

In April I wrote about the orgs that are missing from the FY26 Food for Progress table and why (Who’s Missing From the FY26 Table). The Center for Global Development released an analysis last month that lets us see the other side of that picture: where the money actually went.

The data, plainly

Charles Kenny at CGD pulled new contracts and assistance awards from USAspending.gov for the main non-military foreign assistance accounts, covering January 20, 2025 through March 30, 2026. In the last year of the Biden administration, the same accounts issued 2,604 new awards totaling $8.881 billion. In the Trump administration’s first 14 months, the same accounts issued 345 new awards totaling $3.482 billion.

The top 10 recipients took 94.8% of those new dollars. UNOCHA alone took 57.5%. UNOCHA, the Global Fund, and the International Organization for Migration together took roughly 80%. If you want to look at the data yourself check out usaspending.org. That is the picture. Money is moving again, and it is moving to a small handful of multilateral institutions and to government-to-government bilateral arrangements.

The frame being applied

Kenny calls this shift “mostly a good thing in and of itself.” That language borrows, deliberately or not, from the localization argument that has been alive in development discourse for at least a decade. Localization advocates have argued for years that fewer, larger, more flexible agreements with actors closer to affected people would be an improvement on the US contractor-and-cooperative-agreement model. That position has integrity and a long literature behind it. I am not going to argue against it.

But the data does not show that movement.

The distinction that matters

A pooled fund managed by UNOCHA in New York and Geneva is not closer to affected people than a cooperative agreement with a US implementer that ran country offices staffed by nationals. A contribution to the Global Fund is not closer to affected people than a maternal and child health program working through district health offices. A bilateral arrangement between two governments routes funds through national systems, which is a real shift, but national governments are not the same as affected communities. Some bilateral arrangements will move decisions and resources closer to people. Others will reinforce the patterns those people have been organizing against for years. The architecture does not produce the outcome on its own.

Localization, as the term has been used, has meant shifting decision-making and resource control closer to the people the work is meant to reach. What the data shows is something different. It shows fewer institutional addresses, mostly in Geneva, Rome, and Atlanta, plus partner government capitals, holding much larger flows.

The geometry

The architecture now has fewer addresses, with larger flows per address. In many cases, the money is more distant, not less, from the people it is meant to reach. The accountability surface is smaller and harder for an affected community to access than it was under the old model, not larger and more open. The new architecture may turn out to be better than the one it is replacing on a range of dimensions, including lower transaction costs, fewer intermediaries taking a cut, more flexibility, and faster response. Those are real possibilities. They are not, on the evidence so far, what localization meant.

A practitioner note

I have been in rooms where the localization argument was made well, and rooms where it was made badly. I have helped write proposals that claimed local ownership while routing most of the real decisions and most of the money through US headquarters. The frame has always been available for use in either direction. The question now is which direction the current consolidation is actually moving in. The early data suggests upward, not outward.

Lingering questions

I don’t know whether the new bilateral country-to-government arrangements will move decision-making toward affected communities, or whether they will replicate the same patterns at a different level of government. I don’t know whether the multilateral pooled funds now receiving most of the money will invest in the partnership and learning infrastructure that affects accountability, or treat affected populations as recipients of disbursements. I don’t know where the new consolidation is functioning closer to true localization, and what makes those cases different from the rest.

What I do know is that the money has started moving, and it is moving to a small number of places. That picture deserves to be looked at on its own terms, before any frame gets placed over it.

Subscribe now

Leave a comment

Share

Anthralytic is a strategy and evaluation studio that helps mission-driven organizations clarify and amplify their impact.

Mike Lee’s recent piece, “A New Blueprint for Big Food,” names the pattern and offers a different wager. Stop fighting the profit motive, he writes. Redirect it. If the financial logic that runs Big Food can be re-aimed at soil health, nutrient density, flavor, and worker welfare, then the activist investor and the sustainability advocate end up pushing for the same outcome. It is a sharp piece, and the move is the right one. Lee takes financial logic seriously as the system to be redirected, refuses the moral lecture that has not worked for forty years, and stays honest about the gaps in his own case.

I want to build on his piece. There is a question sitting underneath it that he does not quite ask, and naming it changes how the wager reads. The question is what food is actually for.

Lee inherits an answer from the system he is trying to redirect: food is a consumer good, grown for sale and accounted for on balance sheets, and the argument is which costs that system should price. Any serious proposal for fixing the food system has to take that reality seriously, and Lee’s does. The question is not whether food operates inside markets. It does. The question is which of food’s functions the market sees and rewards, and which it leaves to whatever happens around the edges.

Food does many things at once. It sustains bodies, but never just bodies. It carries pleasure, but pleasure shaped by what you grew up tasting and where. It connects people: the eaters and the cooks and the growers and the carriers. It marks place: soil and climate, kitchen and table. It carries culture, the recipes and rituals and meanings that travel forward. It expresses sovereignty, the ability of a community to feed itself on terms it chooses. It participates in living systems, drawing from soil and water and weather and labor and feeding back into them. Each function is upstream and downstream of the others. The parts do not run in parallel, they run in relationship. That is what it means to call food mutually causal.

What food is for also varies by what food we are talking about. Wine is for pleasure, ritual, conviviality, the expression of place. Nobody buys wine for nutrient density. Vegetables sit at the other end: nutrient density and accessibility are the dominant functions, with pleasure and culture alongside. Lee’s flavor-revenue path borrows its strongest evidence from wine, where flavor and place have been linked for centuries, and stretches it across categories where the dominant functions are different. The flavor-soil-nutrition chain may hold for one category and break for another. What food is for is not a single answer. It is many answers, depending on what is on the plate, who is doing the eating, where, and under what circumstances.

Any blueprint reaches some of these functions and misses others. Lee’s reaches pleasure and partially reaches nutrition through the flavor-revenue path. It does not, by design, reach connection, place, culture, sovereignty, or living-system participation, except where those happen to be downstream effects of changes in soil and worker welfare. That is not a failure of his blueprint. It is its shape, fitted to the goal he is working from.

His goal, which he does not quite name, is to make Big Food less harmful without replacing it: to keep the consolidated industry and the market system that runs it roughly intact while the financial logic starts rewarding different things. That goal is real and defensible. It is not the only one. Other goals would produce different blueprints: dismantling Big Food, moving food out of market logic, centering food sovereignty, building regional systems on commons or cooperative principles.

Two paths, two theory levels

Lee proposes two paths. The first is what I will call the flavor-revenue path: better soil produces more flavorful and more nutrient-dense food, and eaters reliably pay for flavor, so regenerative agriculture finds its way to revenue through people’s mouths and wallets. The second is the externality-pricing path: get the true cost of conventional food (soil loss, polluted water, healthcare costs, exploited labor) into accounting standards and supplier contracts, and the cheap commodity ingredient stops being cheap.

He bundles them as complementary tools. Pulling them apart and asking whether the parts hold together takes a couple of distinctions worth borrowing from evaluation work.

A theory of change is an explanation of how change happens in a given context. It traces the causal mechanisms that connect where we are to where we want to be, and surfaces the assumptions that have to hold for the connection to work. The term comes out of evaluation theory, particularly Carol Weiss’s writing on theory-driven evaluation. A theory of change is mostly descriptive: this is how the world rearranges itself when the right conditions are in place.

A theory of action is a plan for what specific actors will do to make change happen. It identifies the activities, the actors, and the resources needed to produce an outcome the system would not produce on its own. A theory of action is mostly prescriptive: this is what we will do.

The evaluative question is whether the two align. Does the proposed action actually trigger the change the theory of change describes? Where the action is too narrow for the mechanism the change theory implies, there is an alignment problem worth surfacing.

This gets sharper with a second distinction, from Chris Argyris and Donald Schön. They observed that people and organizations carry two theories of action at once. The espoused theory is the one we offer when asked: the world view and values we believe our behavior is based on. The theory-in-use is the one our actions reveal: the world view and assumptions implied by what we actually do. The two are often misaligned. The most important fact about the gap is that people are usually unaware of it. Argyris’s claim is that effectiveness comes from developing congruence between the two, which means surfacing the gap is constructive work, not gotcha work.

Their example is concrete. A management consultant said he would handle a disagreement with a client by first stating his understanding of the disagreement and then negotiating what data they could agree would resolve it. That was his espoused theory: joint control of the problem. A tape recording of the same consultant in such a situation showed him advocating his own view and dismissing the client’s. That was his theory-in-use: unilateral control. The consultant did not know the gap was there.

A public proposal carries both layers too. There is an espoused theory of change and an espoused theory of action, the version Lee writes for public reading. There is also a theory-in-use embedded in the design, made of the assumptions the blueprint quietly depends on. Whether they align is the evaluation question.

For Lee, the espoused theories sit cleanly with each other on the surface. The espoused theory of change says markets respond to better measurement, regenerative becomes profitable, and Big Food reorganizes around soil, flavor, and worker welfare. The espoused theory of action follows: fund the science, build the measurement tools, change the accounting standards, and wait for the reorganization to follow. Read on the surface, the action plan addresses the conditions the change theory says are needed.

The theory-in-use is harder to see and more interesting. The blueprint as designed depends on a number of assumptions the espoused theory does not name. Markets reliably translate measured signals into desired outcomes, which they often do not, because indicators get captured by the producers being measured. Big Food’s consolidated industry structure is compatible with regenerative supply chains, which it often is not, because consolidation tends to compress regenerative farmers when they get pulled into industrial supply. Measurement infrastructure is a neutral tool rather than a lever controlled by particular actors. Consumer behavior reliably tracks measurable signals at scale, despite the well-documented class stratification of palate. Accounting standards respond to a clear intellectual case, despite the Rockefeller Foundation’s true-cost report sitting in circulation for years without translating into FASB or IFRS movement. The financial logic is the dominant or sufficient lever for change.

None of those assumptions is named in Lee’s piece. All of them are doing structural work in the design. This is not a charge of bad faith. It is the standard pattern Argyris and Schön describe: the blueprint’s author probably could not see the in-use theory if asked, because that is what theories-in-use do.

The forty-year pattern Lee opens his piece with is a useful test case for the misalignment. Annie’s, Stonyfield, Honest Tea: each tried to operate inside an in-use theory that scale and mission would coexist. Scale rewarded itself, and mission got compressed. The pattern is not a fluke. It is what happens when an espoused theory and a theory-in-use are out of alignment, and the actors involved are unaware of the gap.

The misalignment in Lee’s blueprint is between what the proposal says about how change will happen and what the proposal actually depends on. The espoused theory pulls toward an ambitious reorganization of the food system. The theory-in-use defaults to the conditions of the existing food system, with measurement infrastructure as the only adjustable variable. When the in-use assumptions fail, and indicator capture is one well-documented way they fail, the espoused outcome does not arrive. What arrives instead is what the theory-in-use is actually fitted to: a less destructive Big Food that has absorbed regenerative aesthetics without changing structure.

That is the alignment finding. The hidden goal I named earlier is the goal embedded in the theory-in-use. The work the evaluation does is to surface the gap, so that the choice between the espoused theory and the in-use theory becomes a choice we make rather than a default we inherit.

The empirical assumption underneath

The flavor-revenue path also rests on a science question that deserves to be named, even if I cannot settle it here.

Lee’s path depends on a chain of empirical claims, each of which needs its own evidence base. Healthier soil produces more nutrient-dense food. Nutrient density correlates with flavor. Eaters reliably perceive the difference at the bite. Eaters will pay for that perception at scale.

Here is what I think we know, gesturally. The wine and terroir literature is reasonably solid on the relationship between soil, climate, plant stress, and flavor compounds. Plant physiology supports the secondary metabolite story: plants under mild stress do produce more of the polyphenols and aromatics that make food taste like something. The dilution effect in produce, where yields have gone up while nutrient density per unit weight has gone down, is real but contested in its interpretation. The connection between soil health specifically and human nutrition is an emerging field with real research and a long way to go before it can carry procurement or policy weight.

The point is not to settle the science here. Lee himself calls for the kind of research that would test these assumptions at the statistical confidence we apply to drugs, and he is right to. Before measurement infrastructure gets built on top of a partial chain, the chain itself has to be tested. The pieces that are well established need to be defended. The pieces that are partial need controlled studies. The pieces that are gestural need to be flagged as such, not built into the architecture as if they were settled. We are agreeing with Lee on what needs to happen next, and we are agreeing that the work has to happen before the wager scales.

When you optimize for the wrong thing

This is where the practitioner reflex kicks in.

The minute regenerative becomes “a measurable input that can be priced,” the indicator captures the work. I have watched this happen across enough fields, including ones I work in, that it now reads as a near-certainty. Carrots get bred for the spectrometer. Breeding programs, fertilization timing, harvest conditions, and post-harvest handling all get tuned to optimize the signal that drives the price premium. The reading goes up. Whether the carrots are actually more nourishing becomes a separate question that nobody is checking, because the reading has become the proxy.

This is the standard fate of an indicator that gets too important too fast: the number drifts from the thing the number was supposed to track, the spreadsheet still says we are on track, and the underlying thing is somewhere else.

What gets captured by the indicator is whatever goal the indicator was designed to serve, named or unnamed. If the goal is unnamed, the capture is invisible. We optimize for it without knowing we are optimizing for anything in particular. That is the through-line of every measurement system that has been built on top of an unstated objective. It is also why the question I named at the top is not optional. If the goal has not been named, the indicator will name it for us, and the name will be the one with the best signal-to-noise ratio, not the one we would have chosen.

A pre-mortem, in one scenario

A pre-mortem is a planning method from the psychologist Gary Klein. You imagine that the project has failed, then work backward and ask why. The point is to surface the assumptions a plan is hiding by treating failure as the starting condition rather than the worst case. It is one of the few tools I know for finding the hidden goal of a hidden theory of change.

Here is one pre-mortem for Lee’s blueprint, deliberately limited to a single scenario.

It is 2041. The wager worked. Healthier soil reliably produces more flavorful and more nutrient-dense food. The handheld spectrometer is in every grocery store, and true cost accounting moved into financial disclosures a decade ago. The CFO’s spreadsheet rewards regenerative supply chains, and the activist investor pushes for more rotation acres. By every metric in Lee’s piece, this is a win.

Here is what failed. Carrots are bred for the spectrometer. The reading went up year over year, but whether the carrots are actually more nourishing has become a separate question that nobody is checking, because the reading is the proxy. Breeding programs, fertilization timing, harvest conditions, and post-harvest handling all got tuned to optimize the signal. The signal is high. The underlying biology drifted somewhere else.

That is one scenario. Others are easy to imagine. There is the small regenerative farmer who cannot operationalize the new standards as fast as Big Food and gets bought out by the buyer who can. There is the premium-priced flavor that sorts by income and deepens the class divide in what people eat. There is the relationship between an eater and a farmer mediated by a device, where trust now lives in a number rather than in a name. There is the question of governance, where the new accounting standards get written by the same class of people who wrote the old ones, leaving the question of who actually decides about food exactly where it was. Each is its own pre-mortem worth running.

Each is also a different answer to the question of what food is for. The small farmer scenario asks about food as livelihood. The class divide scenario asks about food as access and dignity. The device-mediated relationship asks about food as trust between people. The governance scenario asks about food as something we decide together. None of these questions is forbidden by Lee’s blueprint. None of them is asked by it either.

What the question opens

Holding the objective question open is uncomfortable. It does not deliver a slogan or produce a campaign. What it does is change our relationship with proposals like Lee’s. You can take the wager seriously without absorbing the goal it is fitted to. You can fund the science the flavor path depends on without committing to flavor as the thing food is most for. You can push externality-pricing into accounting standards without conceding that accounting standards are the right place for the question of what we owe to soil and to each other.

The work of evaluation is to make hidden things visible: the goals, assumptions, and trade-offs that shape decisions before anyone realizes a decision is being made. When those are surfaced, the choices that come next are choices people can be held accountable for, instead of choices that get made for us by an objective nobody named.

Lee’s piece is worth taking seriously, and so is the question underneath it. The wager he describes is one good answer to one of the questions food asks of us. The other questions are still here, on the table, waiting for someone to ask them out loud.

Leave a comment

Share

Subscribe now

Anthralytic is a strategy and evaluation studio for mission-driven organizations, working at the intersection of measurement, impact, and the systems that shape how decisions get made.

Other Related Posts

It took me a decade to realize that the systems I had been working with abroad could solve a problem in my own yard.

What I Built When I Got Home

Years after that visit, after a decade of supporting country directors whose programs ran in places where the grid was unreliable and the rainfall sparse, I built a system of my own. I’d rather have bought it but nothing existed on the market that could do what I needed: supply rain water from my barrel with solar power directly to the plants with individual emitters. It was a simple drip system with lines running through the garden beds, the kind of setup a smallholder farmer near the Nile would recognize immediately, because the principles were the same.

I built it because I travel for work and my garden was dying every time I left. I knew what I was building, having thought about systems like it for a decade. What I did not expect was that other people would want one.

But they did. Friends came over for garden parties and stopped to ask about the drip lines, wanting to know how the system worked and whether I could set them up too. Some had never thought about where their irrigation water came from. Others had been trying to grow tomatoes and giving up. All of them wanted what was in my side yard.

That was when I understood that the international agricultural development I had spent my career inside was not as far from home as I had treated it.

Technology Transfer from the Global South

International agricultural development has been working on circular systems for as long as I have been in the field, and longer. It was not a brand but a necessity, because there was no other option.

The water you have is the water that fell. The inputs you have are the inputs you can carry in. The energy you have is the energy you collect. The equipment you have is the equipment you can repair. Smallholder agriculture, in the places I worked, was circular by necessity, and the systems that survived were the ones that closed their own loops.

I worked on circular economy directly during those years, by other names. The portfolio included a public-private partnership on recycling in Egypt, a social cost-benefit analysis of a solid waste management social venture, and research for a family foundation on Tetra Pak. The questions about flexible packaging, recovery, and what to do with what we had been calling waste were not new; they were just elsewhere.

We rarely called any of this what it actually was. Sometimes we called it resilience, sometimes climate-smart agriculture, sometimes appropriate technology. The frame that fit best was circular economy at human scale.

Agricultural technology transfer was supposed to be a one-way pipe, from research institutions and donor countries toward smallholder farmers. It has been flowing both directions for a long time. What I built in my Minneapolis backyard is just the latest piece of evidence.

The Upper Midwest Is Starting to Ask the Same Questions

The questions I worked on abroad are now the questions being asked here.

Climate is changing how water moves through this region, and soil health is back on the agenda after a generation of treating it as a fixed asset. Flexible film waste is becoming visible because it has nowhere else to go. Coalitions are forming around circular economy in food and agriculture, and capital is moving toward work that would not have been funded ten years ago.

Some of what is now being called innovation in domestic agriculture is what international agriculture has been doing for decades, at smaller scale, with less capital, and often with more ingenuity. I say this with affection for both fields, having worked in one for a decade and writing now from inside the other.

What Gets Counted, Gets Funded

The work that is keeping people fed in this region right now is mostly invisible to the people deciding where capital goes.

I am thinking about what grew here during Operation Metro Surge. Mutual aid networks expanded fast, community fridges multiplied, and neighbors shared canned food because it was the middle of winter. Now that it’s spring, neighbors are sharing seeds and seedlings, garden tools and unused rain barrels. Soon we will the fruits — the eggplants, tomatoes, herbs, and zuccinni. Inflation has pushed more people into household food production this year than at any point I can remember, and Operation Metro Surge made what was already happening visible to people who had not been paying attention.

None of this shows up in the data that funders use. There is no indicator for households closing their own water loop, and no line item for the neighbors who shared seedlings or the rain barrel that fed three gardens on the same block. The measurement infrastructure that exists in agriculture was built for commodity markets, supply chains, and donor reporting, not for what is happening on a residential block in Saint Paul or a community garden in North Minneapolis.

This is an evaluation problem with real stakes: what is not measured is not funded, and what is not funded does not scale. The household and mutual-aid scale of circular agriculture is doing significant work right now, in this region, under conditions that make it more necessary every month, and it deserves to be seen.

The next decade of agricultural evaluation has to take this seriously, not to replace what corporate sustainability reporting does but to complete it. The full picture of circular agriculture in the Upper Midwest includes the largest companies in the world and the gardener with a rain barrel, and right now the data only sees one of them.

What I Am Building

My son and I are building Greenway Drip, the simple drip system that started in my backyard and is now being made buildable for other people.

It is a solar powered, rain-barrel fed, drip irrigation system: the kind of system a smallholder farmer near the Nile would recognize, and the kind of thing the people at my garden parties kept asking how to copy. It is small, and that is the point. Circular agriculture has to be able to operate at every scale, including the one where a parent and a kid in a backyard want to grow food without leaving the grid running for three weeks.

It is also a measurement opportunity. A household that captures its own water, runs on its own solar, and grows its own food is a circular system that funders and policymakers cannot currently see, and if we build the tools to see it, the case for supporting it gets easier to make.

The future of circular agriculture in this region will depend on whether it can see itself at every scale. The largest companies in the world are working on it, and the gardener two blocks over is working on it too. The coalitions, capital, and measurement systems that get built in the next few years will decide whether both kinds of work get counted.

I think both kinds should.

For more information on our solar-powered, rain barrel drip irrigation system go to GreenwayDrip.Com and read our Substack, the Drip Drop.

Anthralytic is a strategy and evaluation studio working at the intersection of data, AI, and mission-driven impact. Subscribe for writing on evaluation, agriculture, AI governance, and what it takes to build systems that actually serve the people they claim to.

Share

Leave a comment

Related Posts

The AGI clause

In 2019, when OpenAI was still a nonprofit and Microsoft was about to invest a billion dollars, the two companies wrote an unusual provision into their contract. If OpenAI ever built AI that could outperform humans at most economically valuable work, Microsoft would lose access to it. The technical name for that kind of AI is artificial general intelligence, or AGI. The provision became known as the AGI clause.

The clause was meant to do something specific. OpenAI had been founded as a nonprofit with a mission to make sure transformative AI benefited humanity rather than any single company. Taking a billion dollars from Microsoft created an obvious tension with that mission. The clause was the contractual bridge between the two. Microsoft got commercial access up to the point where AI became transformative. After that, the mission was supposed to take over.

The bridge was already weaker than it looked. According to reporting from The Information, by 2023 the two companies had operationalized AGI in their contract as a system generating one hundred billion dollars in profits, a threshold OpenAI is nowhere near reaching. For the past three years, the clause has been a constraint that may never have operated in practice. But contingent constraints are still constraints. They shape how parties negotiate, what they assume about the future, and what they can be held to. On Monday, the contingent constraint became no constraint at all. AGI is no longer a trigger for anything in the agreement. It’s just a word now.

This is familiar territory for those in the social impact sector. There’s a name in the field for what happens when an institution’s stated mission and its actual operating logic come apart: mission drift, and it’s one of the oldest problems in our work.

A pattern within OpenAI

The clause didn’t get retired in isolation. Within OpenAI, it’s the third mission-era commitment that has been weakened in the past eighteen months.

The first was a hard ceiling on how much money investors could make from the company. When OpenAI took its first round of for-profit investment in 2019, it set a cap. Investors could earn up to a hundred times what they put in. Anything above that was supposed to flow back to the mission. That ceiling is gone. It was removed as part of OpenAI’s October 2025 corporate restructuring.

The second was nonprofit control over the for-profit business. The whole point of OpenAI’s original structure was that the nonprofit, with its commitment to public benefit, would oversee the commercial work. The October 2025 restructuring loosened that oversight. Critics including the OpenAI Files project have argued the nonprofit’s authority was meaningfully diluted. The mission language was preserved. The architecture that gave the mission force wasn’t.

Three commitments designed during the era when AGI was treated as a discrete future event have been retired or weakened in eighteen months. The AGI clause is gone. The profit cap is gone. The nonprofit’s grip on the for-profit has been loosened. That’s a pattern.

Readers from mission-driven organizations will recognize the shape of it. What’s unusual is what was being softened. The capped profit, the nonprofit oversight, the AGI clause: these weren’t standard commercial protections. They were specific commitments designed to address the specific problem of building something the founders themselves believed could be dangerous to humanity.

The other direction

Two weeks before the Microsoft and OpenAI announcement, Anthropic crossed a different threshold.

Anthropic is the company that makes Claude. It was founded by people who left OpenAI because they thought OpenAI wasn’t taking AI safety seriously enough. They built a different governance structure. A small body called the Long-Term Benefit Trust, made up of people whose job is to care about the mission rather than the stock price, picks members of the board. The Trust was designed in 2023 to phase in over time. It started with authority to pick a small number of directors. Eventually it would pick the majority.

That phase-in just completed. On April 14, the addition of one new board member pushed Trust-appointed directors into a majority for the first time.

It’s tempting to read this as the clean counterexample to the OpenAI story. Architecture used versus architecture abandoned. The temptation should be resisted.

The full text of the Trust Agreement hasn’t been published. Anthropic itself has acknowledged that stockholders can amend the Trust and its powers via supermajority, without trustee consent. The Trust didn’t exercise the board authority it already had until this month, despite having held that authority since 2024. The threshold that just got crossed is real. It’s also the first time the structure has actually been used. A governance instrument that hasn’t been tested isn’t yet evidence of constraint. It’s evidence of a possibility.

What we can say is that Anthropic has moved board control under its own published rules rather than around them. Whether the rules constrain anything the company wouldn’t have done anyway is a question the next twelve months will start to answer. The OpenAI move is a clear retirement of architecture. The Anthropic move is, for now, an activation that hasn’t yet been load-tested.

Two of the most consequential AI labs are responding to the same moment with opposite contractual gestures. Whether either gesture produces accountability is itself part of the story.

What contracts do that speeches don’t

Even as the contractual commitments have been weakened, public rhetoric about AI risk is at least as loud as it has ever been.

Anthropic’s CEO spent January at Davos warning that artificial general intelligence could cause civilization-level harm. He has been publishing essays predicting that powerful AI will arrive within the next two years. OpenAI’s leadership has been making similar predictions, more cautiously phrased but still extraordinary by any normal standard. The story that AI is heading toward a transformative event, one that will require new kinds of governance, is alive and growing.

Contracts and speeches do different work.

Contracts bind. Speeches don’t.

When OpenAI’s mission commitments lived in contractual machinery, they had teeth, or at least the appearance of teeth. The profit cap limited what investors could earn. The nonprofit board had real authority over the company’s leadership, and exercised it briefly in November 2023 before being reversed within days. The AGI clause threatened to constrain Microsoft’s access if AGI was declared, even after the 2023 profit-threshold operationalization rendered that constraint mostly theoretical. As those commitments have been moved out of contracts and into mission statements, blog posts, and CEO essays, they have lost even the contingent force they had.

Mission drift is what happens when an institution’s revenue or funding pressures begin to compromise its founding commitments. The deeper problem is older. Chris Argyris and Donald Schön distinguished between an organization’s espoused theory, the logic it communicates to the world, and its theory-in-use, the actual map driving decisions. Organizations fail to learn, they argued, when they ignore the gap between the two.

A parallel distinction lives in evaluation practice. A theory of change describes how change actually happens: what produces what, under which conditions, with what contributions from outside the organization. A theory of action describes only what the organization does. Most organizations think they have the first. What they have is the second, mislabeled. The mission language names outcomes the work is supposed to produce. The architecture is what produces them, or fails to. Without architecture, the mission language is a theory of action claiming credit it cannot earn.

The OpenAI case shows what happens when the architecture and the language come apart visibly. The capped profit, the nonprofit oversight, the AGI clause: these were the architecture. They were the mechanism by which the espoused theory was supposed to discipline the theory-in-use, but that architecture has been weakened. The mission language hasn't changed. The same words mean different things depending on whether they sit in the contract or in the press release. What used to be binding is now positioning.

The Google Approach

Google DeepMind offers a third pattern. When Google acquired DeepMind in 2014, the founders insisted on an AI ethics board as a condition of sale. The board was discussed but never properly stood up. What Google DeepMind has instead is the Frontier Safety Framework, an AGI Safety Council, a set of AI Principles. These are internal policy documents, not contractual constraints. They live entirely on the espoused side of the Argyris and Schön distinction. There's no architecture to retire because none was built.

What’s left

The mission commitments most worth taking seriously are the ones that have survived a real attempt to remove them.

OpenAI’s didn’t. The architecture is gone. The mission paragraph remains. Anthropic’s haven’t been tested. The Trust has authority on paper, but hasn’t yet been pressed against a decision the company would have made differently without it. Until it is, the structure is a hypothesis about constraint, not a demonstration of one.

The thing that gets quietly retired is rarely the language. The language stays. What gets retired is the machinery that made the language do anything.

For anyone in the social impact sector, the question to ask about a mission, your own or someone else’s, isn’t whether the language is good. The question is what architecture supports it, what that architecture binds, and what happens when someone with power tries to remove it.

That’s the part to watch.

Leave a comment

Share

Anthralytic helps mission-driven organizations clarify and amplify their impact. We work at the intersection of evaluation, AI governance, and the systems that shape both.

A few days later my son called someone an NPC.

That was two phrases in one week, from two different people. I have not been able to stop thinking about the pair.

What both terms do

“High value female” sorts people on a scale. Sexual desirability, utility, worth as a partner, an asset. It imports market logic into the description of a person and pretends that is just honest talk.

My son is reading The Handmaid’s Tale for school right now. The novel classifies women by function: Handmaid, Wife, Martha, Unwoman. “High value female” uses the same logic with better clothes and fewer categories.

Some people reach for “high value male” and assume the symmetry cleans it up. It does not. The logic of ranking a person by income, status, fitness, and dominance is the problem, not the gender of the person being ranked. Using the phrase for men spreads the commodifying frame rather than neutralizing it.

NPC does something different. It does not rank, it erases. The people around you are background characters in your game, not protagonists of their own lives. The term comes from video games, where non-player characters exist to give quests, sell potions, or stand in the way of the hero.

The mechanics differ. The move is the same. Both decide, quickly and cheaply, who counts.

My trade is valuation

Here is the uncomfortable part for an evaluator, an economist, or a sociologist. These careers all ascribe value to human lives, though in different ways, with different assumptions, and for different reasons.

Evaluation has the word value in it. Many of us in the field are uncomfortable with the word. The idea is that we are assigning worth to programs, interventions, outcomes, and sometimes to lives. That is the work I trained in.

I admit I am in the camp that thinks it may be better to find a different descriptor for what we actually do, because ultimately the point is improvement. We use the tools of assigning numbers in service of making something better than it was.

I have written reports that reduced human experiences to a rate, a number, a ratio, a score. I have done it carefully. I have done it with caveats and footnotes. I have still done it.

The phrases that made me flinch this week are crude versions of frameworks I use professionally, only with the assumptions and calculations obscured. The vocabulary is cruder. The logic is not alien.

How we value lives, officially

We value human lives in more ways than most people realize, and the ways contradict each other. Ultimately all of these methods exist to help us make choices about finite resources.

Statistical value of a life estimates what society will pay to reduce the risk of a death, and regulators use the number to justify safety rules and environmental protections. A dollar figure stands in for a person. The method sounds cold until you realize the alternative is not valuing the life at all, which is what the EPA did this spring when it set the value to zero in pollution rulemaking. The method also has a quieter problem. Willingness to pay correlates with income, so the math can encode, without saying so, that richer lives are more worth protecting than poorer ones.

QALYs and DALYs adjust years of life by quality or disability, which lets analysts compare one health intervention against another. The math is useful, though it is an abstraction, of course. The premise that a year can be weighted and summed across people is a premise, not a fact. Disability-rights scholars have argued for decades that these metrics can systematically undervalue lives with disabilities, because the weighting often reflects how non-disabled people imagine disability feels rather than what disabled people actually report. Early DALY calculations went further and weighted working-age years more heavily than childhood or old age. That assumption has been softened, but the instinct behind it, that productive years count for more, never fully left.

Human capital approaches value a life by projected future earnings. They are used in wrongful-death lawsuits, insurance settlements, and cost-benefit analyses of premature mortality. They consistently rank unpaid caregivers, retirees, and disabled people as worth less. Marilyn Waring dismantled the underlying logic in If Women Counted decades ago, showing how national accounting treats oil spills and wars as contributions to growth while rendering the labor that sustains everyday life invisible. We still have not fixed it.

Rights frameworks reject ranking altogether and assert equal, inherent worth. They are harder to operationalize, and no less real for that.

Indigenous relational ontologies locate personhood in kinship and reciprocity rather than in individual utility. Nicole Bowman and others have been clear that this is a methodology, not a metaphor, and it produces different questions, different data, and different ideas of what a good outcome looks like.

Bhutan’s Gross National Happiness framework decenters GDP and measures psychological well-being, community vitality, ecological health, and cultural continuity across nine domains. Countries that copy the headline miss the philosophy underneath it.

Rosenberg and Prilleltensky’s work on mattering asks a different question entirely. Does a person feel they are significant to others, and are they treated as significant in return? Mattering is not a score. It is a condition.

Each of these is a framework. Each is a decision about what counts. None is neutral.

The crude terms come from somewhere

“High value female” sits downstream of human capital logic and the “sexual market value” frame that moved from pickup-artist forums into the wider manosphere. That frame ranks women by youth, appearance, and perceived loyalty, and ranks men by status, income, and dominance. It imports market language into mate selection without embarrassment.

“Sexual market value” is, underneath, a back-of-the-envelope human capital calculation for the dating market. You could formalize it the way economists formalized wrongful-death valuations. Survey a sample of single people, ask them to weight the attributes they look for in a partner, aggregate the responses, and produce a score for any given person. That exercise sounds crass. It is also roughly what dating apps already run. Algorithms assign desirability scores to users based on who swipes on whom, and match accordingly. “High value female” is the vernacular translation of a calculation the apps perform silently on everyone who signs up.

NPC sits downstream of game design and a gamer worldview, with a dose of online contempt for people perceived as following scripts rather than thinking.

Neither term invented its underlying logic. Each inherited an older frame and pushed it one honest step further.

What the crude terms do differently is drop the veil. The respectable frames keep their ranking logic politely abstracted. “Customer lifetime value” does the same sorting work as “high value female,” only inside a spreadsheet, with better grammar and a quarterly review. A recommendation algorithm treats most users the way NPC treats most people, only with better manners. The crude words say the quiet part. The respectable frames murmur it.

That is what made me flinch. The teenager is naming, a little too accurately, what the systems around him already do. He does not realize he has done this.

The systems are already sorting

This is not a fringe vocabulary describing a fringe practice. Algorithms rank people by predicted value at scale every day. A job applicant is scored and filtered before a human sees the file. A patient is flagged for adherence risk, a driver for fraud risk, a user for engagement potential. Credit models decide who gets a loan and at what rate. Customer lifetime value decides who gets the support line that actually picks up. Content moderation queues and targeted advertising both run on inferred worth to someone.

The infrastructure for sorting humans by estimated value is built, deployed, and mostly invisible to the people being sorted. The crude words are the surface version of a practice that has become much more sophisticated in polite company. When my son calls someone an NPC, he is not describing a novel move. He is describing, in plain English, what the systems around him do constantly, more thoroughly and less visibly.

If the vocabulary feels new and ugly, the logic it names is older and better dressed.

Valuable to whom, for what purpose

In this work I return to one question: valuable to whom, and for what purpose? The question does not dissolve the problem. It names it. Every valuation has a perspective baked in, and pretending otherwise is how we got here.

Some answers do not resolve to a score. Whether a person matters, whether a child is treated as real, whether a woman is seen as a whole person rather than ranked on a scale: these are not measurement problems. They are moral stances that measurement can serve or distort, but cannot replace.

What I did not say well

I do not know that I corrected my son well. I am a slow processor, and I often need to wait for my thoughts and feelings about a conversation to emerge. I said something about the people he was calling NPCs being as real as he is, as fully occupied with their own lives as he is with his. He nodded the way teenagers nod. It may or may not have landed.

I do not know that I told my friend anything useful either. I made a face. I made a small comment about how I do not love that term. I changed the subject. Neither of us came out of it thinking harder.

That is partly why I am writing this. The frameworks I flinched at this week are not fringe. They are the vernacular version of infrastructure I participate in. Calling the vocabulary ugly is easy. Interrupting the logic upstream is the work.

Leave a comment

Share

Anthralytic is a strategy and evaluation studio helping mission-driven organizations clarify and amplify their impact. We work on the question of what counts, and for whom, without pretending the answer is obvious.

The floor debate cited two numbers on the pro-repeal side. Seven hundred and fifty jobs. Trillions of dollars in critical minerals. Those are the metrics that won.

I have spent most of my career watching decisions get made on the strength of the wrong numbers. Not wrong in the sense of false. Wrong in the sense of narrow, early, and unchallenged. This is a decision like that. It is also, if you care about evaluation, a case study in how measurement works as a gatekeeper. What counts as evidence is the whole question. By the time anyone votes, the real contest is already over. And the people who lose are never in the room.

The metrics that did not make the floor

There is a peer-reviewed economic analysis of this exact tradeoff. It was published in Ecological Economics in March 2020, by James Stock, a former chair of Harvard’s economics department and former member of the White House Council of Economic Advisors, and his coauthor. They modeled seventy-two income scenarios and thirty-six employment scenarios comparing a twenty-year mining ban against the proposed Twin Metals mine. The ban produced more jobs and more income in ninety-six percent of scenarios. In the best case for protection, the region would see up to 4,500 more jobs and up to $900 million more in personal income over twenty years than it would under the mining scenario. The authors received no compensation and disclosed no financial interest.

Seven hundred and fifty jobs from the mine was the most generous estimate. Forty-five hundred over twenty years was the comparison. One number made it to the floor. The other did not.

There is also a study published in March 2025 by Northeastern Minnesotans for Wilderness that looked at eight modern hardrock mines permitted in the United States since 1990. All eight degraded downstream surface water. Seven of eight polluted groundwater. Data was unavailable for the eighth. At the six mines where predictions could be compared to actual outcomes, water pollution was significantly worse than forecast. At the Kensington mine in Alaska, sulfate levels downstream rose fifty-fold. At the Beartrack mine in Idaho, mercury levels in brook trout tripled, making them unsafe to eat by EPA standards. Water quality often got worse after the mines closed. Two of the three mines in closure had their worst water impacts after operations stopped.

Modern mining’s own predictive models are biased in a known direction. They underestimate pollution. The error is systematic and documented. A 2013 Earthworks report found that forty existing hardrock mines in the U.S. generate an estimated 17 to 27 billion gallons of polluted water each year, every year, in perpetuity. Treatment costs run into the tens of billions annually.

None of this was the frame on the Senate floor.

The time horizon problem

Part of what happened on April 16 is that three different clocks were placed on the same scale and weighed as if they were comparable.

The jobs clock runs five to twenty years. That is how long a mine operates before ore bodies deplete or prices shift. The critical minerals clock runs ten to thirty years, the window in which a given deposit matters for a given technology. The pollution clock runs centuries. Acid mine drainage at similar sites has required water treatment in perpetuity. The Forest Service has called this kind of mining, in this kind of place, an “unacceptable risk of irreparable harm.”

You cannot weigh these against each other without first deciding whose time frame counts. The vote decided. The short clocks were treated as the real numbers. The long clock was treated as a caveat.

This is a familiar move in program evaluation. Ask a funder what success looks like and you will often get a twelve-month answer for a problem that takes a decade to move. The evaluation that follows inherits that frame. Whatever it measures well, it measures inside a horizon that was never the actual horizon of the work.

Reversibility is not optional

The Congressional Review Act mechanism used to pass this resolution prevents any future administration from issuing a substantially similar protection without new congressional authorization. That is a legal fact, not a rhetorical one. It means one side of the decision, access to minerals, is treated as urgent and flexible. The other side, wilderness once contaminated, is treated as permanent and subject to political reversal.

This is backwards. The reversibility of the two sides of this decision is not symmetrical. Critical minerals policy can change. Watersheds, once contaminated with sulfuric acid and heavy metals from sulfide-ore mining, cannot be uncontaminated. There is no mine in the United States that has demonstrated it can stop acid mine drainage on a large scale once it begins.

Any evaluation worth the name weighs reversibility. When one branch of a decision tree is genuinely irreversible, the cost of being wrong on that branch is categorically different from the cost of being wrong on the other. A measurement framework that does not surface that asymmetry is not neutral. It has already chosen.

The evidence that the critical minerals frame does not hold

Even on the pro-mine framing’s own terms, the argument is thin. The USGS added copper to its Critical Minerals List in November 2025, but as recently as the 2021 draft list the agency declined to include it, citing “domestic production, lack of import dependence.”2 The United States is already a top-five world producer. Sixty-five percent of U.S. refined copper imports come from Chile3 , the same country that owns Twin Metals.

The mine is a project of Antofagasta plc, a London-listed Chilean conglomerate controlled by the Luksic family. Antofagasta ships its copper concentrate from its Chilean operations to smelters in China. It is the industry benchmark seller. In December 2025, the company agreed to zero-dollar processing fees with a Chinese smelter for 2026, meaning Chinese smelters now process Antofagasta’s concentrate for free because their capacity so dominates the market4. Twin Metals’ own production would likely follow the same path.

“Critical minerals for America” is the argument. “Chilean-owned mine shipping concentrate to China” is the project.

The gap between those two sentences is where measurement-as-gatekeeper lives. A decision made on the first framing would be different from a decision made on the second.

The quiet part

This is an evaluation story, not just a policy story. The fight over the Boundary Waters was lost at the metric-selection stage, well before the vote. That is the pattern to notice. Most of the consequential fights are lost there. By the time we are counting, we have already decided what counts.

Evaluators know this. We live with it. Every program we assess has a logic model someone else drew, outcomes someone else named, and a time horizon someone else chose. Our rigor operates within a frame we did not set. The most honest thing an evaluator can do is name the frame and its exclusions, even when naming them changes nothing about the verdict.

In this case, the verdict is a 50-49 vote. The exclusions are a peer-reviewed economic study, a documented pattern of prediction error in modern mines, eighteen billion gallons a year of perpetually polluted water at similar sites, a Forest Service finding of unacceptable risk, 1854 treaty rights to hunt, fish, and gather manoomin in the Ceded Territory, and roughly two-thirds of a million public comments favoring protection.

Seven hundred and fifty jobs counted. Forty-five hundred did not.

That is the evaluation.

Leave a comment

Share

Anthralytic is a strategy and evaluation studio for mission-driven organizations. We help people make decisions about resources in the presence of uncertainty, long time horizons, and values that do not reduce to a single number.

I worked for Land O’Lakes Venture37 for a decade. That organization and many others that usually are in the space aren’t there. Not because they have decided not to bid this year, but because they no longer exist. The orgs still bidding are fewer, and the gaps are real.

I want to talk about why the names that are missing are missing, because the answer says something uncomfortable about how this field actually works.

How Food for Progress is structured

For readers who don’t live inside this work, a quick explainer.

USDA’s Food for Progress are federal cooperative agreements, which means they don’t pay profit or fee. The implementing organization takes American agricultural commodities, sells them in the country where the project is implemented, and uses the proceeds to fund the project’s activities. Those proceeds are not the implementer’s to keep, and they have to be spent on the project as specified in the agreement. For-profits can technically apply but rarely do, because there is no profit advantage to be had, which means whoever wins these awards operates under the same financial constraints regardless of tax status.

So how do these implementing organizations actually pay for the things that aren’t project-specific? The HQ finance team, legal, monitoring and evaluation methodology, country offices, and all the shared infrastructure that makes any of the work possible.

There is one mechanism for that, and it’s called indirect cost recovery. Every implementer with a federal portfolio negotiates a Negotiated Indirect Cost Rate Agreement (NICRA) with what is called its cognizant federal agency, meaning the agency that provides the largest share of its federal funding. That rate gets applied across the org’s whole federal portfolio, and it is the only way to recover shared infrastructure costs from federal awards.

The NICRA system is built on the assumption that an org’s federal portfolio is reasonably stable across multiple agencies over time. There’s a five-year cognizance rule, and negotiated rates apply to all federal awards. The whole structure assumes the floor doesn’t drop out from under you.

What happened when the assumption broke

USAID closed on July 1, 2025, with eighty-three percent of programs cancelled and the remaining functions transferred to the State Department. NICRA cognizance for affected organizations transferred too, with a new email address at AQM-NICRA@state.gov for orgs with transferred awards.

The structural problems are several layers deeper than the email address suggests.

The cognizance rule is sticky by design, which means an org whose USAID portfolio collapsed in early 2025 still technically has USAID, now State, as its cognizant agency, even if its real federal portfolio now consists of a couple of USDA awards. Cognizance can’t legally shift quickly. State also inherited a system that was already failing audits: the 2024 USAID Office of Inspector General audit found the agency was tracking NICRAs in an Excel spreadsheet called “NICRA Tracker,” with automation planned for 2025, the same year the agency was dismantled. Whatever State got, it isn’t infrastructure built for this scale.

The math problem is the layer underneath all of it. The fixed indirect cost pool didn’t shrink when the USAID portfolio did. The HQ staff are still there, the country offices are still there, the compliance infrastructure is still there. But the direct cost base those costs get applied to has collapsed, so the rate is the same and the dollars are not.

Venture37, briefly and honestly

Venture37 was structurally tied to American agriculture through Land O’Lakes, the cooperative, which is the kind of structural alignment that should have made it durable in this exact moment, when the political climate favors American agricultural interests and the FFPr program continues. It folded anyway.

The international development work had to fund itself out of its own portfolio, and when the USAID side of that portfolio collapsed, the remaining USDA work could not carry the indirect costs. The parent company was not going to absorb the gap, and the math didn’t add up. That is the whole story. There are political and contextual layers around it, but the mechanism is the math.

What this is not about

It is not about whether the work was working, and it is not about evaluation findings or which organizations were better at impact or worse at it. The decisions about which orgs continue and which don’t are being made on financial structure, and impact is not what’s deciding this.

I want to sit with that for a minute, because I think it deserves more weight than it usually gets.

We spend our careers, those of us who do this work, trying to make impact legible to the people who decide where resources go. We build measurement systems, we run evaluations, and we produce findings that are supposed to inform resource allocation. The whole point of evaluation, in the version of it I was trained in, is to put evidence in front of decision-makers so the decisions get better. And then a moment like this comes, and the resource allocation happens on financial mechanics that have nothing to do with any of it. The orgs that survived didn’t survive because the evidence said they should, but because their financial structures could absorb the shock.

That’s not a critique of the survivors, who are doing what they have to do. It’s a question about what the evaluation work was for.

Two questions that don’t sit easily together

There are really two existential questions running underneath this moment, and they don’t sit easily with each other.

The first is financial. Can the math add up. Can the org carry its indirect costs. Can the cooperative agreement model survive the collapse of its largest funder. This is the question that decided who’s at the FY26 table and who isn’t.

The second is about the vision of the work. What is this field for. Who is it accountable to. Whether it is actually doing what it says it’s doing, and whether the answer to that question matters to the people making the resource decisions.

In social impact, we often act as if the second question is the one that drives everything, because it’s the one we built our methodologies and our careers around. But moments like this one make it hard to keep believing that. The financial question is the one that’s actually deciding outcomes right now, and the vision question is being answered by what survives, not by what the evidence says should survive.

I don’t think these two questions can be collapsed into one, and I don’t think one is more real than the other. But they are pulling in different directions, and the pulling is what this moment is exposing.

Lingering questions

There’s a lot I can’t claim about this moment, and I want to be honest about that.

I don’t know whether the broader field is being reshaped in some patterned way, because the surviving orgs are an idiosyncratic mix and any pattern I might claim would be too clean for the actual data.

I don’t know whether the orgs at the FY26 table now will still be at the table in five years, because the math problem doesn’t go away just because you survived this round. I don’t know whether the State Department’s AQM-NICRA team can actually handle the volume of renegotiations coming, or whether orgs are going to spend years in indirect cost recovery limbo. I don’t know whether there is any version of the federal cooperative agreement model that survives this transition, or whether something different replaces it.

What I do know is that the proposal teams are forming and the work continues for the orgs that can still carry it. The question I am sitting with as an evaluator is what we owe to the work that didn’t get to continue, and to the people who did it, when the reason it stopped had nothing to do with whether it was good.

Leave a comment

Share

Anthralytic is a strategy and evaluation studio that helps mission-driven organizations clarify and amplify their impact.

The latest example is agricultural, but the pattern is not.

In December 2025, the USDA launched a $700 million Regenerative Pilot Program through its Natural Resources Conservation Service. Farmers who enroll conduct a whole-farm assessment, adopt at least one primary practice from a list of fifteen, and agree to soil health testing at the start and end of a five-year contract. Outcomes are tracked at the farm level against resource concern thresholds.

That is what the program does.

What the program claims is something else. The stated purpose ties the work to reducing chronic disease and improving community prosperity, under the banner of Make America Healthy Again. The Department of Health and Human Services is separately funding research on the connection between regenerative agriculture and public health.

Read that again. The research on the connection is happening after the program launched.

A soil test on a farm in Iowa cannot tell you whether chronic disease is declining. Those are different scales of claim. Farm-level resource data cannot be aggregated into population-level health outcomes without a set of intermediate steps that this program does not articulate. The measurement that exists is fine for what it measures. It cannot do the work the claim is asking it to do.

The missing floor

The step that would at least begin to bridge the scales is a theory of change. I do not mean a document. I mean a conversation. It is the conversation where someone says out loud: we believe that if farmers do this, then that will happen on the farm, which might contribute to the other thing at the community level, because of these intermediate mechanisms we can name and, where possible, test.

I want to be honest about what a theory of change is and what it is not. It is an abstraction. It cannot capture every condition, every feedback loop, every way one outcome might shape another. It will not tell you whether the causal arrow runs forward or back, or whether two outcomes are really one outcome in disguise. Causal loop diagrams try to hold all of that and collapse under their own weight. A theory of change lives in the tension between specifying everything and specifying nothing, and it chooses to specify the bridge.

Which is why it matters here. A theory of change is not the whole answer. It is the floor. It is the least strict requirement to at least try to do what you are claiming. It forces you to name your assumptions so they can be examined, argued with, and, where the data allows, tested.

Anything less than that is not a program. It is a wish with a budget.

This is not a piece about farming

The pattern I am describing is not unique to agriculture. It shows up any time a program’s activities operate at one scale and its claims operate at another, and no one in the design room does the work of naming the bridging assumptions. I have seen it in education funding. I have seen it in public health initiatives. I have seen it in workforce programs and housing pilots. The particulars change. The shape does not.

The organizations that end up doing the work are the ones who hit the activity targets, file the outcome reports, and wonder later why the numbers do not add up to the thing everyone said they wanted. They did not design the gap. They inherited it.

The first conversation

The next time you see a big check attached to a big claim, the first question is not what should we measure. The first question is whether anyone in the room can say out loud, in one sentence, how this work is supposed to get to that outcome. If nobody can, no measurement framework will save it.

That sentence is the floor. It is the first conversation, not the last.

Share

Leave a comment

Anthralytic is a strategy and evaluation studio helping mission-driven organizations clarify and amplify their impact. We build the tools and conversations that help teams name what they are doing, why they believe it will work, and how they will know.

I had a conversation recently that helped me name the thing I’ve been seeing for a long time. I’ve known the pattern was there. I just hadn’t put my finger on it.

Let’s dive in.

One way to conceptualize the social sector’s relationship to measurement, evaluation, and learning is a two-by-two. Understanding which quadrant you’re talking to changes everything about what you build and who you build it for.

The Two-by-Two

If you map organizations on two axes, openness to better measurement and potential gain from better tools, four quadrants emerge.

The not-yet-ready sit in the lower left. Low openness, real but unrealized potential gain. These are organizations doing critical work, providing beds, meals, services, holding communities together with sweat and grit. They’re not hostile to measurement. They’re just focused on something else right now. The potential gain is real, but the readiness isn’t. They’ll come when the tools are proven, normalized, and easy enough that adoption doesn’t feel like a project.

The already-there sit in the lower right. High openness, low marginal gain. Organizations with dedicated evaluation staff, external evaluators, quasi-experimental designs. They’re already doing it. They’re already convinced. Better tools might sharpen their work, but the most sophisticated organizations in any sector are not where the unlock is.

The trap sits in the upper left. Low openness, high potential gain. Organizations that could benefit enormously from better tools but won’t engage. Maybe it’s leadership resistance, maybe it’s cultural, maybe it’s just not the right moment. The temptation is to pour energy here because the theoretical payoff is so large. But theoretical payoff and actual adoption are different things, and the field has spent decades learning that lesson the hard way.

The upper right is high openness and high potential gain. Organizations that care about impact, are open to better approaches, and where the gap between where they are and where they could be is enormous. The two-by-two tells you this is the highest-return quadrant. But the two-by-two alone doesn’t tell you what’s actually happening inside it.

For that, I want to shift to a different frame, one grounded in social and behavior change science.

The Moveable Middle

The term “moveable middle” comes from public opinion research, where it describes the segment of an audience that isn’t already committed but can be reached with the right message and the right tools. If you know Rogers’ Diffusion of Innovation model, you’ll recognize the pattern. The already-there are the early adopters. They moved first because they had the resources and the conviction. The not-yet-ready are the late majority. They’ll come when the path is clear. And the moveable middle is the early majority, the group that determines whether better measurement ever becomes normal in the social sector.

The chasm in Rogers’ model falls exactly where the design problem lives: between the organizations that already do this work and the ones that would, if someone built the bridge. I’m borrowing this frame because the two-by-two tells you where organizations sit. The moveable middle, and the diffusion curve behind it, tells you why they’re stuck and what it takes to move them.

These are organizations that care about impact. They’re already doing some form of measurement. But they’re trapped in what I’d call the compliance loop: tracking what the funder asks for rather than what would actually help them learn and improve. Historically, nonprofits have focused on evaluating social impact to meet reporting requirements, with limited space to innovate or leverage evidence from their own services. Lean staffing and restricted funds keep them locked in that cycle, tracking outputs for compliance instead of learning what’s working.

And here’s the part that should bother everyone: both sides lose. Grantee organizations report on indicators that may be less useful to them, often at the expense of more mission-critical work. And funders, despite requiring those measures, end up with a shallow understanding of what their money actually did.

The middle isn’t opposed to doing this better. They face a capacity gap: not enough funds, not enough people, not enough time, not enough evaluation expertise, and too little support from boards and leadership. It’s not that they don’t want to think more carefully about impact. It’s that they can’t afford to, and nobody has shown them a right-sized version of what that looks like.

I’ve seen this up close. Organizations that can’t answer basic questions about their own effectiveness because nobody set up the system to find out. Organizations that call asking for “an evaluation” when what they actually need is help with program design. I once had a client ask for “an evaluation” and mean two completely different things in two different conversations. The terminology itself is a barrier. Words like “evaluation” and “performance management” shift meaning depending on who’s in the room, and the confusion starts before anyone picks up a survey instrument.

This is the moveable middle in practice. They have the intent but not the infrastructure. They’re not a hard sell. They’re an unserved market.

Here’s the thing about the moveable middle: they’re not resistant. They’re the first in line for something built for them. The already-there adopted sophisticated evaluation years ago because they had the budget and the staff. The moveable middle will move the moment someone bridges the chasm with the right tools at the right scale. And when they do, when they have results to show, the not-yet-ready won’t be far behind.

The Design Problem

The social impact field keeps building for the already-there and then wondering why nobody else adopts it. We produce sophisticated frameworks, dense methodological guides, and expensive consulting engagements that assume organizations already have the capacity to absorb them. Then we’re puzzled when the moveable middle keeps doing what it’s always done: tracking outputs for the funder report and calling it evaluation.

What the moveable middle needs isn’t a watered-down version of what the top tier uses. It needs something fundamentally different: right-sized tools for the full cycle of monitoring, evaluation, and learning. Tools that start where organizations actually are, not where we wish they were.

The advice that resonates most with me is the simplest. Start with a back-of-the-envelope theory of change. Ask: what do you know and not know about how you are making an impact? Prioritize what to measure based on what you don’t know. Start small and early. Go through the cycle once, then iterate. Don’t build a dashboard. Track three metrics in a spreadsheet and review them during a leadership call.

That’s not dumbed-down rigor. That’s rigor that fits.

What I’m Building

This is the space Anthralytic works in, and it’s where I’ve been putting my energy.

I’ve been experimenting with lightweight tools designed specifically for the moveable middle. An evaluability assessment that helps organizations figure out what they’re actually ready to measure before they spend money finding out. An impact wizard that walks teams through the basics of articulating their theory of change without requiring a consultant in the room. These are early experiments, but they’ve confirmed something I suspected: when you meet organizations where they are, with the right language and the right scope, they move. Quickly.

What I’m building now takes that further. It’s a set of tools and targeted consulting designed to help nonprofits through the full cycle of monitoring, evaluation, and learning. Not a $40,000 evaluation. Not a team of PhDs. Something accessible, right-sized, and designed for organizations that have the intent but not the infrastructure. Something that makes measurement useful instead of performative.

More on this soon.

If the moveable middle is where the real leverage is, then the solution has to be built for the middle. That’s what I’m doing.

Anthralytic is a strategy and evaluation studio helping mission-driven organizations clarify and amplify their impact.

Leave a comment

Share

I know because I have done this work. Years ago, I conducted a social cost-benefit analysis in Egypt for a social enterprise in solid waste management and recycling. We scoped it carefully. We monetized what we could. And at the end, the analysis was defensible but incomplete, because that is what these analyses always are. Later in my career I conducted many analyses with a kind of SCBA-light methodology because the full methodology takes months and the underlying valuations are always contestable.

Subscribe now

This is not a confession of failure. It is a description of reality. And it matters because across philanthropy, government, and the social sector, there is an honest and reasonable desire to know whether the money is working. The question is whether the tools we reach for actually answer that question, or whether they just make us feel like we have answered it.

The Toolkit Has Real Limits

Social cost-benefit analysis (SCBA) and social return on investment (SROI) both require you to monetize outcomes. What is housing someone worth in dollar terms? Fifty thousand in avoided shelter costs? Thirty thousand? Eighty thousand? Two analysts can produce wildly different numbers from the same program data. That is not a flaw in either analyst. It is a feature of a methodology that asks you to assign dollar values to things that do not naturally have them.

The incentive structure compounds the problem. When your continued funding depends on demonstrating impact, and the methodology lets you adjust assumptions, the pressure to produce impressive numbers is real. I have seen it. Funders end up preferring whatever is easiest to monetize rather than whatever produces the most meaningful change, and the measurement tail wags the program dog.

Quasi-experimental evaluation sits at the other end of the spectrum. It can isolate a program’s effect with genuine rigor. On a USDA trade program in Egypt, I managed a team from NORC that designed a quasi-experimental approach using commodity-level trade data, comparing export trajectories of program-targeted commodities against similar ones to isolate the program’s contribution within a volatile trade environment. The methodology was sound. It was also expensive, time-consuming, and required a specialized team. You cannot run that kind of analysis across a portfolio of twenty investments. It answers one question very well, but it cannot be your operating system.

What Cost-Effectiveness Gets Right

Cost-effectiveness analysis sidesteps the monetization problem entirely. It keeps outcomes in their natural units. How many people did you house, and what did it cost per person? The dollars stay on the spending side, not the value side. You never have to argue about what housing is “worth.” You just know what it costs to produce.

This is genuinely useful for funders comparing programs that pursue the same outcome. But cost-effectiveness can mislead if you strip out the context. Program A costs more per family housed, but rents in that geography have doubled over the past two years. Program B looks efficient on paper, but that reflects a favorable housing market, not better program design. The number is clean. The story underneath it is not.

Pair cost-effectiveness with real understanding of conditions, and you get intelligence that actually improves allocation decisions. Use it alone, and you get a spreadsheet that rewards luck.

Value for Money at Different Scales

Here is where it gets practical. Not every investment needs the same level of analysis, and pretending otherwise wastes time and money on both sides.

Julian King, a New Zealand-based evaluator, has developed what he calls the Value for Investment (VfI) approach, and it offers a useful way to think about this. VfI reframes value for money as a question about good resource use rather than a question that can only be answered with dollar figures. Instead of requiring you to monetize everything, it uses evaluative rubrics, transparent criteria, and multiple sources of evidence (qualitative and quantitative) to make structured judgments about whether an investment is creating enough value. The approach treats policies and programs as investments in value propositions, with the potential to create social, cultural, environmental, and economic value, not just financial returns. It is not another method. It is a system for guiding the use of existing methods, matched to the question and the context.

This matters because the funder’s real question is rarely “what is the precise social return on this dollar?” It is usually something more like: Is this working? Should we keep funding it? What would make it work better? A small community grant does not need a quasi-experimental design. It needs a clear logic model, honest reporting on what happened and what did not, and a conversation about what was learned. A multi-million-dollar systems change initiative probably does need a more rigorous approach, but even then, the question should drive the method, not the other way around.

Those questions can be answered at a fraction of the cost of a full SCBA, if you are willing to accept that the answer will be judgment-informed rather than mathematically precise. King’s VfI framework makes that judgment explicit and transparent rather than hidden behind a veneer of false precision.

The Answer Is That It Depends

I am not ideological about methodology. If there is something better and simpler that gives funders what they need to make good decisions, I want to find it. But after years of doing this work across different contexts, countries, and scales, I have come to believe that the tension between rigor and practicality is not a problem to be solved. It is the work itself.

The honest answer to “which methodology should we use?” is that it depends. It depends on the question you are trying to answer, the resources you have, the stakes involved, and what decisions the analysis will actually inform. That is not a dodge. It is the most rigorous thing I can tell you.

The danger is not in choosing a less-than-perfect methodology. The danger is in choosing a methodology because it looks rigorous rather than because it answers the question you actually need answered. Or in demanding a level of precision that the underlying data cannot support. Or in treating measurement as a substitute for judgment rather than an input to it.

The best evaluation I have seen did not always use the most sophisticated methods. It used the right methods for the situation, reported honestly about limitations, and produced findings that someone actually used to make a better decision. That is a lower bar than “prove this worked.” It is also a more honest one.

I think that tension, the pull between wanting certainty and accepting that good judgment is the best we can offer, is actually healthy. It keeps us honest. It keeps us asking whether the measurement is serving the mission or the other way around.

The work is learning to sit with that tension rather than resolving it prematurely with a number.

Anthralytic is a strategy and evaluation studio helping mission-driven teams clarify and amplify their impact.

Share

Leave a comment

I was writing about geopolitics. But I was also writing about something I see constantly in social impact work.

There is always a moment during a theory of change workshop when someone questions whether an outcome causes another outcome or is caused by it, or whether they simply arise together, each conditioning the other. It is a good question. It is also a question that the logic model format cannot hold. Logic models draw arrows. The world is a web. Outputs become inputs. Outcomes shape other outcomes. The system reorganizes continuously, and the chain cannot hold what is actually happening.

That is true at the scale of nations. It is also true at the scale of a housing program in Minneapolis or a health intervention in rural Guatemala. The scale is different. The structure of the problem is the same.

I have been building something that takes that seriously.

The Problem

I want to be clear about something. Theories of change and logic models are useful. I have built them. I have helped organizations build them. They organize thinking, they make programs legible to funders, and they force a kind of discipline about what you think is going to happen and why.

But they are abstractions. They are maps, not the territory. And like all maps, they have limitations. The most important limitation is that they draw arrows where the world has webs. They flatten mutual causality into sequence. They make it look like conditions hold still while programs move through them. Conditions do not hold still.

Most organizations build logic models in a conference room or a grant application, drawing arrows from activities to outcomes without asking what conditions have to be true for those arrows to hold. The arrows assume a world. Nobody checked whether that world exists.

A job training program assumes employers are hiring. A housing program assumes available units. A health intervention assumes people can get to the clinic. These are not inputs. They are not things the program controls. They are conditions, the landscape the program operates within, and when they shift, the logic model breaks without anyone understanding why.

The usual response is to add more indicators. Track more outputs. But the problem was never insufficient measurement. The problem is structural. A chain cannot hold what is actually happening. It is really more of a web than a chain.

What the Conditions Web Does

The Conditions Web is an AI-guided conversational mapping process that helps social impact organizations see the full landscape of conditions their program operates within, before they design strategy or evaluation.

Instead of starting with a logic model, it maps conditions across eight domains of social reality: historical and systemic forces, the current situation, the population served, community and cultural life, structural and political rules, organizational dynamics, market and economic conditions, and ecological and place-based factors.

The mapping happens in two layers. First the organization itself: what has to be true internally for it to do its work. Then the program level: what surrounds the people it serves. The result is a relational web of interconnected conditions rather than a linear chain of causes and effects.

Cross-cutting dimensions like power, time, felt experience (whether people feel like they matter, like they belong, like their dignity is intact), and leverage are woven through every domain rather than treated as separate topics. Subpopulation variation gets surfaced early, because a conditions map that collapses meaningfully different paths into a single story is not a map at all.

How It Works

The conversation happens in plain language. The eight-domain framework is the internal structure, not jargon imposed on the user. The person doing the mapping just talks about their work, their community, their situation. The structure stays in the background where it belongs.

Sessions run five to fifteen minutes per domain, so organizations without dedicated evaluation staff can do the mapping in pieces over multiple sittings. You do not need an evaluator on staff. You do not need to know what a theory of change is. You just need to be willing to describe the world your program operates in honestly.

What It Produces

From the same underlying map, the tool generates what both the organization and its funders need: an interactive visual map of conditions, a narrative summary, and a theory of change and logic model that are grounded in the actual situation rather than built from scratch in a vacuum.

The logic model still has arrows. Funders still need them. But the arrows are grounded in a web of conditions that were actually examined rather than assumed. And when conditions shift, the organization can see why outcomes changed rather than just reporting that they did.

Why Now

The field has no shortage of measurement tools: developmental evaluation, contribution analysis, outcome harvesting, most significant change, and more. Every one of them is better than nothing. But none of them helps if you have not seen the system you are trying to measure.

You cannot evaluate a program’s contribution to systems change if you have not mapped the system. You cannot assess whether conditions shifted if you never documented what conditions existed at the start. You cannot distinguish between a program that failed and a program that operated in conditions where success was not possible.

The measurement comes after. The seeing comes first.

Try It

The Conditions Web is still under construction. Things will be rough. But I would rather put it in people’s hands now and learn from how they use it than polish it in private until it is perfect.

If you work in social impact and want to see what conditions mapping looks like in practice, try it and tell me what works and what does not.

Link to Conditions Web

Anthralytic is a strategy and evaluation studio helping mission-driven teams clarify and amplify their impact.

I thought about doing a retrospective. I thought about a subscriber drive. I thought about a greatest hits list. Instead I want to leave you with a question. It’s a simple question, but like a koan, the more you meditate on it the more complex it becomes.

If you stopped measuring, would you still know if it was working?

Before you try to answer it and move on. Wait. Sit with the question for a second.

Most people I work with in the social sector would immediately say yes. They would know. They would know from the conversations that changed, from what they see shifting in the people they serve, from what the community tells them without anyone designing a survey. The knowing is already there. It has been there the whole time.

So if you would still know, then what is the measurement for? Who is it serving? Is it helping you learn, or is it helping you prove something to someone else?

It’s not a trick question. Accountability matters. Funders have a right to understand how resources are used. But there is a difference between measurement that serves learning and measurement that serves justification. Between evaluation that helps a community see what it already knows and evaluation that extracts data from a community so someone far away can make a decision.

And some of you would sit with that question and realize no, actually, you would not know. You are so deep inside the work that you have lost perspective. For you, the question points the other direction. Not away from measurement, but toward it. Toward evaluation as a way of seeing what you are too close to see.

Both answers are honest. That is what makes this a real question and not a gotcha.

I have spent nearly one year writing about evaluation, AI, social impact, and also Zen, and the spaces where those things press against each other. The thread running through all of it is this: the purpose of evaluation is not the report. It is not even the learning. It is the human connection that happens when people sit together and try to understand what is working, what is not, and what to do next. The value is relational. It is reciprocal. It lives between people, not in a dashboard.

That is also why AI in this space makes people uneasy. Not because the technology is inherently bad, but because it threatens to replace the relational core with something algorithmic. Something that feels efficient but hollow. It does not have to be that way. AI can be a tool that supports human connection rather than substituting for it. But only if we are honest about what we are protecting and why.

So here is the question again. No framework attached. No rubric. No theory of change.

If you stopped measuring, would you still know if it was working?

I do not know your answer. But I think the question is worth 100 more posts and I invite you to sit with it today.

~Sadie

Anthralytic is a strategy and evaluation studio that helps mission-driven teams clarify and amplify their impact. If you make decisions about resources in the social sector, whether you call yourself an evaluator or not, this newsletter is for you.

Share

Leave a comment

Here is a scene that will be familiar to many people who work for small or resource-constrained non-profits.

It’s December. Time for the funder report scramble.

Someone opens a spreadsheet that was last updated in March. A program manager pulls numbers from memory, or from emails, or from a stack of sign-in sheets that have been sitting in a folder since June. Or from a haphazardly created database by someone who doesn’t actually know how the work works so pulling together useful data is difficult and the result is suspect. The report gets written. It sounds fine. Nobody is sure how much of it is accurate.

This is not a failure of commitment. The tracking and measuring is secondary to implementation. Almost everyone in this situation cares deeply about their work and wants to be accountable for it, but not at the expense of producing more impact. But there is the rub. How do you know if you’re producing more impact if the monitoring, evaluation and learning system — if there is one — was built for someone else? Does it answer questions the funder asked rather than what the team is actually trying to answer? Perhaps it requires more maintenance than anyone has capacity for or it produces information too late to use. Is it created for a team with full time MEL staff but there isn’t enough funding to pay for that?

The result is a system that creates reporting burden without creating learning. That is the opposite of what monitoring and evaluation is supposed to do.

What a useful MEL system actually does

A MEL system that works for a small nonprofit does three things.

First, it answers questions your team is already arguing about. Not hypothetical questions, not questions a consultant decided were important, not questions lifted from a logic model template. The questions that actually come up in staff meetings. Are we reaching the people we said we would reach? Is this program component worth the time it takes? Are things getting better for the people we serve, or are we just busy?

Second, it is light enough for someone to maintain without a dedicated data person. If your MEL system requires a database administrator, a data analyst, or more than a few hours a week of staff time, it will not survive contact with reality in a small organization. They don’t have that capacity, and building a system that assumes they do guarantees it will be abandoned.

Thirdly, it produces information when decisions need to be made, not just when it’s report-writing time. A report that arrives six months after a program ends cannot improve that program. Data that lives in a spreadsheet nobody knows how to read cannot inform a staffing decision. Useful MEL is timely, accessible, and connected to the moments when it can actually change something.

Having an external person come in and build a dashboard doesn’t usually solve the problem.

The difference between outputs and outcomes

This is the distinction that matters most, and it is the one most small nonprofits get wrong — not because they do not understand it, but because outputs are much easier to count.

Outputs are what you do. Number of people served. Sessions held. meals distributed. Hours of training delivered. These are real and worth tracking. They tell you whether your program is operating as designed. They are necessary for funder reporting. They are not, by themselves, evidence that anything changed.

Outcomes are what changes as a result of what you do. A parent who attended twelve parenting workshops is an output. A parent who reports feeling more confident and less overwhelmed three months later is an outcome. A child placed in stable housing is an output. That child still in stable housing a year later, with improved school attendance, is an outcome.

The difference sounds obvious when stated plainly. In practice, the pressure to count things that are easy to count pushes organizations toward output-heavy systems. Outputs are available immediately. Outcomes require follow-up, often long after the program interaction has ended. Outcomes require asking people how they are doing, which takes time and trust and sometimes feels intrusive.

But a MEL system built entirely on outputs cannot answer the question funders and boards and communities actually want answered: is this working? It can tell you how much you did. It cannot tell you whether it mattered.

A useful MEL system tracks both, with a deliberate ratio. For most small nonprofits, a handful of outcome indicators — three to five — anchored to your core theory of change, combined with the output tracking you are already doing, is enough. You do not need twenty outcome measures. You need a small number of good ones, collected consistently, over enough time to see whether anything is moving.

What to track and what to leave out

The discipline of a good MEL system is not addition. It is subtraction.

Every indicator you add is a commitment to collect that data, clean it, store it, and use it. Most organizations add indicators because a funder asks for them, or because they seem important in the abstract, or because a consultant recommended them. Very few organizations ask the harder question: will we actually use this?

A reasonable target for a small nonprofit is no more than 20 indicators total, across all programs. Fewer is better. If no one explain in one sentence why you are tracking something and what decision it would inform, it probably does not belong in your system.

The other discipline is alignment. Your indicators should connect directly to your theory of change. Not the theory of change you wrote for a grant proposal, but the actual logic of why you believe your program produces the results it does. If your theory of change says that building relationships between youth and trusted adults leads to better decision-making, your outcome indicators should measure something about those relationships and something about decision-making.

Tools that are actually usable

For most small nonprofits, the right tool is the simplest one that does the job.

Google Sheets works for organizations with modest data volume and limited technical capacity. It is free, familiar, and accessible to everyone on your team. Its limitations are real — it is not a database, it does not handle relational data well, and it breaks down at scale — but for a small program tracking a few hundred participants across a handful of indicators, it is often enough.

Airtable sits one step up in sophistication. It handles relational data, supports multiple views of the same information, and has a low enough learning curve that non-technical staff can use it after minimal orientation. It is worth considering when your data has more complexity — multiple programs, multiple sites, data that needs to connect across tables.

Neither tool requires a data team. Both require someone with enough ownership of the system to keep it maintained and updated. That person does not need technical expertise. They need time, clarity about what the system is supposed to do, and organizational support to actually use it.

A reasonable starting point

Do not build your MEL system from a template. Do not start with a logic model and work backward to indicators. Start with questions.

Ask your program staff: what do you argue about? What do you wish you knew? What would help you make better decisions next month? Write those questions down. Then ask: what data would actually answer them?

Build your system around three questions. Collect that data for six months. See whether it gets used. Expand only when you have evidence the system is working — meaning people are looking at the data and it is changing how they think.

A good MEL system is not impressive. It does not have dashboards nobody reads or indicator libraries that run to forty rows. It is used. Someone looks at it before a program decision. Someone brings it to a board meeting and it changes the conversation. Someone says: we thought this was working, but the data says otherwise, so we changed it.

That is the whole goal.

Leave a comment

Share

Anthralytic is a strategy and evaluation studio helping mission-driven teams clarify and amplify their impact. If someone in your network makes decisions about resources in the social sector, this newsletter is for them.

What strikes me, though, is how lopsided that adoption is. Some organizations have gone all in: AI embedded in operations, staff trained, workflows rebuilt around it. Others are still trying to figure out where to start, or have tried a few tools and can’t quite make them work. Most are somewhere in the middle: using it inconsistently, without much of a shared understanding of what it is or what it is for. That gap is widening, and it is worth paying attention to.

But early does not mean untouched. Your organization is probably already encountering AI, even if you have not made any deliberate decisions about it. Maybe someone on your team has been drafting with ChatGPT on their own. Maybe your email platform is ranking messages with it. Maybe your browser is serving you AI-generated answers to your search queries. Maybe a funder portal you log into every quarter has it built into the interface.

The question was never really whether to use AI. The question is whether you are using it intentionally.

Here is what I can offer: a plain-language account of what AI actually is, what it is not, and how to use it more deliberately, in a way that serves your mission without creating new problems in the process.

A quick note on terminology

These terms get used interchangeably, and they shouldn’t. Machine learning is the broader category: algorithms that find patterns in data and improve over time. It has been embedded in software nonprofits use for years: the algorithm that ranks your email, the tool that flags duplicate donor records, the platform that recommends content. That is one conversation about AI, and it is worth having separately.

Generative AI is a more recent layer: systems that produce content (text, images, audio) in response to prompts. Large language models, or LLMs, are the specific technology behind the tools most people are now talking about: ChatGPT, Claude, Gemini. They were trained on enormous amounts of text and generate responses by predicting what words should come next, in sequence, based on patterns in that training data.

When people in the social sector talk about AI right now, they usually mean generative AI, and specifically LLMs. That is what this post is about.

What AI actually is

Here is the short version of how it works: the model was trained on an enormous amount of text. It learned patterns in that text: which words tend to follow which other words, in which contexts. When you ask it a question or give it a task, it generates a response by predicting what words should come next, based on everything it has seen.

That is most of it.

It is not thinking. It is not reasoning in the way you reason. It is very sophisticated pattern matching, operating at a scale that can feel like understanding.

Sometimes it is genuinely useful. Sometimes it produces confident nonsense. Knowing which is which is your job, not the tool’s.

What it is not

It does not know your community. It does not know your history, your relationships, your context, or what your participants actually need. It has never been to your city. It cannot read the room.

It is not neutral. It was trained on text that exists on the internet, which reflects existing power structures, dominant voices, and historical exclusions. It will reproduce those patterns unless you actively push back.

It makes things up. This is not a bug that will eventually be fixed. It is a feature of how the technology works. The model generates plausible-sounding text. Plausible and accurate are not the same thing. It will cite sources that do not exist. It will state incorrect statistics with complete confidence. Every output needs a human review.

It is not faster by default. Drafting with AI, then reviewing, correcting, and rewriting to make it sound like you, often takes about as long as writing a decent first draft yourself. The time savings are real in some situations and illusory in others.

Where it can actually help

Drafting. Grant narratives, donor emails, job postings, board updates. Feed it context about your program, your population, and what you are trying to say, and it will get words on the page. A program director I know used it to turn her messy bullet points into a first draft of a foundation narrative in twenty minutes. She spent another hour making it sound like her organization. That is still faster than starting from scratch.

Summarizing. Long reports, meeting transcripts, policy documents. Paste in the text, ask for a summary, check it against the original. If your team produces a lot of documentation that nobody has time to read, this is where AI earns its keep. A thirty-page evaluation report becomes a two-page brief. A two-hour meeting transcript becomes a list of decisions and next steps.

Generating first-pass questions. Interview guides, survey drafts, focus group prompts. Tell it who you are talking to and what you are trying to learn, and it will give you a starting set of questions in minutes. You will still need to adapt them for your specific community and context, but you are editing rather than staring at a blank document.

Brainstorming. Program names, communications angles, ways to frame a problem. Treat the output as raw material, not finished thinking. It is useful for getting unstuck, less useful for getting to something genuinely yours.

One capability that tends to surprise people: AI can do things that are simply not feasible for humans at scale. The clearest example I have seen is qualitative analysis of open-ended survey responses. Hundreds of respondents, open-ended questions. A human team would need weeks to code and synthesize that data, and most small nonprofits would never attempt it. AI can surface themes, patterns, and outliers in that dataset in a fraction of the time, with a human reviewing and interpreting the output. That is not AI replacing judgment. That is AI making a certain kind of analysis possible that was not practical before.

Tools worth knowing about

Claude (Anthropic), ChatGPT (OpenAI), and Gemini (Google) are the frontier models, the underlying systems that most other AI tools are built on. They all have free tiers that are sufficient for most of the tasks above. If your team already lives in Google Workspace, Gemini is built into Docs and Gmail and has a low-friction on-ramp. Airtable has AI features built into its interface that can help with data organization and summarization if you are already using it for program tracking.

A lot of the AI products marketed to nonprofits (writing assistants, grant tools, donor engagement platforms) are wrappers around these frontier models. They add a specialized interface, prompts tuned for specific tasks, and sometimes workflow integrations. That can be genuinely useful. But it is worth knowing that the underlying model is usually one of these three. When you are evaluating whether to pay for a specialized tool, part of the question is whether the wrapper adds enough value over using the frontier model directly.

You do not need to pay for a premium tier to get started. Start with the free version of a frontier model and see whether it actually fits into your workflow before spending money on a specialized product.

What to protect

Before you put anything into an AI tool, know what tier you are on and what the terms actually say.

This is not just a free-account problem. Free accounts on most major platforms default to using your inputs to improve their models, but so do many paid individual plans. ChatGPT Plus and Claude Pro, for example, carry the same default data policies as their free tiers unless you actively opt out in your account settings. Opting out is usually available and does not affect your experience, but you have to do it deliberately.

The real protection comes at the business or enterprise tier, where providers offer contractual commitments about how your data is handled, no training on your inputs by default, and in some cases the ability to enter into a business associate agreement for sensitive data. If your organization is working with client information, health data, or anything that carries legal or ethical obligations around privacy, that is the tier worth looking at — and the cost is worth comparing against the risk.

The question to ask before using any tool with sensitive data is not “is this AI?” It is “what does our agreement with this vendor actually say, and does it meet our obligations to the people we serve?”

That is a governance question, not a technology question. And it is one your organization should answer deliberately, not by default.

Your judgment does not get delegated

The tool produces output. You decide what to do with it. If something the tool generates feels off, it probably is. Trust that.

AI can help you work faster on certain tasks. It cannot tell you what your community needs, what your values require, or whether a decision is the right one. That knowledge lives with you and the people you work with. No tool has access to it.

Using it more intentionally

You do not have to have a position on AI to use it carefully. You do not have to be an early adopter or a skeptic. Being early in adoption is actually an advantage. You have room to make deliberate decisions before habits form and tools accumulate.

If your use of AI right now is ad hoc (someone drafts with it occasionally, someone else avoids it, no one has talked about the data question) that is worth changing. Not because AI requires a grand strategy, but because mission-driven organizations have specific obligations: to the people they serve, to the data they hold, and to the values that brought them to this work in the first place.

Start with one conversation. What are we using, who is using it, and what are we putting into it? Then try one low-stakes task deliberately. Notice where it helps and where it misses.

That is enough for now.

Anthralytic is a strategy and evaluation studio helping mission-driven teams clarify and amplify their impact. If someone in your network makes decisions about resources in the social sector, this newsletter is for them.

Share

Leave a comment

I think I know what it is. And I think it matters for a lot more than art.

What We Are Actually Responding To

The usual argument goes like this: AI art is not real art because a human did not make it. But a human did prompt it, select it, refine it. So the counter-argument goes: a human was involved, just differently. And the debate spirals from there, into questions about originality, intent, and craft.

But I don’t think the objection is really about whether a human was present. A human can be present and still feel nothing. A painting made cynically, a song written to manipulate, a novel produced to fill a market gap. Humans made those. They can still feel empty.

What people are responding to, whether they name it or not, is the absence of relationship. Art at its best is not a product delivered from maker to consumer. It is a relational act. Someone struggled to make sense of their experience, shaped it into form, and offered it to someone else. The viewer or listener or reader receives it and is changed, not just informed but moved, challenged, seen. The meaning is not in the object. It is in the exchange. It lives between people.

AI breaks that exchange. Not because it is artificial, but because there is no relationship in it. Nothing was risked. Nobody was changed in the making. There is no reciprocity. You receive an image, but nobody offered it to you. That is what feels hollow.

Evaluation Is the Same

I have been an evaluation practitioner for a long time, and I have slowly come to believe that the same thing is true about evaluation.

We talk about evaluation as if the value is in the product: the report, the findings, the recommendations, the data visualizations. We measure the success of an evaluation by whether the report was delivered on time, whether the methodology was sound, whether the findings were actionable.

But the most valuable evaluations I have been part of were not the ones with the best reports. They were the ones where something happened between people. A program officer and a community organizer looking at the same evidence and realizing they understood the problem differently. A team sitting with findings that contradicted what they expected and being willing to say so out loud. A funder hearing directly from the people a program serves and having their assumptions rearranged.

The value was not in the data. It was in the relationship that made the data meaningful. It was in the willingness to be changed by what someone else showed you.

Nicole Bowman’s work in Indigenous evaluation names this more clearly than most Western evaluation theory does. In her framework, evaluation is not something done to communities or even for them. It is done with them, as relatives. Knowledge belongs to the community. The evaluator’s role is relational, not extractive. The purpose of inquiry is not just to know but to strengthen the bonds between people in the process of knowing.

That framing has changed how I think about all evaluation, not just Indigenous evaluation. If the value of evaluation is relational, then anything that weakens the relational core weakens the evaluation itself. It does not matter how rigorous the methods are. If nobody was in relationship during the process, the findings will sit on a shelf.

The Hollow Version

You do not need AI to produce the hollow version of evaluation. Organizations have been doing it for decades. Reports that nobody reads. Data dashboards that nobody discusses. Surveys collected from communities who never see the results. Findings delivered upward to funders while the people who shared their stories are left with nothing.

That is evaluation without relationship. The product exists, but the relational exchange never happened. Nobody was changed. Nobody was seen.

AI makes the hollow version faster. You can use AI to generate a survey, analyze the responses, summarize the findings, and produce a report without anyone sitting together to make sense of what it means. The output will be competent. It might even be accurate. But if the meaning was never made between people, you have not done evaluation. You have done data processing with a nice cover page.

The risk is not that AI will replace evaluators. The risk is that it will make it easier to skip the part that matters.

What Relationship Looks Like in Practice

The parts of evaluation that are relational are also the parts that produce the most change.

Sitting with a community partner and hearing something that surprises you. Presenting preliminary findings to a team and watching the room get quiet because everyone has to rethink something. Disagreeing about what the evidence means and staying in the conversation long enough to learn from the disagreement. Going back to the people who gave you their stories and asking whether your interpretation gets it right.

These moments require trust. They require presence. They require a willingness to be changed by the encounter. None of that can be automated. None of it should be.

Where AI Belongs

This is not an argument against AI in evaluation. It is an argument for knowing which parts are relational and protecting them.

AI is good at the parts that do not require relationship. Processing large datasets. Identifying patterns across documents. Drafting interview protocols. Cleaning and organizing qualitative data. Summarizing long reports. Translating materials. These tasks take time, and time is the scarcest resource in the social sector.

If AI handles the mechanical work, it can free people up for the relational work. More time in the room together. More time making sense of what the data means. More time going back to communities with findings and asking what they think.

But that trade only works if you actually use the time you save for relationship. If you use it to do more evaluations faster with fewer people in the room, you have just automated the hollow version.

How Anthralytic Approaches This

This is the principle behind how I build tools at Anthralytic. Technology as scaffolding for human connection, not a substitute for it. Every tool I design starts with a question: does this bring people closer to understanding each other, or does it let them skip the conversation?

People cannot do this work alone in front of their screens. Evaluation, like art, is relational. The meaning is made between people, in the exchange, in the reciprocity, in the willingness to be changed by what you hear. Take that out and you might still have data. But you do not have understanding.

And understanding, in the end, is the whole point.

Share

Leave a comment

Subscribe now

Anthralytic is a strategy and evaluation studio that helps mission-driven teams clarify and amplify their impact. If you make decisions about resources in the social sector, whether you call yourself an evaluator or not, this newsletter is for you.

The irony is not lost on me.

Marilyn Waring named the problem in 1988. In If Women Counted, she laid out the argument clearly: GDP was not a neutral accounting tool. It was a political choice about what matters. And that choice rendered invisible most of the work that keeps human life running: caregiving, subsistence farming, raising children, tending the sick, holding communities together. Work done overwhelmingly by women. Work that, by the official measure of national economic health, did not exist.

Waring was a New Zealand parliamentarian who had the audacity to read the UN System of National Accounts cover to cover and ask what it was actually counting. The answer: market transactions. If money changed hands, it counted. If it didn’t, it didn’t. A woman who cooks three meals a day for her family contributes nothing to GDP. If she has a breakdown and the family hires a cook, GDP goes up. This is not a bug. It is the architecture.

The accounting has not fundamentally changed since 1988. We have added satellite accounts and time-use surveys that gesture toward unpaid work. But the core logic holds: what gets measured gets valued, and what gets valued shapes what gets resourced, protected, and seen.

Some governments have tried to ask a different prior question. Bhutan’s Gross National Happiness framework, proposed in the 1970s, was not a rejection of economic development but a challenge to its premise: development toward what? GNH includes nine domains: living standards, yes, but also psychological wellbeing, time use, community vitality, cultural resilience, ecological health. Time use. That is where women’s labor begins to appear. GNH is imperfect and has been used politically in ways its originators did not intend. But the conceptual move matters: what a society measures reflects what it believes a good life looks like. And a good life, it turns out, requires a lot of things GDP cannot see.

I have been sitting lately with a related concept: mattering. Not achievement. Not productivity. Not impact in the social sector sense of the word. Mattering is simpler and harder than any of those. It is the experience of being noticed, being needed, having your absence register. It is the felt sense that you count, to someone, to something, to the fabric of a community. Research on community well-being suggests that mattering is not just emotionally significant. It is structurally significant. Communities where people feel they matter show up differently on almost every indicator we care about: health, resilience, civic participation, the ability to recover from disruption.

And yet mattering is almost never what we measure. We measure outputs. We measure reach. We measure satisfaction scores and pre-post surveys. We do not measure whether the grandmother in the corner of the room feels that anyone would notice if she stopped coming. We do not measure whether the volunteer coordinator’s judgment is trusted or just her hours. We do not ask who is holding the relational infrastructure and whether they know that we know.

This is an evaluation problem. It is also a Women’s History Month problem, an International Women’s Day problem, a looking-at-my-mother-and-my-daughter-across-a-farm-table problem. The work women have always done, the relational labor, the care work, the community maintenance, is precisely what our measurement systems were built not to see. Not because it is invisible. Because counting it would require admitting it matters. And admitting it matters would change what we owe.

If we measured mattering, we would have to resource it. If we counted care, we would have to compensate it. If GNH replaced GDP as our north star, we would have to confront that the economy we have built is not optimizing for human flourishing. It is optimizing for transactions. Waring knew this in 1988. Women doing unpaid work knew it long before she wrote it down.

What would it look like to measure what actually matters? Not a complete overhaul of every evaluation framework. Something more practical: adding questions about relational contribution to program assessments. Asking who holds the informal knowledge in an organization and whether that knowledge is protected. Designing time-use components into community needs assessments. Treating mattering as a measurable condition of community health, not a soft feeling that sits outside the data. We have the tools. We mostly lack the will to use them, because using them changes the answer to the question of whose work counts.

My mother spent decades cleaning other people’s houses. That work moves through the economy, but through what economists call the shadow economy, the informal sector where transactions are real and labor is hard and protections are scarce. No benefits. No unemployment insurance. No social security credits that reflect what she actually earned. The money changed hands, but the system was designed not to see her. And then there was raising three children, much of it on her own. That part didn’t register anywhere at all. My daughter is watching both of us, learning what counts. I want her to inherit better accounting than we did. Not just more indicators, but a different prior question. What is this for? Who does this serve? Whose work do we see?

The farm outside the window does not care about GDP. The morning does not care about outputs. And the three women at this table have always known that the most important things are the hardest to count.

That is not a reason to stop trying.

Share

Subscribe now

Leave a comment

Anthralytic is a strategy and evaluation studio helping mission-driven teams clarify and amplify their impact. If you make decisions about resources in the social sector — whether you call yourself an evaluator or not — this newsletter is for you.