A school district pilots an AI tutor. A city uses a model to decide where inspectors should go. A public agency introduces a chatbot to help people navigate services. Each decision begins inside an institution, often as a purchase, a pilot, or an attempt to make work more efficient. But the effects don’t necessarily stay there.
Most people will never choose, configure, or directly interact with these systems. Their agency comes from the structures surrounding them: professional standards, public records, oversight, independent evidence, appeals, and carefully defined opportunities to participate.
This is what makes the public position different. Public agency is mostly institutional. The challenge is building institutions capable of matching the strength of their governance to the scale of AI’s consequences.
This is also the final position in this series. I’ve written about being the user, the subject, the source, and the worker. Each position offers at least some opportunity for individual action. The public position asks a different question: What must the institutions acting on our behalf be able to do?
Public agency lives in institutions
We delegate decisions all the time. Elected officials set policy. Public employees administer programs. Teachers choose instructional methods. Doctors exercise professional judgment. Procurement teams negotiate contracts.
That delegation is necessary. A functioning society can’t ask every person to participate in every operational decision. Public agency doesn’t require constant public involvement. It requires institutions that use their delegated authority competently, transparently, and within appropriate limits.
AI complicates that work because a system that appears to be a routine tool can quietly change how an institution sees and acts on the world. It can influence which cases receive attention, which patterns become visible, which risks matter, and which people have to work harder to be understood. It can also create real benefits by helping institutions find information, reduce delays, identify unmet needs, and deliver services more consistently.
The task is to distinguish between a bounded use of technology and a decision that changes public conditions.
That distinction should determine the level of governance.
NIST provides an operating structure
The NIST AI Risk Management Framework is one of the foundations of the AI safety framework I’ve been developing for Anthralytic. I keep returning to it because it’s an ongoing organizational practice rather than a verdict about whether AI is good or bad.
NIST organizes that practice into four functions: Govern, Map, Measure, and Manage.
Govern establishes responsibility and the institution’s approach to risk. Map examines the purpose, setting, people, and possible effects of a particular system. Measure tests performance and makes impacts visible. Manage turns the evidence into decisions.
These functions are connected. An institution can’t measure the right things if it hasn’t understood the context. It can’t manage a risk if nobody has the authority to act on the findings. It can’t govern responsibly if it treats the original purchase as the end of the decision.
This structure also gives public participation a clearer role. Participation isn’t a universal requirement applied in the same way to every system. It’s one way institutions gain knowledge, test their assumptions, represent affected interests, and establish the legitimacy of consequential decisions.
The form and strength of that participation should follow from the risk.
Govern: Set the thresholds before the pressure arrives
Governance begins before a specific vendor or product is under consideration. An institution needs to decide who is accountable for AI, what evidence is required, which uses need additional approval, and how much risk it’s willing to accept.
Without those thresholds, every decision becomes an improvisation. Low-risk tools can become trapped in unnecessary bureaucracy, while consequential systems may move forward because nobody has been assigned the authority to slow them down.
A useful governance policy creates a graduated path. Bounded internal tools can move through an ordinary review. Public-facing tools require testing, transparency, and feedback. Systems with significant consequences require more independent evidence, broader representation, stronger approval, and ongoing oversight.
The exact categories will differ by institution, but the triggers should be established in advance. A hospital, school district, police department, foundation, and workforce agency operate in different environments. Their risk thresholds should reflect the authority they hold, the populations they serve, and the consequences of getting something wrong.
Good governance makes those differences explicit.
Map: Understand the public consequences
NIST’s Map function asks organizations to understand the system’s intended purpose, setting, expected benefits, limitations, and potential effects on individuals, communities, organizations, society, and the planet. Its AI RMF Playbook also encourages collaboration with external groups when they can help identify risks, alternatives, and acceptable boundaries.
This is where an institution determines what kind of decision it’s actually making.
Six characteristics are especially useful: consequence, reach, choice, reversibility, distribution, and uncertainty.
Consequence concerns what the system can change. A tool that helps draft an internal document is different from one that influences eligibility, employment, education, health care, safety, or liberty.
Reach concerns how widely the effects travel. A voluntary tool used by a small team has a different public footprint from a system operating across an entire school district, benefits program, or transportation network.
Choice asks whether people can realistically avoid the system. An opt-out isn’t especially meaningful when declining also means losing access to a job, public service, or essential opportunity.
Reversibility considers what happens after a mistake. Can the error be detected and corrected? Can the original decision be restored? Is there someone with the authority to repair the harm?
Distribution looks at where the benefits and burdens land. A system may save staff time while shifting work to residents. It may improve service overall while producing more costly errors for people with disabilities, limited English, or unusual circumstances.
Uncertainty concerns how much the institution actually knows. A familiar technology with years of evidence calls for a different approach from a novel system whose capabilities, data practices, or performance may change rapidly.
These characteristics aren’t a formula that produces an automatic answer. They help the institution see when a technical decision has acquired a public dimension. As the consequences, reach, lack of choice, difficulty of repair, unequal distribution, and uncertainty increase, so should the strength of the governance surrounding the system.
Measure: Treat experience as evidence
Institutions often evaluate technology through technical performance, cost savings, staff time, or vendor demonstrations. Those measures matter, but they rarely capture the whole public experience.
A chatbot may produce accurate answers in testing while confusing people who don’t already understand the program. A translation tool may perform well in widely spoken languages and poorly in the languages most needed in a particular community. A model may improve average processing time while creating long delays for the cases it can’t easily categorize.
NIST’s Measure function calls for testing in conditions similar to the actual deployment setting. It also includes feedback and appeals from users and impacted communities as part of evaluating system performance.
That’s an important shift. A complaint isn’t only a customer-service issue. An appeal isn’t only an administrative burden. Both can reveal something the institution’s original measures failed to capture.
This is one of the clearest roles for participation. People who experience a system can identify effects that are invisible to its designers. Frontline workers know where the formal process and the real process diverge. Community organizations may recognize barriers or patterns that don’t appear in aggregate data. Domain experts can determine whether a technically impressive result is useful in practice.
Participation improves measurement when it brings necessary knowledge into the institution.
Manage: Make the evidence consequential
The final function is Manage. Once an institution has mapped the context and measured the results, it has to decide what to do.
Sometimes the evidence will support broader adoption. Sometimes it will show that the system works only for a narrower use. The institution may need to change the interface, add human review, restrict access to certain data, improve an appeal process, renegotiate the contract, or conduct another pilot.
It may also need to stop.
The important thing is that the evidence can change the system. If a review can’t affect the scope, safeguards, resources, or decission to continue, then it’s documentation rather than governance.
Contracts matter here because an institution can manage only what it retains the authority to change. It may need access to audit records, notice of system updates, control over data retention, the ability to test vendor claims, and a workable way to export information or end the relationship. Those provisions are part of public governance because they preserve the institution’s ability to respond to what it learns.
Management also requires a schedule. A system should return for review when its purpose changes, new data is added, the vendor updates a model, evidence of harm emerges, or the original assumptions no longer hold. Higher-impact uses may need formal renewal dates so continuation becomes a decision rather than a default.
Participation needs a defined purpose
Public participation can contribute to each part of this process, but institutions should be clear about what people are being asked to influence.
Some participation helps establish context. Residents, workers, or service users may identify needs and barriers before a system is designed.
Some participation shapes implementation. People may influence accessibility, data practices, permitted uses, communication, or the design of an appeal process.
Some supports oversight. Affected groups may help interpret performance data, identify emerging harms, or recommend changes after deployment.
The most consequential uses may also require participation in authorization. That could include an oversight board, elected body, worker representatives, civil rights organizations, or a formal public process with the authority to set limits or reject a proposed use.
Timing determines what participation can accomplish. A public meeting held after a product has been selected may still improve implementation, but it can’t shape the selection itself. Institutions should be honest about which questions remain open and which decisions have already been made.
Public participation is meaningful when it can change the part of the decision it was convened to address.
Trust comes from capable institutions
Public trust doesn’t require consensus. People can review the same information and reach different conclusions about acceptable risk, appropriate spending, or the role of technology in public life.
Trust grows when people can see that an institution understood the decision it was making. There is a responsible owner. The expected benefit is clear. The affected groups and possible tradeoffs have been considered. Claims have been tested in the real setting. People can report problems and challenge outcomes. Someone has the authority to change course.
This is what the public position adds to AI safety. Individual agency still matters, but many of AI’s effects can’t be managed individually. We rely on institutions to recognize public consequences, draw on the right forms of knowledge and participation, and apply governance that is strong enough for the decision in front of them.
That question becomes especially tangible when AI arrives in physical form. A data center may begin as an economic development project or infrastructure agreement, but its public consequences can include land, electricity, water, tax policy, jobs, noise, emissions, and pressure on shared infrastructure. The scale and distribution of those effects make institutional governance especially important. Berkeley Lab’s research shows how quickly the electricity demands associated with data centers are growing.
That’s where I’m going next: the good, the bad, and the ugly of the data centers behind AI, and how AI itself might help reduce their footprint.
Anthralytic helps mission-driven organizations use strategy, evaluation, data, and AI to understand their impact and make better decisions.
