This is the third installment of Where We Stand: Position, Proximity, and Power in AI Safety.
The first piece in this series considered the user, the person directing the system. The second considered the subject, the person a system observes, classifies, or creates knowledge about. This piece turns to the source, the person whose writing, images, data, judgment, experience, or ideas become part of what the system can do.
When you’re the subject, the system creates knowledge about you. When you’re the source, it creates value from something you contributed. Often, you are both. The participant in a focus group may become the subject of an analysis while also supplying the stories that make the analysis possible. Someone using a chatbot may be the user during the conversation, then become a source if that interaction is retained and used to improve the product.
The category is broad because AI depends on many kinds of human contribution. A novelist, a programmer, an employee uploading organizational documents, a researcher publishing findings, and a community sharing its experience do not have the same rights or occupy the same position. Still, they share one important feature: the system becomes more useful because of something they supplied.
AI does not learn from nowhere, even when the people it learned from have disappeared from view.
When sharing only moves one way
I have been circling this problem for a while through my writing on data sovereignty. In AI Governance Has a Thanksgiving Problem, I used the familiar Thanksgiving story to think about the way extraction gets described as collaboration. The story presents a shared table and mutual exchange while obscuring what was taken, who benefited, and what happened afterward.
AI governance has developed its own version of that story. We say a model learned from the internet, data was publicly available, users helped improve the product, or communities contributed their knowledge. The language makes the relationship sound mutual, but following the value tells a different story. The company gains a more capable model, the organization produces a report, the evaluator gets paid, and the user receives an answer. The people whose knowledge made those things possible may receive very little, and they may not even know they participated.
As I wrote in that earlier piece, “The table was built from what was taken, and the feast is made of it.”
That does not make every use of existing knowledge an act of theft. People have always learned from one another, adapted methods, cited previous work, and combined old ideas into new ones. Shared knowledge is a public good, and AI can help us find, translate, connect, and apply it in ways that are genuinely useful. I use AI for exactly those reasons.
The problem begins when we call something sharing without asking whether the relationship runs in both directions.
In Whose Data Is It, Anyway?, I described how information usually moves through the social sector. Communities supply experiences, outcomes, and stories. Organizations collect them, evaluators turn them into findings, and funders receive the resulting reports. Data and value travel upward, while very little returns to the people who supplied the original knowledge.
Nobody has to set out to exploit anyone for this to happen. The consent forms can be signed, the data can be de-identified, and the servers can be secure. Everyone involved can follow the approved process with reasonable care, while the people at the beginning of the process retain no authority over what their contribution becomes. Extraction does not always require a villain. Sometimes it only requires a system whose defaults point in one direction.
Sovereignty is about authority
This is where data sovereignty changes the question.
Privacy asks whether information is protected. Security asks who can enter the system. Access asks whether someone can retrieve the data. Sovereignty asks who has the authority to decide what happens to it.
The First Nations principles of OCAP® name ownership, control, access, and possession as connected forms of authority over First Nations data. OCAP® is not a generic framework that can simply be lifted out of its political and historical context. It applies specifically to First Nations, whose sovereignty is a legal and political reality; not a metaphor for feeling respected.
The CARE Principles for Indigenous Data Governance similarly ask whether data practices produce collective benefit, recognize authority to control, fulfill responsibilities to the people represented, and meet ethical obligations throughout the life of the data. These principles emerged from particular histories in which Indigenous peoples were intensely studied while receiving little benefit and retaining little authority over the knowledge extracted from them.
A writer, employee, or chatbot user does not occupy the same position as an Indigenous nation, and I do not want to flatten those differences. What these frameworks expose, however, is the weakness of a model built entirely around individual consent and technical protection. A person can sign a form and still have no meaningful ability to object later. A community can receive access to a dataset without receiving anything it can use. Information can be protected inside a system that the people who supplied it never wanted it to enter.
In AI Governance From Below, I argued that community data should not become the unrestricted property of an organization simply because the organization collected it. AI makes that principle more important because it expands the number of possible future uses. An interview collected for an evaluation can become input to an organizational chatbot. A set of case notes can become training material. Community language developed for one report can reappear in grant proposals, strategy documents, or products that no one imagined when the information was first provided.
The usual question is whether the organization obtained the data legally. The sovereignty question goes further by asking who retains authority over what the data may become.
The sources inside my own work
I am encountering a smaller version of this problem while building the Conditions Web, an AI-guided conversational process that helps social impact organizations examine the conditions surrounding their work before developing a strategy or evaluation. Rather than beginning with a linear logic model, it maps historical, social, organizational, political, economic, cultural, and place-based conditions as an interconnected web.
I designed the particular structure, decided how its domains relate to one another, and developed the process through which a conversation becomes something an organization can use. I believe the synthesis itself has value, but I did not invent all the thinking inside it. The Conditions Web draws from ideas that have evolved across evaluation and systems practice for decades, including developmental evaluation, contribution analysis, participatory evaluation, systems thinking, empowerment evaluation, the Zen business model, and work on power, context, complexity, and unintended consequences.
Some of those ideas are associated with particular scholars and practitioners, while others grew through communities of practice and do not belong neatly to one person. Either way, the framework has an intellectual lineage, and the people within that lineage are sources.
I plan to publish the people behind the framework alongside the framework itself. It will work something like a bibliography, but I want it to do more than list publications. I want to explain what each person or body of work contributed, where someone can find the original thinking, and how I have borrowed, combined, modified, or departed from it.
The language used to describe those relationships matters. Created by, adapted from, influenced by, and consistent with are not different ways of saying the same thing. They tell the reader how close the relationship is and where my contribution begins.
Making that lineage visible is partly about giving credit, but it is also about making the Conditions Web easier to question. Someone who can trace an idea back to its source can read the fuller argument, examine my interpretation, and decide whether I used it well. A visible lineage makes a framework less magical and more accountable because it shows that the authority inside the tool came from somewhere.
It also makes clear that the AI did not invent the framework. AI may guide a conversation or help synthesize what a user says, but the ideas organizing that process came from people who spent years working through real problems. I am not outside the source question simply because I recognize it. I am building something that creates value by combining other people’s thinking, which means I have obligations to those people as well as to the eventual users.
The questions changed the tool
Writing down these obligations forced me to turn them back on the Conditions Web. The intellectual bibliography I had planned still mattered, but it only addressed the people whose ideas shaped the framework. It did not address the people and communities who will contribute knowledge while using it.
Those are different source relationships, and they create different obligations. A scholar whose published work influenced the methodology should be accurately credited, with permission sought when I reproduce protected language, diagrams, instruments, or proprietary methods. A participant describing the conditions in their community needs something more. They need to know how their contribution will be processed, retain some authority over its use, see what the system made from it, and receive something useful in return.
I have now written that distinction into the Conditions Web’s intellectual lineage and methodology. Before the tool invites direct participation, it must explain whether contributions reach a third-party AI provider, what is retained, and whether Anthralytic or the provider uses submissions for product improvement or model training. Any use for improvement or training must require separate consent rather than being bundled into ordinary use.
Participants must also be able to choose how they are identified and place limits on how a particular contribution may be analyzed, quoted, shared, or reused. Their original account should remain connected to anything Q synthesizes from it, and the system should make clear when Q has drawn an inference that nobody actually stated. Participants should be able to correct or withdraw what they contributed, with that decision carried through the conditions, relationships, summaries, and exports derived from it.
Most importantly, participation cannot remain another one-way data collection exercise. People who contribute knowledge should have an opportunity to review how they were represented and receive the result in a form they can actually use. That might be a readable summary, a relevant portion of the web, or another return agreed upon in advance. The organization commissioning the work should not be the only party that benefits.
The same principle changes how organizational documents enter the system. Conditions Web will ask whether the person uploading a document has permission to use it in an AI-assisted analysis. Possessing a file does not necessarily create authority to reuse everything inside it, especially when it contains material written by employees, partners, clients, or community members for another purpose.
I also added these obligations to the product’s evaluation rubric. Disclosure, contribution-level permissions, correction, withdrawal, participant validation, reciprocal return, upload authority, and the distinction between citation and permission are now release checks. If the product cannot meet them, that part of the experience should not be treated as ready.
This does not mean the problem is solved. Writing a requirement is easier than building it, and building a control is easier than knowing whether it works for the people it is intended to protect. The next step is implementation and testing with people whose knowledge the Conditions Web will ask them to share.
Still, the questions have already changed what I am building. They led me to a standard that is stronger than simply saying users own their data:
Anyone whose knowledge enters the Conditions Web should be able to understand how it will be used, see how it was represented, challenge what the system made from it, and receive something useful in return.
That is what sovereignty begins to look like inside a product. It is not only ownership language in a privacy policy. It is authority made visible through the design.
AI can help us build from what people know, and that is part of its value. The test is whether doing so requires making those people disappear. When you are the source, safety means retaining some authority over what your contribution becomes, receiving something meaningful when value is created from it, and remaining visible inside the systems you helped make useful.
Anthralytic helps mission-driven organizations use strategy, evaluation, data, and AI to understand their impact and make better decisions.
OCAP® is a registered trademark of the First Nations Information Governance Centre.