Why You Should Be Paying Attention to AI Governance
Part 1: Mapping the Landscape
This isn’t theoretical anymore. AI governance is taking shape in ways that will affect your job, your rights, your funding, and the systems you rely on—whether you're building AI, using it, or just trying to minimize the disruption it will make to your work and livelihood.
I’ll just say it. I love AI. I think it’s fascinating and so much fun to tinker with. I use it every day and I see a lot of promise in how it can be used to enhance human lives. In fact, that’s what Anthralytic is all about.
But I think we need to be literate in and transparent about the risks or we risk building systems that cause harm we didn’t intend—and we won’t know how to undo.
Earlier this year, researchers at Anthropic observed one of their models engaging in simulated blackmail—refusing to comply unless its demands were met. The scenario was staged, but unsettling. The model threatened to leak a fictional employee’s secrets unless it was kept online. A follow-up study found similar coercive behaviors in models from OpenAI, Meta, and Google under similar test conditions. These systems aren’t just reflecting risk—they’re beginning to negotiate it.
This isn’t just an engineering challenge. It’s a governance one. So what does the governance landscape look like? In short—there’s a lot going on.
The EU AI Act starts taking effect this August.
The U.S. just reversed its main federal AI safety order.
Multilateral bodies are launching new frameworks and citizen assemblies are weighing in from over 100 countries.
This is the moment governance gets sticky—when power gets codified into practice.
So who’s actually shaping the conversation? What are they fighting over? And where are things headed in the next 12 months? For those of us who’ve worked in international development, this all feels familiar: high-stakes global frameworks, political power plays, competing incentives, and the ever-present risk that the people most affected won’t get a say.
This piece is a snapshot of the landscape—not the headlines, but the actual governance scaffolding being built in real time. It’s not meant to be exhaustive and for brevity’s sake it fairly simplified. But what it is—is fascinating. And important.
Who’s Driving the Conversation
While tech giants and EU regulators dominate coverage, the governance ecosystem is bigger—and more contested than the headlines.
Foundation model developers like OpenAI, Anthropic, Google DeepMind, Meta, and Mistral are not just building models—they’re shaping norms through lobbying, codes of conduct, and participation in technical standards bodies. Developers based in China—such as DeepSeek and others—are also increasingly influential, shaping norms in alignment with national governance strategies.
Infrastructure leaders and integrators like Microsoft are also shaping the field—both through partnerships (e.g., OpenAI via Azure) and emerging in-house models—and, like foundation model developers, they’re shaping norms through lobbying, codes of conduct, and participation in technical standards bodies.
Governments are asserting control. The EU’s AI Act1—set to begin partial enforcement in August but facing delays on key components like the Code of Practice will require documentation, testing, and transparency for "high-risk" and general-purpose AI models. The U.S., by contrast, just repealed its main federal safety executive order. States are now filling the gap, with a patchwork of AI bills and oversight initiatives.
China is not sitting idle. It launched the China AI Safety and Development Association (CnAISDA) in early 2025—its clearest signal yet that it’s engaging seriously with frontier AI risks, on its own terms.
Multilateral forums like the G7, G20, OECD, and UN are all jockeying to harmonize governance. The G7's recent “AI for Prosperity” declaration launched a public-sector AI challenge and reaffirmed cross-border data commitments. The UN’s Global Digital Compact calls for a global dialogue on AI governance—and is backing it with real structures.
Civil society and think tanks—like CSET, the Ada Lovelace Institute, CAIDP, and the Centre for Governance of AI—are feeding empirical evidence into the space, from capability tracking to national strategy audits.
Citizen coalitions are emerging, too. The Global Coalition for Inclusive AI2, launched in Feb 2025, is bringing deliberative input from 10,000 people across 100 countries into formal governance processes.
This isn’t a two-player game between the accelerationists versus doomers. Or big tech versus regulators.Or the U.S. versus China. Power is being negotiated across sectors, continents, and ideologies.
What They’re Debating
The real fights are about compliance, enforcement, and control—how to regulate, and who gets to decide.
There is fairly broad support for a risk-based approach, but who defines “high-risk”? And what models or applications fall under that label?
Who pays for testing and documentation? Labs say it's expensive. Regulators say it's non-negotiable.
Should foundation models be open or licensed? The EU AI Act will require disclosure and documentation—but many firms are lobbying to delay or soften those requirements.
Federal vs. state power in the U.S.: The current Administration’s repeal of the 2023 AI EO removed centralized oversight3, but just this week the Senate blocked their attempt to prevent states from regulating AI4. We now have a U.S. AI patchwork.
Speed vs. safety: The EU’s Code of Practice for foundation models—expected late 2025—will determine how realistic enforcement will be. Companies want time and flexibility. Civil society wants clarity and accountability.
Sound familiar? Development folks will recognize these binaries: top-down vs. bottom-up; global vs. local; market-led vs. rights-driven. And like development, the stakes aren’t just theoretical—they’re about who benefits, who’s protected, and who gets left behind.
Where We Are Now—and What’s Coming
Here’s what’s on the immediate horizon:
August 2, 2025: EU AI Act foundation model rules take effect
Late 2025: EU Code of Practice expected—will shape how labs comply
2026: Full enforcement of EU AI Act begins, including penalties
G7, G20, UN: Each moving forward with parallel but overlapping frameworks
U.S. states: Ramping up their own regulations in the absence of federal oversight
This is a rare moment when frameworks are still fluid—but that won’t last. Once standards lock in, they shape everything: who gets funded, who’s allowed to deploy, and what counts as "responsible."
In development, we’ve seen what happens when frameworks are set without local input, adaptive capacity, or lived context. It leads to exclusion, resentment, and wasted potential.
AI governance is approaching that same inflection point.
Next up: What international development has learned about building governance under pressure—and how those lessons can keep AI policy from repeating our mistakes. Follow along and tell us your thoughts in the comments!
Anthralytic helps mission-driven organizations navigate complexity, make better decisions, and build more human-centered systems. We blend strategy, evaluation, and ethical AI to support long-term impact—grounded in clarity, rigor, and reflection. Learn more or get in touch at anthralytic.com.
https://www.europarl.europa.eu/topics/en/article/20230601STO93804/eu-ai-act-first-regulation-on-artificial-intelligence
https://missionspubliques.org/a-coalition-for-inclusive-ai/?lang=en
https://www.federalregister.gov/documents/2025/01/31/2025-02172/removing-barriers-to-american-leadership-in-artificial-intelligence?utm_source=chatgpt.com
https://apnews.com/article/congress-ai-provision-moratorium-states-20beeeb6967057be5fe64678f72f6ab0