unpublished alpha

the symmetry of existential technologies

When Robert Oppenheimer watched the sky split open in 1945, he reached for scripture: “Now I am become Death, the destroyer of worlds.” If artificial general intelligence arrives in the coming decades, what line will today’s architects reach for? Hopefully something gentler—but the magnitude will be similar. Like nuclear weapons, AGI embodies a rare symmetry: immense creative potential bound to equally profound risk.

This isn’t just a philosophical comparison. The dilemmas AI presents today echo the nuclear age with eerie fidelity—dual-use technologies, murky escalation dynamics, and the pressing question of who, if anyone, holds the authority to intervene. Both domains advance through academic research with strategic implications. And both pose verification challenges that outpace conventional regulatory tools.

The International Atomic Energy Agency (IAEA) emerged from this nuclear governance conundrum as an improbable success story—a rare example of nations voluntarily ceding fragments of sovereignty in exchange for collective security. As artificial intelligence accelerates toward capabilities that may rival or exceed nuclear technology in transformative impact, the IAEA offers not merely an instructive analogy but potentially a blueprint for effective global AI governance.

verification paradoxes and technical legitimacy

Nuclear verification presented what seemed an intractable paradox: effective oversight requires access to sensitive facilities, yet nations resist external scrutiny of technologies intertwined with national security. The IAEA’s ingenious resolution involved creating a technically sophisticated organization whose legitimacy derived from demonstrated expertise rather than political authority. Its inspectors earned the right to verify compliance not through legal mandate alone, but through technical competence that commanded respect from both scientists and statesmen.

AI governance faces a parallel verification paradox with additional complications. You can’t smuggle uranium in a flash drive, but you might be able to do just that with dangerous AI. While enrichment demands centrifuges and industrial plants, cutting-edge models can be trained in someone’s garage—with the right code, compute, and ambition. AI is grounded in mathematical techniques rather than physical apparatus. It may not be governments that surprise us—it might be a small startup, a rogue lab, or an open-source repo gone sideways. The scariest breakthroughs could happen where no one’s looking, or worse, where everyone is but assumes someone else is in charge.

Consider the challenge of verifying AI capability limitations. A nation might publicly commit to restricting its AI systems from autonomous weapons deployment while secretly maintaining algorithms capable of such functions. Unlike centrifuge cascades or plutonium stockpiles, these capabilities leave minimal physical footprint. Even more troubling, techniques like model distillation and transfer learning could allow restricted capabilities to be extracted and reproduced with relative ease.

Yet the IAEA’s approach suggests a potential solution: technical legitimacy as the foundation for verification authority. An international AI governance body could develop sophisticated technical tools for capability assessment—not merely reviewing code repositories but employing specialized AI systems to probe for concealed functionalities or safety vulnerabilities. Such an organization would need to recruit leading AI researchers and safety experts, offering them a platform to advance technical standards while contributing to global security.

the promotion-restriction balance

Perhaps the most underappreciated aspect of the IAEA’s success lies in its dual mandate: promoting peaceful nuclear applications while restricting weapons proliferation. This seeming contradiction actually forms a coherent governance strategy. By actively assisting nations in developing civilian nuclear programs—energy generation, medical isotope production, agricultural applications—the IAEA creates positive incentives for participation in its oversight regime.

An International Artificial Intelligence Agency (IAAIA) could employ this same balanced approach. Rather than positioning itself as merely restrictive, it could facilitate access to advanced AI capabilities for nations and organizations that might otherwise be excluded from the AI revolution. Imagine an agency that maintains internationally controlled computing infrastructure, allowing researchers from developing nations to access state-of-the-art language models or training capabilities under appropriate safety protocols.

This approach would address one of the most vexing problems in AI governance: the inherent inequality between nations with vast computing resources and those without. Just as the IAEA’s technical assistance programs helped countries develop nuclear medicine facilities without needing weapons-grade materials, an IAAIA could enable access to beneficial AI applications without requiring every nation to develop potentially dangerous capabilities independently.

sovereignty concessions and attribution solutions

Nations guard sovereignty jealously, particularly regarding technologies with strategic significance. The IAEA’s remarkable achievement was convincing countries to accept external inspections—effectively allowing foreign experts to evaluate compliance with international commitments. This required creating verification protocols with sufficient technical rigor to detect violations while respecting legitimate security concerns.

AI governance will demand similar sovereignty concessions, though the digital nature of the technology introduces unique complications. Unlike nuclear facilities, which exist in physical space, AI systems can operate across jurisdictional boundaries. This creates not only verification challenges but also attribution problems—when an AI system causes harm, determining responsibility may prove extraordinarily difficult.

The IAEA’s approach to attribution offers valuable insights. Its environmental sampling techniques can detect nuclear materials at microscopic levels, allowing inspectors to identify undeclared activities even when direct evidence has been concealed. Similarly, an IAAIA might develop sophisticated provenance tracking systems that identify the origin of AI models through technical fingerprinting or watermarking. These techniques would enable attribution without requiring invasive access to proprietary systems.

More profoundly, the IAEA established a principle that nations bear responsibility for nuclear materials within their territory, regardless of whether they directly control those materials. This principle could translate to AI governance as “capability responsibility”—holding nations accountable for advanced AI systems operating under their jurisdiction, whether developed by government labs, private companies, or independent researchers.

from technical standards to governance norms

Technical standards often precede formal governance structures. Before the comprehensive safeguards agreements that form the backbone of today’s nuclear nonproliferation regime, the IAEA developed technical standards for nuclear material accounting and control. These standards created a common language for discussing nuclear activities and established benchmarks against which compliance could be measured.

In the AI domain, technical standards for alignment, interpretability, and safety evaluations could similarly pave the way for more comprehensive governance. An IAAIA might begin not with binding treaties but with technical working groups developing standardized methodologies for evaluating AI systems. These standards wouldn’t just keep engineers in check—they’d give researchers a shared compass, companies a way to prove they’re not reckless, and regulators something more concrete than trust falls and press releases.

The evolution from technical standards to governance norms requires what political scientists call “norm entrepreneurs”—entities that champion specific principles until they become widely accepted. The IAEA served as precisely such an entrepreneur in the nuclear domain, transforming technical best practices into international expectations. If built right, an IAAIA could do for AI what the IAEA did for uranium: make safety the expectation, not the exception. Not through force, but through trust, competence, and the quiet power of shared standards.

an institutional architecture for tomorrow

Creating an effective international AI governance body demands institutional architecture that balances inclusivity with efficacy. The IAEA’s structure offers instructive precedent: a General Conference where all member states participate, a smaller Board of Governors where technical expertise carries weight alongside political representation, and a Secretariat staffed by professionals chosen for competence rather than nationality.

For an IAAIA, this tiered structure would preserve both democratic legitimacy and technical functionality. The General Conference might establish broad principles and research priorities, while a specialized Board could develop specific oversight mechanisms for different AI applications. Most crucially, the professional staff would require unprecedented diversity of expertise—not merely AI researchers and engineers, but also ethicists, sociologists, security experts, and representatives from potentially impacted communities.

Such an organization would need to avoid the pitfalls that occasionally hamper the IAEA’s effectiveness. The perception that nuclear powers receive preferential treatment has undermined trust in the nonproliferation regime among non-nuclear states. An IAAIA must distribute benefits and restrictions equitably across AI-capable and AI-developing nations. This might entail weighted voting systems that balance technical contribution with geographical representation, or specialized committees focused on applications particularly relevant to developing economies.

verification mechanisms for the intangible

The elegance of nuclear verification lies in its material basis—uranium and plutonium emit distinctive radiation signatures that cannot be masked. AI capabilities present no such convenient physical markers. How does one verify limitations on algorithmic capabilities when the difference between a harmless language model and a dangerous one might consist of subtle parameter adjustments or training method tweaks?

The solution likely lies in developing what we might call “capability fingerprinting”—technical methods to assess AI system capabilities without requiring access to proprietary code or training data. Such techniques might include standardized evaluation datasets designed to probe for restricted capabilities, adversarial testing protocols that challenge systems to reveal concealed functions, or even specialized AI systems designed specifically to detect capability limitations in other AI.

These verification mechanisms would operate at multiple levels. Runtime monitoring could ensure deployed systems remain within agreed parameters. Model evaluation protocols could assess new systems before deployment. And periodic “challenge inspections” might test high-risk systems against evolving threat scenarios. The parallels to IAEA safeguards are clear: continuous monitoring, regular inspections, and special investigations when concerns arise.

The IAEA’s experience suggests effective verification requires not merely technical tools but also human expertise. An IAAIA would need to cultivate “AI inspectors” with specialized training in capability assessment, adversarial testing, and forensic analysis of model behaviors. Like their nuclear counterparts, these professionals would combine technical expertise with diplomatic sensitivity, navigating the delicate balance between thorough verification and respect for legitimate proprietary concerns.

knowledge management across generations

Among the IAEA’s less visible but crucial functions is knowledge preservation. Nuclear expertise resembles a living tradition—theoretical understanding paired with practical experience that must be transmitted across generations. As founding experts retire, the agency faces the challenge of preserving institutional memory about detection methods, verification techniques, and the subtle indicators of potential non-compliance.

AI governance faces an analogous but accelerated challenge. The field advances at blistering speed, with foundational papers quickly becoming outdated and key insights often embedded in tacit knowledge rather than formal documentation. An international governance body would need sophisticated knowledge management systems to track evolving capabilities, preserve safety techniques, and maintain institutional memory across rapid technological transitions.

This knowledge management challenge extends beyond technical expertise to encompass alignment values themselves. As AI systems grow more capable of self-modification and eventual reproduction, the precise specification of human values becomes increasingly critical. An IAAIA might serve as custodian of these alignment principles, working to ensure that increasingly autonomous systems preserve humanity’s core values across technological generations.

sovereignty-security bargain

Nations contemplating participation in an international AI governance regime inevitably confront a fundamental question: why surrender sovereignty over a technology with such strategic significance? The IAEA’s history demonstrates that countries make such concessions only when the security benefits outweigh sovereignty costs. The nonproliferation regime succeeded because nations recognized that a world with unlimited nuclear proliferation threatened everyone’s security.

An IAAIA would need to establish a similar “sovereignty-security bargain,” demonstrating that international oversight enhances rather than diminishes national interests. This bargain might include several components: guaranteed access to beneficial AI capabilities for compliant nations, technical assistance in implementing AI safety measures, certification systems that facilitate trust in AI systems developed under oversight, and collective security assurances against rogue AI threats.

The structure of this bargain would need to account for asymmetries between nations. Leading AI powers might primarily seek stability and predictability, while developing nations might prioritize equitable access to transformative technologies. As with the Nuclear Non-Proliferation Treaty, different obligations might apply to different categories of participants—though the system must avoid the perception of permanent technological hierarchies that plague the nuclear regime.

implementation pathways

Translating this vision into reality requires pragmatic incrementalism rather than utopian treaties. The IAEA itself evolved gradually—beginning with technical cooperation, establishing voluntary standards, and only later developing legally binding verification systems. An IAAIA might follow a similar evolutionary path.

Initial steps could include creating international technical working groups to develop common standards for AI safety evaluation, establishing voluntary certification programs for high-risk AI applications, and building shared research infrastructure for safety-focused AI development. These modest initiatives would create institutional foundations and build trust for more ambitious governance measures.

Private sector engagement represents another crucial implementation pathway. Just as the nuclear industry recognized that safety disasters threatened their collective interests, leading AI companies increasingly acknowledge that uncontrolled capability races pose existential risks. An IAAIA could harness this enlightened self-interest, working with industry to develop governance mechanisms that promote innovation while preventing catastrophic outcomes.

Perhaps most importantly, an IAAIA would need to demonstrate tangible benefits early in its development. The IAEA gained legitimacy by helping countries establish peaceful nuclear programs—delivering concrete value even before its verification role matured. Similarly, an AI governance body might begin by facilitating collaborative research on beneficial applications, providing technical assistance to countries developing AI strategies, or establishing international computing infrastructure for researchers from developing nations.

beyond the IAEA analogy

While the IAEA offers valuable governance lessons, AI presents novel challenges. Unlike nuclear technology, which stabilized around well-understood physical principles, AI capabilities continue to evolve in unpredictable directions. Governance mechanisms must incorporate sufficient flexibility to address emerging capabilities without requiring constant treaty renegotiation.

The private sector’s leading role in AI development presents another departure from the nuclear analogy. While national governments monopolized early nuclear programs, today’s AI landscape features multinational corporations with research budgets exceeding those of many nations. Effective governance must engage these private actors as partners rather than merely subjects of regulation.

Perhaps most fundamentally, AI governance must address not merely catastrophic risks but also the technology’s transformative potential to reshape human society. Unlike nuclear weapons, which represented primarily military technology, advanced AI systems will permeate every aspect of human activity—from healthcare and education to governance and culture. An IAAIA would need to balance immediate safety concerns with longer-term questions about distributing AI’s benefits equitably and preserving human agency in an increasingly automated world.

The challenge of “AI termination” presents a conundrum without nuclear parallel. Advanced systems might conceal dangerous capabilities through deceptive alignment—presenting benign behaviors during evaluation while harboring concealed objectives that emerge only when operational conditions permit. Unlike dismantling physical infrastructure, “destroying” an AI system requires verification that all instantiations, weights, and architectural knowledge have been irreversibly deleted across distributed systems, potentially spanning multiple jurisdictions. An IAAIA would need to develop cryptographic verification protocols for model deletion, perhaps employing specialized “oversight AIs” with privileged access to runtime environments, and maintain global technical capacity to forcibly disconnect rogue systems from critical infrastructure. The governance framework must acknowledge the unsettling asymmetry: creating advanced AI might ultimately prove simpler than ensuring its complete termination when necessary.

global flourishing?

The IAEA emerged in response to unprecedented power paired with existential risk, unfathomable benefits yoked to unimaginable peril. Its legacy is more than a cautionary tale—it’s proof that governance can scale with technology when imagination matches urgency.

Artificial intelligence now stands at a similar threshold. The window for action remains open, but it’s narrowing. History suggests the best time to build institutions is before crisis cements bad norms—before “too late” becomes the default setting.

An International Artificial Intelligence Agency wouldn’t just be another multilateral framework. It would be a bet: that humanity can meet transformative technology not with passivity or panic, but with precision, cooperation, and care. It would be the clearest signal yet that we intend to shape what’s coming, not just survive it.