Agentic AI and the next intelligence explosion

This post is the second of two recursive returns to engage the ideas of Blaise Agüera y Arcas, which were the subject MindBlog posts on 3/13/26 and 3/16/26.  Here is text of Evans, Bratton, and Agüera y Arcas, as summarized by ChatGPT:

The article, “Agentic AI and the next intelligence explosion,” by James Evans, Benjamin Bratton, and Blaise Agüera y Arcas, argues against the familiar “singularity” image of one superintelligent machine bootstrapping itself into godlike autonomy. The authors say that model is probably wrong at its core. Intelligence, in their view, is not a single scalar quantity that one mind simply has more or less of. It is plural, relational, distributed, and social. The next “intelligence explosion” will not look like one silicon mind rising above us, but like a vast ecology of human and nonhuman agents interacting, arguing, coordinating, competing, and forming institutions.

Their key move is to treat agentic AI as continuous with earlier evolutionary jumps in intelligence. Primate intelligence scaled with social group life; human language created a “cultural ratchet”; writing, law, bureaucracy, and markets externalized cognition into institutions that no individual fully understood. AI, in this picture, is another step in that sequence: the accumulated products of human social cognition have been compressed into models and made operational in a new substrate. What is becoming powerful is not isolated abstract reason, but social intelligence reanimated in computational form.

The article also points inward, to what happens inside reasoning models. The authors cite work suggesting that frontier reasoning models do not merely improve by “thinking longer.” They appear to generate internal, multi-perspective conversations: arguing, checking, revising, and reconciling. They call this a “society of thought.” The claim is that strong reasoning often emerges from structured disagreement, even when the “group” is simulated inside one model. This echoes an older cognitive-science idea: reasoning is not just private calculation but an internalized social process.

From there the authors shift to design. If intelligence is social, then better AI will not come only from larger models or more compute. It will come from building richer agent societies: systems with roles, hierarchy, specialization, parallel deliberation, devil’s advocacy, conflict norms, and institutional checks. Current reasoning models are likened to a single “AI town hall transcript”; future systems may need architectures closer to organizations, courts, labs, markets, or bureaucracies, where different agents occupy different functional roles and constrain one another.

This leads to their main governance argument. The dominant alignment picture, reinforcement learning from human feedback, is framed as too dyadic: a parent correcting a child. That may not scale to worlds containing billions or trillions of interacting agents. The authors propose “institutional alignment” instead: not just making individual agents nice, but designing persistent protocols, roles, audits, checks, and countervailing powers. A courtroom works because judge, attorney, jury, procedure, evidence, and appeal are structured roles; similarly, AI systems will need institutional architectures, not merely tuned personalities.

The concluding image is that the intelligence explosion is already beginning, but as a city, not a single meta-mind. It is visible in human-AI “centaur” workflows, internal societies of thought within models, recursive agent systems that fork and recombine, and emerging questions of constitutional governance among artificial and human actors. Their final message is blunt: the central issue is not whether intelligence will become more powerful, but whether we build social and institutional infrastructure adequate to the kind of intelligence that is actually emerging.