Remaking Our Lives and Institutions in the Image of Neural Networks? No! - Smash the Machines!

After reading through a challenging article in Science by Evans and Weyl I asked ChatGPT, Claude, and Gemini to help me cook it down to a form more accessible to MindBlog readers.  I have edited their efforts to generate the summary below.  Seeing the future they want to suggest tempts me to run for the exit, help tear down the data centers, and become a Luddite. 

Here's the condensation:

We are pouring unprecedented resources into AI infrastructure, yet aggregate economic output and productivity barely reflect this investment. Evans and E. Glen Weyl argue that this productivity paradox stems from the fundamental mismatch that our societal institutions are not built to absorb this new form of machine cognition. To capture AI's benefits, they propose "reverse alignment"—the deliberate redesign of our institutions, norms, and skills to mirror the architectural grammar of neural networks.   

Historically, institutions that thrive are those that successfully reproduce the logic of their era’s dominant technology. Electrified factories delivered little for decades because they kept the old steam-shaft layout. Productivity came only after plants were rebuilt around distributed motors. (While matching is necessary it is not necessarily sufficient. Blockchain-style organizations faithfully copied a technology that turned out not to matter much.) Just as the multidivisional corporation mirrored the centralized aggregation of the assembly line, and the organizations governing the internet mirrored its packet-switched protocol stack, our future structures must instantiate the computational primitives of AI at a societal scale. 

Evans and Weyl suggest that AI’s threats come largely from several types of mismatch. Our current systems identify people by a few credentials, which are now cheap to fake. They make decisions by simple aggregation such as votes and markets, which can be flooded with bots. And they reward discrete outputs such as papers and albums, which AI can now replicate.They outline three features of neural networks and critical shifts they think will be necessary:   

Verification through Distributed Representations: Single points of verification (like a degree or social security number) are highly vulnerable to AI-generated fakes. Identity and trust should instead be anchored in a distributed web of attestations and microcredentials, verifying human uniqueness and heterogeneous skills without relying on a single vulnerable master key.   

Choice through Attentional Bridging: Traditional majority voting tallies and flattens a crowd, but AI's attention mechanism weights inputs based on mutual relevance. Civic institutions should adopt algorithms (similar to Twitter's Community Notes or Taiwan's Polis) that embed participants in latent opinion spaces and amplify contributions that bridge divides and build consensus, rather than simply counting the most votes.   

Value through Credit Assignment: Generative models devalue discrete outputs (like papers or musical albums) by rapidly replicating them. Instead, institutions should adopt backpropagation-style credit assignment, tracing and rewarding the marginal effect of a contribution. In science, as a primary test bed, research could be valued and funded based on the novelty and "surprise" it introduces to a field, rather than through easily gamed citation counts.   

Evans and Weyl argue that institutions able to absorb AI need to apply these principles at the scale of society.  The ultimate friction in this transition is psychological. Neural networks generate their predictive power within fluid, high-dimensional spaces that largely defy human interpretation. Yet, humans demand legibility for their institutions to remain legitimate and navigable. Bridging this gap between the complex architecture necessary to harness AI and the human need for legible systems will be the defining policy challenge of our era.