AI as discovery, not invention.

After reading Venkatesh Rao's essay "The Crooked Timber of AI" several times, I turned to my trio of ChatGPT, Gemini, and Claude subscriptions to condense its central points more clearly than I would be able to. I am increasingly finding that ChatGPT's responses are more clear and useful than those of Gemini and Claude. Here is the ChatGPT summary:  

The Crooked Timber of AI — Condensed

Venkatesh Rao’s central claim is that we misunderstand artificial intelligence because we treat it primarily as an invention—a machine deliberately designed according to intelligible principles—when its most consequential aspect is better understood as a discovery. Neural networks are engineered artifacts, but the vast behavioral worlds revealed when they are trained on human language and culture were not explicitly designed. They are territories we have stumbled into.

This distinction matters because inventions and discoveries require different ways of understanding them. An invention such as a jet engine can be understood by examining its components, deriving mathematical models, and identifying predictable relationships between design and performance. A discovery—a continent, an ecosystem, or an unfamiliar species—must instead be explored, mapped, classified, and gradually inhabited. One does not deduce a continent from first principles. One learns it by traveling through it. Rao argues that AI research and governance are still dominated by the mentality appropriate to inventions, even though the most important questions now concern a newly discovered behavioral landscape.

AI as concentrated human crookedness

Rao borrows Kant’s phrase, popularized by Isaiah Berlin: “Out of the crooked timber of humanity, no straight thing was ever made.” Human beings are inconsistent, historically situated, conflicted, and incapable of being reduced to a single rational design. AI models, having absorbed enormous portions of humanity’s linguistic and cultural record, inherit this crookedness. Indeed, they concentrate it.

Consequently, AI is not a clean, uniform engineering material. Its behavior has something analogous to wood grain, knots, cracks, variations in strength, and unpredictable responses to pressure. These irregularities arise not merely from errors in the model’s construction but from the heterogeneous historical material on which it was trained. Models are shaped as much by what they consume as by their formal architectures.

Yet AI discussion repeatedly treats concepts such as intelligence, agency, AGI, and alignment as if they were stable natural categories. New model behaviors are forced into these preexisting classifications rather than being allowed to challenge them. Rao calls this an “architectonic” approach: constructing a grand conceptual system first and then interpreting observations through it. The alternative is phenomenological and empirical—carefully observing what particular models do under particular conditions and developing concepts from those encounters.

Why AI’s builders may not understand AI best

A provocative implication is that technical genius and privileged access do not necessarily confer philosophical insight. Frontier-laboratory leaders may understand the machinery better than anyone else, but that does not mean they understand the territory the machinery has uncovered.

Rao distinguishes AI machinery from AI proper. The machinery includes backpropagation, attention mechanisms, processors, parameter counts, context windows, and training infrastructure. These are formidable engineering accomplishments, but their basic principles are comparatively intelligible. The deeper mystery lies in the behaviors that emerge when these mechanisms operate across immense quantities of human-produced data.

The engineers who built marine chronometers helped make oceanic exploration possible, but their knowledge of clocks did not make them authorities on the geography, ecology, and societies of the Americas. Similarly, those who construct model architectures and computing clusters are not necessarily best equipped to explain the strange behavioral spaces those systems reveal.

Understanding such a space requires an expeditionary and cartographic mentality. Researchers must accumulate observations, compare models, document anomalies, devise instruments for examining internal activity, and remain willing to revise their classifications. An ordinary user who spends months investigating a peculiar corner of model behavior may sometimes learn something that a celebrated researcher with enormous computing resources has missed.

The philosophical “Bitter Lesson”

Earlier symbolic or “good old-fashioned” AI was genuinely an invention-centered enterprise. Researchers designed planners, rule systems, theorem provers, chess programs, and knowledge bases. These systems were intellectually intricate and often elegant, even though they failed to achieve the broad intelligence imagined for them.

Deep learning changed the situation. Instead of manually specifying most of the intelligence, researchers created learning machinery and exposed it to vast bodies of data. The result was not simply a better-designed artificial mind. It was the opening of an unexpectedly large behavioral domain—often called “latent space”—whose structure was neither fully anticipated nor explicitly programmed.

Rao interprets the familiar AI “Bitter Lesson” more broadly. The technical lesson was that general learning methods using large amounts of computation tend to outperform hand-crafted representations. The corresponding philosophical lesson is that observation of actual model behavior will increasingly outperform elegant theories constructed in advance.

The old AI researchers were inventors seeking to build intelligence. Instead, their successors became something like clockmakers who accidentally enabled the discovery of a continent. Confusion follows when the expertise needed to build the vessel is assumed to be the expertise needed to understand the landfall.

Where AI’s real mysteries lie

Rao does not think AI has yet substantially illuminated the classic philosophical mysteries of consciousness, subjectivity, selfhood, or the “hard problem.” Present systems may complicate certain thought experiments, but they have not clearly altered the fundamental questions of philosophy of mind.

The genuinely surprising discoveries lie elsewhere. AI is revealing unsuspected properties of:

• language and other structured forms of expression;
• the accumulated historical record encoded in data;
• the relationship between knowing and what is known;
• memory, time, energy, and computation;
• the ways complex behavior can emerge from statistical learning.

“Language” here includes more than words. It can encompass protein structures, bodily movement, images, programming languages, and other patterned domains. Models suggest that vast amounts of implicit structure were present in these records without our recognizing them. AI makes portions of that structure operationally accessible, but not automatically intelligible.

These mysteries cannot be resolved by applying a few abstract principles from an armchair. They demand repeated encounters with models, specialized observational tools, comparative cataloguing, and acceptance that early maps will be incomplete or misleading.

Isaiah Berlin and the failure of perfect alignment

Rao turns to Isaiah Berlin because Berlin’s moral and political pluralism provides a better model for thinking about AI than rationalist schemes promising a final, harmonious solution.

Berlin argued that human beings pursue multiple genuine values that can conflict irreducibly. Freedom, equality, justice, loyalty, creativity, security, mercy, and truth cannot always be maximized together. There may be no higher principle capable of reconciling every conflict. Moral and political life therefore requires situated judgment, compromise, maintenance, and sometimes painful sacrifice.

This challenges the usual geometric metaphor of alignment. Alignment talk often implies that there is a coherent human objective toward which AI systems can be pointed. But humanity has no single line, destination, or internally consistent set of preferences. Even one individual’s values may conflict, and different individuals and societies pursue incompatible ends.

The aspiration to create a perfectly “aligned AGI” can therefore resemble earlier hopes for a final political or moral settlement—a once-and-for-all system that resolves conflict and guarantees a desirable future. Berlin regarded such visions as both incoherent and dangerous. The realistic objective is not perfection but a social order that avoids intolerable outcomes, preserves a reasonable range of human purposes, and remains subject to continuous repair.

Rao also rejects the romantic alternative—the hope that human creativity, taste, artistic judgment, or poetic intuition will supply what rational optimization cannot. Art can challenge oppressive universalisms, but aesthetic judgment is no substitute for moral and political reasoning. Treating society as a work of art to be shaped by visionary creators has historically justified brutality as readily as liberation.

Neither rationalist engineering nor romantic humanism can straighten crooked timber. Both tend to imagine that conflict can be transcended through the right universal framework. Berlin’s harder conclusion is that moral life consists of negotiating particular conflicts without assuming that a complete synthesis exists.

From universal solutions to protocols

Rao’s practical answer is protocols. A protocol is not a comprehensive blueprint for an ideal system. It is a set of procedures, conventions, interfaces, and safeguards that allows differently motivated participants to interact while managing recurring tensions.

Effective protocols usually develop from actual practices—“paved cowpaths”—rather than being imposed wholly from abstract theory. They embody accumulated local experience and can be revised when circumstances change. They do not eliminate crookedness; they make it workable.

This makes protocols especially appropriate for AI deployment. Organizations and societies must now address questions such as:

• how AI should be incorporated into existing institutions;
• how AI-native organizations should operate;
• how national and transnational systems should interact with AI;
• how warfare, diplomacy, commerce, and public administration will change;
• how competing values and jurisdictions can coexist.

These are not questions with permanent technical answers. They involve shifting actors, incompatible interests, unforeseen model behavior, and changing social circumstances. Governance must therefore be provisional, adaptive, and maintained rather than “solved.”

Rao places AI near the transition from an installation phase, dominated by infrastructure construction and rapid proliferation, to a deployment phase, in which societies reorganize themselves around the technology. But AI differs from railroads, electricity, and the internet because installation and discovery are happening simultaneously. We are laying tracks into a territory while still trying to determine what kind of territory it is.

The underlying message

The essay ultimately argues against the fantasy that AI can be reduced to a small number of grand concepts and then brought under control through a definitive theory. AI is not a clean intelligence waiting to be pointed toward a coherent human purpose. It is a historically formed, uneven behavioral medium distilled from humanity’s own accumulated contradictions.

The appropriate response is therefore not to abandon engineering, theory, safety work, or regulation. It is to recognize their limits and supplement them with sustained exploration:

1. Observe before theorizing. Begin with what particular models actually do rather than forcing their behavior into inherited categories.
2. Map variation and irregularity. Treat model behaviors as geographically and materially uneven, not as expressions of a single measurable essence called intelligence.
3. Abandon the search for final alignment. Human purposes are plural, constructed, historically situated, and sometimes irreconcilable.
4. Prefer adaptive institutions and protocols. Build arrangements that manage concrete conflicts, prevent intolerable outcomes, and can be continually revised.
5. Accept permanent incompleteness. AI governance will require ongoing attention and repair, not a one-time technical or philosophical solution.

Rao’s “crooked timber” is therefore not simply a defect. The irregularity of human beings—and of the AI systems trained on their cultural products—is also the source of variety, novelty, and meaning. The task is not to straighten the material completely, which would destroy it, but to learn its grain well enough to build with it.