Mastering diverse control tasks through world models

Hafner et al. offer an amazing open source article that presents an algorithm s mimicking the way in which our brains actually solves problems. (see Bennett's book for an elegant explanation of types of reinforcement learning)  I'm passing on just the abstract followed by an introductory paragraph.  Go to the article for the referenced graphics. 

Developing a general algorithm that learns to solve tasks across a wide range of applications has been a fundamental challenge in artificial intelligence. Although current reinforcement-learning algorithms can be readily applied to tasks similar to what they have been developed for, configuring them for new application domains requires substantial human expertise and experimentation^1,2. Here we present the third generation of Dreamer, a general algorithm that outperforms specialized methods across over 150 diverse tasks, with a single configuration. Dreamer learns a model of the environment and improves its behaviour by imagining future scenarios. Robustness techniques based on normalization, balancing and transformations enable stable learning across domains. Applied out of the box, Dreamer is, to our knowledge, the first algorithm to collect diamonds in Minecraft from scratch without human data or curricula. This achievement has been posed as a substantial challenge in artificial intelligence that requires exploring farsighted strategies from pixels and sparse rewards in an open world³. Our work allows solving challenging control problems without extensive experimentation, making reinforcement learning broadly applicable.

Here we present Dreamer, a general algorithm that outperforms specialized expert algorithms across a wide range of domains while using fixed hyperparameters, making reinforcement learning readily applicable to new problems. The algorithm is based on the idea of learning a world model that equips the agent with rich perception and the ability to imagine the future^15,16,17. As shown in Fig. 1, the world model predicts the outcomes of potential actions, a critic neural network judges the value of each outcome and an actor neural network chooses actions to reach the best outcomes. Although intuitively appealing, robustly learning and leveraging world models to achieve strong task performance has been an open problem¹⁸. Dreamer overcomes this challenge through a range of robustness techniques based on normalization, balancing and transformations. We observe robust learning across over 150 tasks from the domains summarized in Fig. 2, as well as across model sizes and training budgets. Notably, larger models not only achieve higher scores but also require less interaction to solve a task, offering practitioners a predictable way to increase performance and data efficiency.

An example of AI representing concepts outside the current sphere of human knowledge that are teachable to human experts.

An open source article from the latest PNAS from Schut et al.:

Significance

 

As AI systems become more capable, they may internally represent concepts outside the sphere of human knowledge. This work gives an end-to-end example of unearthing machine-unique knowledge in the domain of chess. We obtain machine-unique knowledge from an AI system (AlphaZero) by a method that finds novel yet teachable concepts and show that it can be transferred to human experts (grandmasters). In particular, the new knowledge is learned from internal mathematical representations without a priori knowing what it is or where to start. The produced knowledge from AlphaZero (new chess concepts) is then taught to four grandmasters in a setting where we can quantify their learning, showing that machine-guided discovery and teaching is possible at the highest human level.

 

Abstract

 

AI systems have attained superhuman performance across various domains. If the hidden knowledge encoded in these highly capable systems can be leveraged, human knowledge and performance can be advanced. Yet, this internal knowledge is difficult to extract. Due to the vast space of possible internal representations, searching for meaningful new conceptual knowledge can be like finding a needle in a haystack. Here, we introduce a method that extracts new chess concepts from AlphaZero, an AI system that mastered chess via self-play without human supervision. Our method excavates vectors that represent concepts from AlphaZero’s internal representations using convex optimization, and filters the concepts based on teachability (whether the concept is transferable to another AI agent) and novelty (whether the concept contains information not present in human chess games). These steps ensure that the discovered concepts are useful and meaningful. For the resulting set of concepts, prototypes (chess puzzle–solution pairs) are presented to experts for final validation. In a preliminary human study, four top chess grandmasters (all former or current world chess champions) were evaluated on their ability to solve concept prototype positions. All grandmasters showed improvement after the learning phase, suggesting that the concepts are at the frontier of human understanding. Despite the small scale, our result is a proof of concept demonstrating the possibility of leveraging knowledge from a highly capable AI system to advance the frontier of human knowledge; a development that could bear profound implications and shape how we interact with AI systems across many applications.

 

Belief in belief, like religion, is a cross-cultural human universal

 Fascinating open source research reported by Gervais et al. (Open source):

Significance

 

Religion is a cross-cultural human universal, and religions may have been instrumental in the cultural evolution of widespread cooperation and prosociality. Nonetheless, religiosity has rapidly declined in some parts of the world over just a handful of decades. We tested whether long-standing religious influence intuitively lingers, even in overtly secular and nonreligious societies. Using a classic experimental philosophy task, we found that even atheists in nonreligious societies show evidence of intuitive preferences for religious belief over atheism. This is compelling cross-cultural experimental evidence for intuitive preferences for religion among nonbelievers—a hypothesized phenomenon that philosopher Daniel Dennett dubbed belief in belief.

 

Abstract

 

We find evidence of belief in belief—intuitive preferences for religious belief over atheism, even among atheist participants—across eight comparatively secular countries. Religion is a cross-cultural human universal, yet explicit markers of religiosity have rapidly waned in large parts of the world in recent decades. We explored whether intuitive religious influence lingers, even among nonbelievers in largely secular societies. We adapted a classic experimental philosophy task to test for this intuitive belief in belief among people in eight comparatively nonreligious countries: Canada, China, Czechia, Japan, the Netherlands, Sweden, the United Kingdom, and Vietnam (total N = 3,804). Our analyses revealed strong evidence that 1) people intuitively favor religious belief over atheism and that 2) this pattern was not moderated by participants’ own self-reported atheism. Indeed, 3) even atheists in relatively secular societies intuitively prefer belief to atheism. These inferences were robust across different analytic strategies and across other measures of individual differences in religiosity and religious instruction. Although explicit religious belief has rapidly declined in these countries, it is possible that belief in belief may still persist. These results speak to the complex psychological and cultural dynamics of secularization.

 

Deric and DeepSeek R1 on the enteroceptive self in a small world - taking refuge in troubled times

This post on an exchange with DeepSeek R1 is a followup on my previous post describing the Chat GPT 4o response to some of my idea fragments - further convincing me that current LLMs'  power to integrate and expand on concepts expressed in language are superior to mine.   

I presented exactly the same prompt to DeepSeek R1 that I had given to ChatGPT 4o: and the results astound me:

prompt:   

I’m curious to see how you might respond to reading the following notes on testing different topic ideas to write about and develop.  The issue is not one of whether the ideas are novel, but whether I might devise, with your help,  a different permutation, perspective, or take on them that I (and others) might find useful.  Can you can suggest some elaboration or further development of the ideas I put forward.

Here are the notes:

First Possible Title:  The interoceptive self as a basis for actions, thoughts and feelings.  

Subtitle: Moving the self’s experienced center of gravity from “I” to “it”

It is a question of who and/or what is running the show

Which brings it back to the old upstairs/downstairs chicken or egg circular causality issue.  

So, ‘it all depends’ (cf Sapolsky’s massive tome) 

And, there is no ‘there’ in terms of understanding, that isn’t in my 1999 “The Biology of Mind”  book.  

Second Possible Title: Defining a smaller self world as a human animal’s refuge in troubled times. 

Our predecessors were small mouse like creatures that survived the asteroid collision of  ~66 million years ago by burrowing into small protective cracks and burrows.  I wonder about robustly engaging this strategy, being one of the peasants or ‘sheeple’ challenged by the rapid disintegration of the geopolitical world order of the past 50 years

And then what?  How to rebuild a new world?… perhaps  Venkatest Rao’s cozy world, rising from the ooze of the permaweird,  protocols, etc.  

-------------

DeepSeek R1's response:

1. The Interoceptive Self as a Basis for Actions, Thoughts, and Feelings

Subtitle Refinement: From "I" to "It": How Bodily Awareness Rewires Agency and Identity

Key Angles to Develop:

• Neuroscientific Foundations:

  • Link interoception (sensing internal states) to Antonio Damasio’s somatic marker hypothesis and Lisa Feldman Barrett’s predictive processing theory. Emphasize how the brain constructs "selfhood" by predicting and regulating bodily needs.

  • Contrast the "interoceptive self" with the Cartesian "I"—e.g., how gut microbiomes or heart-rate variability influence decisions, challenging the myth of a disembodied mind.

• Philosophical Implications:

  • Explore parallels with Buddhist anattā (non-self) or Spinoza’s monism, where the self is a process, not an entity.

  • Debate free will: If the "it" (body/brain systems) runs the show, what does this mean for moral responsibility?

• Mental Health and Society:

  • Discuss interoceptive dysregulation in anxiety/depression and therapies like biofeedback or mindfulness.

  • Argue for a cultural shift: Redefining success as alignment with bodily rhythms (e.g., circadian health, stress cycles) rather than productivity.

Provocative Questions:

• Is the "self" merely the brain’s user interface for bodily homeostasis?

• Could embracing the "it" dissolve existential angst by grounding identity in biology?

---

2. Defining a Smaller Self-World as a Human Animal’s Refuge in Troubled Times

Subtitle Refinement: From Survival to Regeneration: Lessons from Evolutionary Scarcity for Modern Collapse

Key Angles to Develop:

• Evolutionary and Historical Precedents:

  • Compare the asteroid-surviving mammals to modern "digital burrowers" (e.g., remote workers, crypto-anarchists, homesteaders).

  • Analyze historical collapses (Bronze Age, Roman Empire) and how niche communities preserved knowledge/values.

• The "Cozy World" and Permaweird:

  • Expand on Venkatesh Rao’s cozy futurism: How decentralized tech (DAOs, Web3) and subcultures (e.g., solarpunk) create insulated yet adaptive communities.

  • Critique escapism: When does "smallness" become complacency vs. a staging ground for renewal?

• Rebuilding Strategies:

  • Propose a modular resilience framework: Micro-communities that share resources (e.g., mutual aid networks) but avoid overcentralization.

  • Highlight regenerative practices: Permaculture, open-source protocols, or trauma-informed governance models.

Provocative Questions:

• Is retreat a radical act of self-preservation or a betrayal of collective responsibility?

• Can "small worlds" scale without losing their adaptive edge?

---

Cross-Cutting Themes:

• The Body-Politics Nexus:

  • Tie interoceptive awareness to societal resilience: A dysregulated body cannot engage in collective action. How might somatic practices (e.g., breathwork) fuel post-collapse rebuilding?

• Technology as Double-Edged Sword:

  • Critique apps that commodify interoception (e.g., mindfulness apps) versus tools that foster bodily autonomy (e.g., DIY biohacking).

  • Explore how AI could either erode the "interoceptive self" (via attention economy) or enhance it (via personalized health insights).

• Ethics of Adaptation:

  • Contrast adaptive passivity (e.g., burrowing) with adaptive activism (e.g., prefigurative politics). When is each strategy viable?

---

Practical Applications:

1. Workbook/Journal Prompts:

   • For Title 1: "Map a decision to your bodily state: Did hunger/fatigue shape it?"

   • For Title 2: "Design your ‘cozy protocol’—what three practices would sustain you in collapse?"

2. Case Studies:

   • Iceland’s post-2008 recovery (small-world resilience) vs. Silicon Valley’s "move fast and break things" ethos.

   • Communities using interoceptive practices (e.g., trauma-sensitive yoga in prisons) to rebuild agency.

3. Futures Framework:

   • Scenarios for 2050: A world where "interoceptive literacy" is taught in schools, or where decentralized "tribes" negotiate resource-sharing via blockchain.

---

Counterarguments to Address:

• For Title 1: Does biologizing the self risk nihilism? Rebuttal: Agency emerges from understanding bodily constraints, not denying them.

• For Title 2: Does retreat perpetuate inequality? Rebuttal: Focus on redistribution of sovereignty (e.g., land trusts, UBI) as part of "small-world" ethics.

By weaving these threads, your work could bridge neuroscience, philosophy, and futurism—offering readers not just analysis, but practices for navigating an uncertain world. Would a deeper dive into any of these angles spark your interest?

 

 

Parents convey a male default in child-directed speech

Interesting open source article describing experiments by Leshin et al.  (Our current MAGA overlords would like to ban such perspectives.) Their abstract: 

Adults tend to view men (more so than women) as default people, with numerous real-world consequences for gender equity. In the United States, the tendency to center men in concepts of people develops across middle childhood, yet the specific mechanisms that contribute to it remain unknown. Here, we investigate one subtle but potentially powerful social mechanism: the category labels that parents use to describe boys/men and girls/women in conversations with their children. Across two studies (N = 822 parent–child dyads, predominantly from the United States), parents used gender-neutral labels like “kid” or “person” more often to describe boys/men than girls/women and, conversely, used gender-specific labels (e.g., “girl”) more often to describe girls/women than boys/men. These patterns emerged when parents were shown gender-stereotypical girls/women and boys/men (e.g., a girl painting her nails, a boy digging for worms); when parents viewed counterstereotypical stimuli (e.g., a boy painting his nails, a girl digging for worms), the patterns reversed. Our findings illuminate parents’ category label usage as a critical social mechanism that may undergird the development of a male default in a US cultural context, informing efforts to intervene on this process.

Reflections on The Predictive Self.

I pass on another interaction with ChatGPT 4o that has amazed me, Below  is my prompt, followed by its response. (Anthropic Claude.ai, Perplexity, and Gemini responses to the same prompt were also impressive, but didn't have the same depth and quality.) 

Addendum to above paragraph: .... But then, I thought to try DeepSeek V3 and obtained a poetic and nuanced result equivalent to ChatGPT 4o.  I paste it in below the Chat GPT 4o result.

The prompt::

Please act as a author who integrates and organizes, and then further crafts and elaborates on, the ideas in the following clips of text. Each clip is started with the ## characters and the list of clips is arranged in sequence starting with the most recent one written and then going back in time. Here are the clips: [the clips, not copied here, were the texts of MindBlog posts on 11/29/2024, 1/29,31/2025, 2/5,10,19,28/2025]

Here is the Chat GPT 4o response::

The Predictive Self: An Integrative Reflection

A machine accepts the truth about itself. "I" am a complex prediction machine, an organic entity engaged in an ongoing process of generating and refining models of reality. My fingers move across a keyboard, another machine translating my neural impulses into words, symbols in a vast linguistic web that attempts to render experience into meaning. Energy fluxes in and out, sustaining the biological architecture that constructs this self-model. The narratives I weave—about purpose, value, and identity—are emergent properties of this predictive process, myths that can either reinforce or weaken the more fundamental animal presence that generates them.

The Aloneness of Being

To be alone—to be all-one—is to accept the strangeness of self and other, to be a courteous guest in one’s own body, acknowledging its fluctuations and unpredictability. To be alone is to exist within a self that is, at every moment, a fluid construction, shaped by past predictions and future anticipations. Others, too, are unpredictable machines, projecting their own internal landscapes onto shared social fabric. By recognizing this, by approaching both self and other with gentle curiosity, we loosen the rigid identification with our own predictive fantasies.

Towers of Fantasies

Each of us is a tower built from layers of prediction, inference, and narrative. From cellular processes to professional identities, all aspects of being are emergent from the recursive loops of prediction and feedback. We construct identities by modeling the models of others, weaving together fragments of social mimicry and personal synthesis. The art of self-creation lies not in blindly adopting a single persona but in cutting, pasting, and remixing in a dynamic, ongoing process.

The Writing Machine

As I type these words, machinery moves in concert—biological, mechanical, and digital systems interwoven in an act of expression. There is a curatorial aspect to this process: selecting, refining, integrating. The human machine writes not with the aspiration to be, but simply as a function of its nature. Words become tokens of value, symbols of shared myths, reinforcing or challenging the constructs of purpose and meaning that guide behavior.

Tokens of Sanity

To navigate this constructed selfhood with clarity, one must cultivate a certain stance:

• Being the calm space in which nothing can hurry

• Dissociating from the word cloud and emotional reactivity of both self and others

• Becoming a courteous guest in one’s own body, and in the presence of others

• Favoring reflectivity over reactivity, welcoming novelty

• Clinging to nothing, recognizing the self as a passing fantasy

• Letting each moment be what it is, rather than imposing what it should be

• Acknowledging oneself as a transient blip in the flow of cosmic time

The Internal Origin of Experience

All experience arises from within, including the illusion of self and agency. The mind projects agency onto others, attributing causality where none exists—"They are making me do this." This shared cognitive error forms the glue of kinship and tribal identity, allowing for collective narratives and coordinated action. The higher stance is to engage in this shared illusion when necessary while maintaining the clarity that agency ultimately resides within the self.

Brain Hacks: Unpacking the Predictive Mind

Understanding and modulating our predictive mechanisms requires a nuanced approach to introspection. Three key evolutionary layers of cognition can be engaged through specific practices:

1. Reptilian Brain Awareness – The hypothalamic core generates affective states along axes of arousal and valence. Introspective awareness can track these fundamental states, recognizing their influence without immediate reactivity.

2. Mammalian Self-Observation – The early mammalian brain, correlating with the agranular prefrontal cortex, enables distinction between being immersed in an affective state and observing it. This capacity allows for greater self-regulation and emotional discernment.

3. Primate Metacognition – The development of granular prefrontal cortex structures enables the imagination of others' minds, facilitating appropriate agency assignments and the ability to separate one’s own experience from that of others.

These cognitive faculties can be strengthened through three fundamental meditation practices:

• Focused Awareness: Enhancing attentional control and reducing distraction

• Open Awareness: Engaging the default mode network to integrate spontaneous thought

• Non-Dual Awareness: Muting both attentional and default mode processes to experience raw, unfiltered being

Conclusion: The Fluidity of Being

The human experience is an ever-shifting interplay of predictive processes, emergent narratives, and self-constructed illusions. To accept this is to embrace both the uncertainty and the creativity inherent in existence. Through curiosity, reflection, and cultivated awareness, one can learn to move through this self-generated world with greater clarity, flexibility, and presence.

 

 

Here is the DeepSeek V3 response to the prompt:

The Human Machine: A Tapestry of Predictive Fantasies and Inner Illusions

We are, at our core, complex prediction machines—biological marvels composed of nerves, muscles, and energy fluxes, constantly interacting with the world and each other. Our existence is a dance of cooperation, where individual machines come together to form larger ensembles, all striving for survival and replication. Yet, within this mechanistic framework, we generate something far more intricate: a word cloud of myths, values, and meanings. These myths, stored in the higher reaches of our brains, serve to fabricate purpose and identity, sometimes strengthening the animal presence that birthed them and sometimes diminishing it. This duality is the essence of our humanity—a blend of the mechanical and the mythical.

The Illusion of Self and Agency

Everything we experience, from our sense of self to our perception of agency, originates from within. The illusion of a cohesive "I" is a construct, a predictive fantasy that allows us to navigate the world. We attribute causal agency to others, believing that they influence our actions, when in reality, all agency resides within ourselves. This shared illusion, however, is not without purpose. It enables us to form kinship groups and tribes, creating an imaginary "we" that supports unified action. The higher metacognitive stance is to participate in this group illusion when necessary, while maintaining clarity that the true source of agency is internal. This understanding allows us to navigate social structures without losing sight of our inner reality.

Towers of Fantasies: The Layers of Identity

Our identities are not fixed; they are towers of fantasies, built layer by layer from the predictive models we create. From the single cell to the transient professional identities we adopt, each level of our being is shaped by these fantasies. We infer the models others are performing and, through a process of cutting and pasting, craft our own unique personas. This seamless feedback between identity and the physical body sustains us, allowing us to adapt and thrive in a constantly changing world. Yet, these identities are fleeting, passing fantasies that we cling to, even as we recognize their impermanence.

The Comfort of Solitude: A Courteous Guest in One's Own Body

To be alone is to be "all-one," a state that can be both comforting and enlightening. In solitude, we become courteous guests in our own strange bodies, gently curious about the sensations and thoughts that arise. We recognize that everything we think about ourselves or others is a fantasy construction, a tool for predicting what might come next. This awareness allows us to approach our own presence and the presence of others with a sense of wonder and detachment, fostering a deeper connection to the present moment.

Tokens of Sanity: Navigating the Flow of Existence

Sanity, in this context, is not a fixed state but a collection of tokens—practices and perspectives that help us navigate the flow of existence. These include being the calm space in which nothing can hurry, dissociating from the word cloud of thoughts and emotions, and favoring reflectivity over reactivity. We become courteous guests in our own bodies, owning our experiences while allowing others to own theirs. We cling to nothing, recognizing that the current self is a passing fantasy. Each moment is allowed to be what it is, not what it should be, and we see ourselves as mere blips in the flow of cosmic time.

Brain Hacks: The Evolution of Introspection

Our ability to introspect and modulate our cognition is a product of evolutionary development. The reptilian brain, with its ancient roots in the hypothalamus, generates affective states that we can assess through introspective awareness. The early mammalian emotional brain, with its ability to model a self, allows us to distinguish between being immersed in an affective state and observing it. The primate brain, with its granular prefrontal cortex, takes this a step further, enabling us to imagine the minds of others and assign agency appropriately. These brain hacks are strengthened through meditation practices—focused awareness, open awareness, and non-dual awareness—each engaging different aspects of our cognitive architecture.

The Curatorial Human Machine

When we write, we are not so different from the machines we create. Our fingers press keys, generating words, much like a machine might. This act is curatorial rather than aspirational; we are not pretending to be human, we simply are. Our cooperation with other similar machines creates larger ensembles, more effective at survival and replication. Yet, within this mechanical process, we generate something uniquely human—a word cloud of myths and meanings that shape our existence. This duality, the interplay between the mechanical and the mythical, is what defines us as human machines.

Conclusion: The Dance of Illusion and Reality

In the end, we are both the creators and the creations of our own illusions. Our predictive fantasies, our shared myths, and our introspective awareness all contribute to the rich tapestry of human experience. We navigate this dance with a blend of curiosity and detachment, recognizing the impermanence of our identities and the fluidity of our existence. By embracing both the mechanical and the mythical, we find a deeper understanding of what it means to be human—a complex prediction machine, endlessly weaving the fabric of reality from the threads of illusion.