Intuitive physics learning in a deep-learning A.I. model

A fascinating open source article from Piloto et al. in Nature Human Behaviour that addresses a major shortcoming of current artifician intelligence systems:

‘Intuitive physics’ enables our pragmatic engagement with the physical world and forms a key component of ‘common sense’ aspects of thought. Current artificial intelligence systems pale in their understanding of intuitive physics, in comparison to even very young children. Here we address this gap between humans and machines by drawing on the field of developmental psychology. First, we introduce and open-source a machine-learning dataset designed to evaluate conceptual understanding of intuitive physics, adopting the violation-of-expectation (VoE) paradigm from developmental psychology. Second, we build a deep-learning system that learns intuitive physics directly from visual data, inspired by studies of visual cognition in children. We demonstrate that our model can learn a diverse set of physical concepts, which depends critically on object-level representations, consistent with findings from developmental psychology. We consider the implications of these results both for AI and for research on human cognition.

The Computational Society

The most recent issue of Trends in Cognitive Sciences presents a forum in its 25th Anniversary Series: Looking Forward. Several of the contribution are open source (you can email me to request access to those that are not), and I would like to point to Nick Charter's brief article "The computational society," passing on his initial and final comments. I suggest you read through his descriptions of what he thinks are four promising lines of work.

How do individual human minds create languages, legal systems, scientific theories, and technologies? From a cognitive science viewpoint, such collective phenomena may be considered a type of distributed computation in which human minds together solve computational problems beyond any individual. This viewpoint may also shift our perspective on individual minds.

To make the computational society more than a metaphor, we need conceptual tools and methods to understand social phenomena in information-processing terms. Fortunately, several different, yet complementary, approaches have emerged in recent years. Here I highlight four promising lines of work: (i) social interaction as computation, (ii) the computational Leviathan, (iii) collective self-correction and rationality, and (iv) computation through spontaneous order.

Cognitive science may stand on the brink of a new revolution, seeing social, organizational, and cultural processes as distributed computation. If so, we will need to look afresh at the computational role of individual minds. For example, rather than seeing each developing child as a lone minilinguist or a scientist-in-the-crib, we may, following Adam Ferguson, see humans as primarily learning to contribute to collective computations beyond the understanding of individual understanding.

The non-duality industry as a panacea for the anxieties of our times?

One of MindBlog's subject threads is meditation, and some recent posts have dealt with characterizing non-dual awareness (to find these, look in the right column of this blog, under "Selected Blog Categories, and click on 'meditation.') One of the descriptions I have pointed to is given by James Low, who has training in several lineages of Tibetan Buddhism, and has been teaching the principles of Dzogchen in Europe for over 20 years. Low's website links to an array of audio and video (YouTube and Vimeo) presentations he has done. One of the participants in a recent discussion at my house urged me to check out YouTube snippets recorded by Rupert Spiro, also British, whose YouTube videos and personal wesite (The Essence of Non-Duality) offer his teachings. 

After finding the YouTube outlets for these two meditation gurus, I googled 'non duality websites' and was rewarded with an array of rabbit holes to jump into...further teachers, and a "Nonduality Institute" that engages scientific studies of non-dual awareness. 

Perhaps an increasing number of people who engage techniques for facilitating non-dual awareness find themselves seeing and experiencing the "I" or self that feels threatened by our anxious times from a more useful perspective - an inclusive expanded awareness that includes the reporting "I" or self as just one of its many contents that include passing thoughts, perceptions, actions, and feelings.  A calm can be found in this expanded awareness that permits a  dis-association of the experienced breathing visceral center of gravity of our animal body from the emotional and linguistic veneer of politics and conflict. This does not remove the necessity of facing various societal dysfunctions, but offers the prospect of doing so without debilitating the organic physiological core from which everything we experience rises. 

The Huberman Lab Cornucopia

A friend mentioned enjoying a podcast on meditation from hubermanlab.com, so I listened to it, and decided to look a bit further into who Andrew Huberman is and what he does. Regarding his "How and Why to Meditate" podcast, I think his pedagogy is good. He does some very effective chunking of just a few core ideas and repeats them over and over again. Starting about a year ago he began to generate - completely separate from his lab research as an associate professor in the Standford University Medical School Neurobiology department - podcasts, interviews, and writing (see The Neural Network Newsletter). at an amazing rate, a veritable orgy of self-optimization nuggets ideally suited for his age cohort of 40- to 50-somethings. He has a rapid, logorrheic and rambling speaking style that, at least to me, detracts from the effectiveness of his presentations. I think MindBlog readers might enjoy clicking some of the above links and grazing through his material. Before his social media with thousands of followers persona burst on the scene, his publication list shows him puttering along the conventional academic research route, with his laboratory generating 1-4 papers a year on brain plasticity and repair, split roughly equally between laboratory research and commentary/review articles.

MindBlog in Crypto-Land Part II - Is the crypto industry headed for oblivion?

In late spring of this year, I was seduced by my son's having made a six hundred-fold return on an investment by virtue of being one of the first cohort to stick little black boxs (Helium miners costing ~$1,000) on their window sills earning HNT (Helium blockchain tokens) for transmitting and receiving signals in an 'Internet of Things" that piggybacks on existing wireless systems of cell towers and cable providers. I decided to take a sip of the koolaid and set up two Helium miners which to date have earned ~ $10 for a 0.01 % return on investment! Fortunately I decided not actually own a significant amount of any of the cryptocurrencies such as BitCoin or Etherium, and didn't face financial damage of the sort mentioned below.  I persist in liking the idea that block chain ledgers and their associated cryptocurrencies offer the promise of being a monetary system that doesn't require trust in financial institutions that are potentially intrusive or corruptible.

Krugman does not agree, and has issued his latest screed against the whole crypto context in a NYTimes Op-Ed occasioned by the recent implosion of Sam Bankman-Fried and his cryptocurrency exchange FTX:

Crypto reached its peak of public prominence last year, when Matt Damon’s “Fortune favors the brave” commercial — sponsored by the Singapore-based exchange Crypto.com — first aired...people who bought after watching the Damon ad have lost more than 70 percent of their investment.,,falling prices needn’t mean that cryptocurrencies are doomed...More telling than prices has been the collapse of crypto institutions...Most recently, FTX, one of the biggest crypto exchanges, filed for bankruptcy — and it appears that the people running it simply made off with billions of depositors’ dollars, probably using the funds in a failed effort to prop up Alameda Research, its sister firm.

After 14 years, however, cryptocurrencies have made almost no inroads into the traditional role of money. They’re too awkward to use for ordinary transactions...[they] are largely purchased through exchanges like Coinbase and, yes, FTX, which take your money and hold crypto tokens in your name...These exchanges are — wait for it — financial institutions, whose ability to attract investors depends on — wait for it again — those investors’ trust. In other words, the crypto ecosystem has basically evolved into exactly what it was supposed to replace: a system of financial intermediaries whose ability to operate depends on their perceived trustworthiness.

But if the government finally moves in to regulate crypto firms, which would, among other things, prevent them from promising impossible-to-deliver returns, it’s hard to see what advantage these firms would have over ordinary banks. Even if the value of Bitcoin doesn’t go to zero (which it still might), there’s a strong case that the crypto industry, which loomed so large just a few months ago, is headed for oblivion.

How focusing on individual-level solutions has led behavioral public policy astray

MindBlog has been staying out of politics lately, but I think it worthwhile to pass on the abstract of a forthcoming article in Behavioral and Brain Science by Chater and Loewenstein, who argue that focusing on individual rather than systemic causes of social problems has yielded disappointing results, and promotes the interests of corporate opponents of systemic change. (Motivated readers can obtain the full text by emailing me.)

An influential line of thinking in behavioral science, to which the two authors have long subscribed, is that many of society’s most pressing problems can be addressed cheaply and effectively at the level of the individual, without modifying the system in which the individual operates. We now believe this was a mistake, along with, we suspect, many colleagues in both the academic and policy communities. Results from such interventions have been disappointingly modest. But more importantly, they have guided many (though by no means all) behavioral scientists to frame policy problems in individual, not systemic, terms: to adopt what we call the “i-frame,” rather than the “s-frame.” The difference may be more consequential than i-frame advocates have realized, by deflecting attention and support away from s-frame policies. Indeed, highlighting the i-frame is a long-established objective of corporate opponents of concerted systemic action such as regulation and taxation. We illustrate our argument briefly for six policy problems, and in depth with the examples of climate change, obesity, retirement savings, and pollution from plastic waste. We argue that the most important way in which behavioral scientists can contributed to public policy is by employing their skills to develop and implement value- creating system-level change.

The neurophysiology of consciousness - neural correlates of qualia

This is a post for consciousness mavens.Tucker, Luu, and Johnson have offered a neurophyiological model of consciousness, Neurophysiological mechanisms of implicit and explicit memory in the process of consciousness. The open source article has useful summary graphics, and embraces the 'Hard Problem' of consciousness - the nature of 'qualia' (how it feels to see red, eat an apple, etc.) Here I pass on brief, and then more lengthy, paragraphs on what the authors think is new and noteworthy about their ideas.

The process of consciousness, generating the qualia that may appear to be irreducible qualities of experience, can be understood to arise from neurophysiological mechanisms of memory. Implicit memory, organized by the lemnothalamic brain stem projections and dorsal limbic consolidation in REM sleep, supports the unconscious field and the quasi-conscious fringe of current awareness. Explicit memory, organized by the collothalamic midbrain projections and ventral limbic consolidation of NREM sleep, supports the focal objects of consciousness.

Neurophysiological mechanisms are increasingly understood to constitute the foundations of human conscious experience. These include the capacity for ongoing memory, achieved through a hierarchy of reentrant cross-laminar connections across limbic, heteromodal, unimodal, and primary cortices. The neurophysiological mechanisms of consciousness also include the capacity for volitional direction of attention to the ongoing cognitive process, through a reentrant fronto-thalamo-cortical network regulation of the inhibitory thalamic reticular nucleus. More elusive is the way that discrete objects of subjective experience, such as the color of deep blue or the sound of middle C, could be generated by neural mechanisms. Explaining such ineffable qualities of subjective experience is what Chalmers has called “the hard problem of consciousness,” which has divided modern neuroscientists and philosophers alike. We propose that insight into the appearance of the hard problem can be gained through integrating classical phenomenological studies of experience with recent progress in the differential neurophysiology of consolidating explicit versus implicit memory. Although the achievement of consciousness, once it is reflected upon, becomes explicit, the underlying process of generating consciousness, through neurophysiological mechanisms, is largely implicit. Studying the neurophysiological mechanisms of adaptive implicit memory, including brain stem, limbic, and thalamic regulation of neocortical representations, may lead to a more extended phenomenological understanding of both the neurophysiological process and the subjective experience of consciousness.

Poisoned by Twitter - Trump, Musk and Kanye

Jaron Lanier, a computer scientist whose writing is always worth reading, has done a succinct must-read kind of piece in the NYTimes. I think you should read the whole brief essay, but will paste in a few clips:

When I compare Mr. Musk, Mr. Trump and Kayne West, I see a convergence of personalities that were once distinct. The garish celebrity playboy, the obsessive engineer and the young artist, as different from one another as they could be, have all veered not in the direction of becoming grumpy old men, but into being bratty little boys in a schoolyard. Maybe we should look at what social media has done to these men.

I believe “Twitter poisoning” is a real thing. It is a side effect that appears when people are acting under an algorithmic system that is designed to engage them to the max. It’s a symptom of being part of a behavior-modification scheme.

The human brain did not evolve to handle modern chemicals or modern media technology and is vulnerable to addiction. That is true for me and for us all.

Behavioral changes occur as a side effect of something called operant conditioning, which is the underlying mechanism of social media addiction. This is the core mechanism analogous to the role alcohol plays in alcoholism...What happened was that the algorithms that optimized the individualized advertising model found their way into it automatically, unintentionally rediscovering methods that had been tested on dogs and pigeons.

What do I think are the symptoms of Twitter poisoning? There is a childish insecurity, where before there was pride. Instead of being above it all, like traditional strongmen throughout history, the modern social media-poisoned alpha male whines and frets. This works because his followers are similarly poisoned and can relate so well.

I’ll suggest a hypothesis about the childishness that comes to the surface in social media addicts. When we were children, we all had to negotiate our way through the jungle of human power relationships at the playground. When we feel those old humiliations, anxieties and sadisms again as adults — over and over, because the algorithm has settled on that pattern as a powerful way to engage us — habit formation restimulates old patterns that had been dormant. We become children again, not in a positive, imaginative sense, but in a pathetic way.

Modern techies have revived a technocratic sensibility: a belief that great engineers can and should guide society. Whether that idea appeals or not, when technology degrades the minds of those same engineers, then the result can only be dysfunction.

Sleep preferentially consolidates negative aspects of human emotional memory

The Nov. 1, 2022 issue of PNAS has a special feature on Sleep, Brain, and Cognition. A large body of research suggests that sleep benefits memory, and I want to point in particular to an article by Denis et al. showing that sleep preferentially consolidates negative aspect of emotional memory. They also found that while research participants demonstrated better memory for positive objects compared to their neutral backgrounds, sleep did not modulate this effect.  

Significance

Recent research has called into question whether sleep improves memory, especially for emotional information. However, many of these studies used a relatively small number of participants and focused only on college student samples, limiting both the power of these findings and their generalizability to the wider population. Here, using the well-established emotional memory trade-off task, we investigated sleep’s impact on memory for emotional components of scenes in a large online sample of adults ranging in age from 18 to 59 y. Despite the limitations inherent in using online samples, this well-powered study provides strong evidence that sleep selectively consolidates negative emotional aspects of memory and that this effect generalizes to participants across young adulthood and middle age.

Abstract

Research suggests that sleep benefits memory. Moreover, it is often claimed that sleep selectively benefits memory for emotionally salient information over neutral information. However, not all scientists are convinced by this relationship [e.g., J. M. Siegel. Curr. Sleep Med. Rep., 7, 15–18 (2021)]. One criticism of the overall sleep and memory literature—like other literature—is that many studies are underpowered and lacking in generalizability [M. J. Cordi, B. Rasch. Curr. Opin. Neurobiol., 67, 1–7 (2021)], thus leaving the evidence mixed and confusing to interpret. Because large replication studies are sorely needed, we recruited over 250 participants spanning various age ranges and backgrounds in an effort to confirm sleep’s preferential emotional memory consolidation benefit using a well-established task. We found that sleep selectively benefits memory for negative emotional objects at the expense of their paired neutral backgrounds, confirming our prior work and clearly demonstrating a role for sleep in emotional memory formation. In a second experiment also using a large sample, we examined whether this effect generalized to positive emotional memory. We found that while participants demonstrated better memory for positive objects compared to their neutral backgrounds, sleep did not modulate this effect. This research provides strong support for a sleep-specific benefit on memory consolidation for specifically negative information and more broadly affirms the benefit of sleep for cognition.

The Neurobiology of long COVID

A number of my friends have reported, having caught break thru Covid even after 3-5 vaccinations, and are having symptoms of long Covid such as brain fog, anosmia, and cognitive impairment. (I am extremely grateful, after five vaccinations, to still be Covid free.) For these friends as well as Mind Blog readers, I want to point to a special issue of Neuron and in particular one open source article "The neurobiology of long Covid," which describes the array of neurological symptoms and their possible causes.

Sadder but Wiser? Maybe Not.

It looks like another universally accepted result of psychological research may be wrong - that depressed people have a more accurate reading of their ability to affect outcomes. Barry points to work by Dev et al. that fails to replicate experiments of Alloy and Abramson done 43 years ago that led to the hypothesis of “depressive realism,” that depressed people having a more realistic view of their conditions because they are free of the optimistic bias of their cheerful peers. The new research was unable to find any association between depressive symptoms and outcome bias. While Barry's review notes debate over whether differences in the design of the older and newer experiments may account for the variance in results, there now is certainly less confidence in the original findings.

Senescent cells targeted by anti-aging therapies may not be all bad

Michael Irving does a brief article in New Atlas that points to work by Reyes et al. showing that some senescent od the sort that accumulate with aging not only secrete inflammatory compounds that can be damaging to tisse around them, but also can play a positive role in repairing tissue damage. This suggests that senolytics research should focus on developing drugs that will target specific subsets of senescent cells that are implicated in disease rather than in regeneration.

Well being increases with diversity of social connections.

From Collins et al.: 

Significance

The link between social connection and well-being is well-documented: Happier people tend to spend more time with others, and people experience greater happiness while socially engaged. But, over and above people’s total amount of social interaction, which set of interactions—with which types of relationship partners (e.g., family members, close friends, acquaintances, strangers), and how many interactions with each type—is most predictive of well-being? Building on research showing the benefits of variety—in activities, experiences, and emotions—for well-being, we document a link between the relational diversity of people’s social portfolios and well-being. Assessing the social interactions and happiness of over 50,000 people reveals that interacting with a more diverse set of relationship types predicts higher well-being.

Abstract

We document a link between the relational diversity of one’s social portfolio—the richness and evenness of relationship types across one’s social interactions—and well-being. Across four distinct samples, respondents from the United States who completed a preregistered survey (n = 578), respondents to the American Time Use Survey (n = 19,197), respondents to the World Health Organization’s Study on Global Aging and Adult Health (n = 10,447), and users of a French mobile application (n = 21,644), specification curve analyses show that the positive relationship between social portfolio diversity and well-being is robust across different metrics of well-being, different categorizations of relationship types, and the inclusion of a wide range of covariates. Over and above people’s total amount of social interaction and the diversity of activities they engage in, the relational diversity of their social portfolio is a unique predictor of well-being, both between individuals and within individuals over time.

Molecular markers of eventual chronic diseases of aging are higher in young adults of lower socioeconomic status.

Sobering work from Shanahan et al.:  

Significance

The analysis of gene expression in peripheral whole blood of US young adults in their late 30s revealed socioeconomic status-based inequalities in the molecular underpinnings of the most common chronic conditions of aging. Associations involved immune, inflammatory, ribosomal, and metabolic pathways, and extra- and intra-cellular signaling. Body mass index was a plausible, sizable mediator of many associations. Results point to new ways of thinking about how social inequalities “get under the skin” and also call for renewed efforts to prevent chronic conditions of aging decades before diagnoses.

Abstract

Many common chronic diseases of aging are negatively associated with socioeconomic status (SES). This study examines whether inequalities can already be observed in the molecular underpinnings of such diseases in the 30s, before many of them become prevalent. Data come from the National Longitudinal Study of Adolescent to Adult Health (Add Health), a large, nationally representative sample of US subjects who were followed for over two decades beginning in adolescence. We now have transcriptomic data (mRNA-seq) from a random subset of 4,543 of these young adults. SES in the household-of-origin and in young adulthood were examined as covariates of a priori-defined mRNA-based disease signatures and of specific gene transcripts identified de novo. An SES composite from young adulthood predicted many disease signatures, as did income and subjective status. Analyses highlighted SES-based inequalities in immune, inflammatory, ribosomal, and metabolic pathways, several of which play central roles in senescence. Many genes are also involved in transcription, translation, and diverse signaling mechanisms. Average causal-mediated effect models suggest that body mass index plays a key role in accounting for these relationships. Overall, the results reveal inequalities in molecular risk factors for chronic diseases often decades before diagnoses and suggest future directions for social signal transduction models that trace how social circumstances regulate the human genome.

Observing the activity of our prosocial brains

Interesting work from Lockwood et al., open source with nice graphics:  

Highlights

• Prosocial behaviors frequently involve exerting effort 

• Human participants completed an effort-based decision-making task during fMRI 

• The anterior cingulate gyrus represented the effort costs of prosocial acts 

• Ventral tegmental area and ventral insula represented value for oneself

Summary

Prosocial behaviors—actions that benefit others—are central to individual and societal well-being. Although the mechanisms underlying the financial and moral costs of prosocial behaviors are increasingly understood, this work has often ignored a key influence on behavior: effort. Many prosocial acts are effortful, and people are averse to the costs of exerting them. However, how the brain encodes effort costs when actions benefit others is unknown. During fMRI, participants completed a decision-making task where they chose in each trial whether to “work” and exert force (30%–70% of maximum grip strength) or “rest” (no effort) for rewards (2–10 credits). Crucially, on separate trials, they made these decisions either to benefit another person or themselves. We used a combination of multivariate representational similarity analysis and model-based univariate analysis to reveal how the costs of prosocial and self-benefiting efforts are processed. Strikingly, we identified a unique neural signature of effort in the anterior cingulate gyrus (ACCg) for prosocial acts, both when choosing to help others and when exerting force to benefit them. This pattern was absent for self-benefiting behaviors. Moreover, stronger, specific representations of prosocial effort in the ACCg were linked to higher levels of empathy and higher subsequent exerted force to benefit others. In contrast, the ventral tegmental area and ventral insula represented value preferentially when choosing for oneself and not for prosocial acts. These findings advance our understanding of the neural mechanisms of prosocial behavior, highlighting the critical role that effort has in the brain circuits that guide helping others.

A lucid exposition on non-dual awareness by James Low

The ‘Waking Up’ app by Sam Harris has posted a series of lectures by James Low that “makes the esoteric teachings of Dzogchen—a non-dual contemplative tradition from Tibet—profoundly accessible.”   I want to pass on to MindBlog readers the following paragraph made up of small clips of text  I have taken from his lecture #4 “The Field of Experience.” Low’s website points to his lectures, writing, and videos of his lectures.

If you want stability, if you want real peace, you already have that in the nature of awareness. But if you look to manifestation, to patterning of yourself, to thinking you could establish a stable personalty, to live a life in which you were happy all the time, or in which you were your own person, that way madness lies. To find our original face, to find the ground of our primordial being, we need to release our fixation on the dialogic movement of subject and object, and allow ourselves to be the space within which the movement of experience is occurring. Awareness means being aware that we are present without being something as such. This is a great mystery. When we look at phenomena the world, things exist as something. A car is not a cow, an apple is not an orange, compare and contrast, category allocation. That’s how our cognition, our conceptual elaboration functions to give a seemingly enduring structure to identifications. But awareness can’t be caught. It’s not a thing. You can’t pin a tail on the donkey, there is no donkey there. The mind is not an object for itself, it is self luminous awareness, but you can’t catch it. You can never know your mind but you can be your mind. We are awareness and that’s a very important distinction.

Generative A.I. - more sociopathic than FaceBook and Twitter?

Have a look at this article on how the sociopathic effects of Facebook and Twitter could be dwarfed by what open source generative A.I. could do. I just played with the DALL-E 2 A.I. system that generates an image when you tell it what you want to see. In response to my request to show "Two abyssinian cats sitting in a window looking out at trees" the following spooky and accurate image appeared.

 

New Perspectives on how our Minds Work

I want to pass on to MindBlog readers this link to a lecture I gave this past Friday (10/21/22) to the Univ. of Texas OLLI (Osher Lifelong Learning Institute) UT FORUM group on Oct. 21, 2022. Here is the brief description of the talk:  

Abstract

Recent research shows that much of what we thought we knew about how our minds work is wrong. Rather than rising from our essential natures, our emotional and social realities are mainly invented by each of us. Modern and ancient perspectives allow us to have some insight into what we have made.

Description

This talk offers a description of how our predictive brains work to generate our perceptions, actions, emotions, concepts, language, and social structures. Our experience that a self or "I" inside our heads is responsible for these behaviors is a useful illusion, but there is in fact no homunculus or discrete place inside our heads where “It all comes together.” Starting before we are born diffuse networks of brain cells begin generating actions and perceiving their consequences to build an internal library of sensing and acting correlations that keep us alive and well, a library that is the source of predictions about what we might expect to happen next in our worlds. Insights from both modern neuroscience research and ancient meditative traditions allow us to partially access and sometimes change this archive that manages our behaviors.

Anxiety - What is it, when is it useful, when is it not?

The title of this post is the discussion topic for the Nov. 6 Austin Rainbow Forum, a monthly discussion group of LGBT seniors that first met in Austin Tx in Jan. 2018. I am using this post to pass on links to some optional background reading:

Martin Seligman and learned helplessness versus helpfulness

Eustress - beneficial stress

Bruce McEwen on good and bad stress

Robert Sapolsky on chronic stress

A deeper look into what our bodies are doing during arousal and calm (click on the arrows at bottom left of presentation frame to expand to full screen)

Facebook has no idea where it keeps our personal data.

A fascinating article from Biddle describing the situation that occured when a court ordered Facebook to turn over information it had collected about a lawsuit’s plaintiffs. Facebook...

...has amassed so much data on so many billions of people and organized it so confusingly that full transparency is impossible on a technical level. In the March 2022 hearing, Zarashaw and Steven Elia, a software engineering manager, described Facebook as a data-processing apparatus so complex that it defies understanding from within. The hearing amounted to two high-ranking engineers at one of the most powerful and resource-flush engineering outfits in history describing their product as an unknowable machine.

The special master at times seemed in disbelief, as when he questioned the engineers over whether any documentation existed for a particular Facebook subsystem. “Someone must have a diagram that says this is where this data is stored,” he said, according to the transcript. Zarashaw responded: “We have a somewhat strange engineering culture compared to most where we don’t generate a lot of artifacts during the engineering process. Effectively the code is its own design document often.” He quickly added, “For what it’s worth, this is terrifying to me when I first joined as well.”

The fundamental problem, according to the engineers in the hearing, is that Facebook’s sprawl has made it impossible to know what it consists of anymore; the company never bothered to cultivate institutional knowledge of how each of these component systems works, what they do, or who’s using them. There is no documentation of what happens to your data once it’s uploaded, because that’s just never been something the company does, the two explained. “It is rare for there to exist artifacts and diagrams on how those systems are then used and what data actually flows through them,”