From Animal to Humans - Multimodality as a safeguard of honesty in communication and language

I pass on the abstracts of an article by Hex et al to appear in Behavioral and Brain Sciences.  Motivated readers can obtain a PDF of the manuscript by emailing me. The abstracts are followed by a commentary on the article.

Short Abstract
Multimodality characterizes nearly every communicative system, and we argue that this feature of communication plays an essential role in safeguarding signal honesty. We first discuss the importance of honesty in communication, and introduce socially-mediated controls as an alternative to intrinsic costs. We next outline how multimodality mitigates signal dishonesty, and highlight the importance of signal honesty in complex, cooperative species, such as humans, wherein acceptance may incentivize dishonesty. Finally, we urge researchers to investigate the role of multimodality and honesty in cooperative, “cheap” signals, emphasizing the need for comparative work on the forces that have shaped the evolution of communication.

Long Abstract

From spider dances to human language, multimodality is ubiquitous in natural communication systems. Much scholarship has been devoted to investigating why multimodality evolved and the role it plays in communication. Here, we highlight the role of multimodality in safeguarding the most fundamental prerequisite of all functioning, extant communication systems: honesty. We begin by introducing the arms race between honesty and deception in natural communication systems, and the critical role socially-mediated controls can play in maintaining signal honesty when classic, intrinsic costs are not sufficient. We next introduce three ways by which multimodality buffers signal honesty by 1) providing insurance against signal unreliability in dynamic environments, 2) forming an honest, multimodal gestalt with which to cross-validate signal honesty, and 3) increasing signal complexity, making the entire signal harder to fake. We then discuss the case of highly cooperative societies, with human language emphasized, and argue that signal honesty is important especially in complex and cooperative societies wherein the need to cooperate and be accepted as part of the group may supersede honesty. Finally, we
propose future directions wherein human and non-human communication research could expand beyond the well trodden realms of competition and mate attraction to investigate the role of multimodality and honesty in cooperative, “cheap” signals, and emphasize the importance of drawing from both the human and non-human literatures in investigating the forces that have shaped the evolution of communication. 

Commentary on this article from an astute MindBlog reader to whom I had sent the manuscript PDF:

What seems most important to me is this: today the problem is not a lack of signals, but their over-complex, recombinant, socially and technically pre-structured excess.

The article still seems to assume that a receiver can construct a reasonably stable basis for communication by integrating several signal channels. Under many older or more localized conditions, that makes sense. But in digital environments this assumption has become fragile. Signals can no longer be clearly assigned to one sender, one intention, or one context. What reaches us is often already a composite: fragments of persons, group styles, algorithmic selection, platform incentives, packaging, emotional cues, and recombined information.

In digital environments, multimodality increasingly loses the very function the article assigns to it. Instead of safeguarding honesty through cross-validation, it can become a vehicle for more persuasive forms of simulation, because the combined signals no longer arise from one coherent communicative source.

What seems necessary today is not just closer attention to signals, but a layered analytical process. At least two loops are needed: one directed at the immediate communicative act — who says what, in what tone, with what apparent intention — and another directed at the conditions that shape this act: group context, platform logic, aesthetic packaging, and algorithmic amplification. These loops cannot be separated cleanly, because the reading of the content changes the reading of the frame, and the reading of the frame changes the meaning of the content. In more complex cases, even a third loop may be needed, one that takes into account the wider circulation and reuse of the signal across the network.

That is why I think a simple theory-of-mind model is no longer enough. It is not sufficient to ask what a person means or wants. We also have to ask how the contribution is shaped before it reaches us, and how its form already prepares its reception.

This does not make the article less valuable. On the contrary, for me it helped clarify how much harder the problem has become. It is no longer only a matter of checking signals across modalities, but of reconstructing who or what is really communicating through them.

 

Dangerous Ideas.......

Deric's MindBlog is almost 20 years old. Its first post appeared on Feb. 8, 2006. The assertions and ideas described in that original post are as fresh and relevant now as they were then, before the appearance of the iPhone, social media, and contracting attention spans.  The Edge.org link that once took you to the essays supporting the 'dangerous ideas' now takes you to their published version on Amazon. The "Reality Club" and John Brockman's "Third Culture" cohort of intellectuals has largely dispersed, although you will note many names still quite prominent today.   Here is the 2006 post:

***************

Edge.org is a website sponsored by the "Reality Club" (i.e. John Brockman, literary agent/impressario/socialite). Brockman has assembled a stable of scientists and other thinkers that he defines as a "third culture" that takes the place of traditional intellectuals in redefining who and what we are.... Each year a question is formulated for all to write on... In 2004 it was "What do you believe is true even though you cannot prove it?" The question for 2005 was "What is your dangerous idea?"

The responses organize themselves into several areas. Here are selected thumbnail summaries most directly relevant to human minds. I've not included cosmology and physics. Go to edge.org to read the essays

I. Nature of the human self or mind (by the way see my "I-Illusion" essay on my website):

Paulos - The self is a conceptual chimera
Shirky - Free will is going away
Nisbett - We are ignorant of our thinking processes
Horgan - We have no souls
Bloom - There are no souls, mind has a material basis.
Provine - This is all there is.
Anderson - Brains cannot become minds without bodies
Metzinger - Is being intellectually honest about the issue of free will compatible with preserving one's mental health?
Clark - Much of our behavior is determined by non-conscious, automatic uptake of cues and information
Turkle - Simulation will replace authenticity as computer simulation becomes fully naturalized.
Dawkins - A faulty person is no different from a faulty car. There is a mechanism determining behavior that needs to be fixed. The idea of responsibility is nonsense.
Smith - What we know may not change us. We will continue to conceive ourselves as centres of experience, self-knowing and free willing agents.

II. Natural explanations of culture

Sperber - Culture is natural.
Taylor - The human brain is a cultural artifact.
Hauser- There is a universal grammar of mental life.
Pinker - People differ genetically in their average talents and temperaments.
Goodwin - Similar coordinating patterns underlie biological and cultural evolution.
Venter - Revealing the genetic basis of personality and behavior will create societal conflicts.


III. Fundamental changes in political, economic, social order

O'donnell - The state will disappear.
Ridley - Government is the problem not the solution.
Shermer - Where goods cross frontiers armies won't.
Harari -Democracy is on its way out.
Csikszentmihalyi- The free market myth is destroying culture.
Goleman - The internet undermines the quality of human interaction.
Harris - Science must destroy religion.
Porco - Confrontation between science and religion might end when role played by science in lives of people is the same played by religion today.
Bering - Science will never silence God
Fisher - Drugs such as antidepressants jeopardize feelings of attachment and love
Iacoboni - Media Violence Induces Imitative Violence - the Problem with Mirrors
Morton - Our planet is not in peril, just humans are.

Energetics and evolutionary fitness

I pass on the first paragraph of a perspectives piece in PNAS by Vermeij et al. that gives their message more thoroughly than the paper's abstract. Motivated readers can obtain a copy of the whole essay from me. 

Organisms acquire energy and material resources and convert these to activity and living biomass (1). The role of energy as currency (or power, energy per unit time) in evolution has long been recognized (2–6), but how energy acquisition and allocation affect evolution remains the subject of disagreement. In this perspective, we show how different assumptions about whether life operates in a dynamic steady state or whether it has expanded over the course of its history lead to contrasting predictions about adaptation, natural selection, and “fitness.” We conclude that models based on steady-state assumptions do not adequately account for observed patterns of adaptive change and evolutionary trends of increasing power and species richness over long periods of time, whereas models based on individual and collective power, which incorporate activity and the effects of organisms on their surroundings as components of survival and reproduction, reflect the history of adaptation more faithfully. The issue is important because energy (the currency of life) and power (energy acquired and expended per unit time) offer a unified framework for interpreting the course and outcomes of evolution. Models based on assumptions that reflect observed patterns should be more predictive than zero-sum models not only in the realm of evolution but also in ecology and economics.

How complex brains and cognition first arose

I have received a draft of an upcoming paper in Behavioral and Brain Sciences by Coombs and Trestman titled "A Multi-Trait Embodied Framework for the Evolution of Brains and Cognition across Animal Phyla "  It has a nice graphic indicating different brain regions whose functionalities are common to humans and phylogenetically different animals with complex brains  (crows, octopuses and honeybees).  Motivated readers can obtain a PDF of the article from me.  Here is the abstract :

Among non-human animals, crows, octopuses and honeybees are well-known for their complex brains and cognitive abilities. Widening the lens from the idiosyncratic abilities of exemplars like these to those of animals across the phylogenetic spectrum begins to reveal the ancient evolutionary process by which complex brains and cognition first arose in different lineages. The distribution of 35 phenotypic traits in 17 metazoan lineages reveals that brain and cognitive complexity in only three lineages (vertebrates, cephalopod mollusks, and euarthropods) can be attributed to the pivotal role played by body, sensory, brain and motor traits in active visual sensing and visuomotor skills. Together, these pivotal traits enabled animals to transition from largely reactive to more proactive behaviors, and from slow and two-dimensional motion to more rapid and complex three-dimensional motion. Among pivotal traits, high-resolution eyes and laminated visual regions of the brain stand out because they increased the processing demands on and the computational power of the brain by several orders of magnitude. The independent acquisition of pivotal traits in cognitively complex (CC) lineages can be explained as the completion of several multi-trait transitions over the course of evolutionary history, each resulting in an increasing level of complexity that arises from a distinct combination of traits. Whereas combined pivotal traits represent the highest level of complexity in CC lineages, combined traits at lower levels characterize many non-CC lineages, suggesting that certain body, sensory and brain traits may have been linked (the trait-linkage hypothesis) during the evolution of both CC and non-CC lineages.