From Animal to Humans - Multimodality as a safeguard of honesty in communication and language

I pass on the abstracts of an article by Hex et al to appear in Behavioral and Brain Sciences.  Motivated readers can obtain a PDF of the manuscript by emailing me. The abstracts are followed by a commentary on the article.

Short Abstract
Multimodality characterizes nearly every communicative system, and we argue that this feature of communication plays an essential role in safeguarding signal honesty. We first discuss the importance of honesty in communication, and introduce socially-mediated controls as an alternative to intrinsic costs. We next outline how multimodality mitigates signal dishonesty, and highlight the importance of signal honesty in complex, cooperative species, such as humans, wherein acceptance may incentivize dishonesty. Finally, we urge researchers to investigate the role of multimodality and honesty in cooperative, “cheap” signals, emphasizing the need for comparative work on the forces that have shaped the evolution of communication.

Long Abstract

From spider dances to human language, multimodality is ubiquitous in natural communication systems. Much scholarship has been devoted to investigating why multimodality evolved and the role it plays in communication. Here, we highlight the role of multimodality in safeguarding the most fundamental prerequisite of all functioning, extant communication systems: honesty. We begin by introducing the arms race between honesty and deception in natural communication systems, and the critical role socially-mediated controls can play in maintaining signal honesty when classic, intrinsic costs are not sufficient. We next introduce three ways by which multimodality buffers signal honesty by 1) providing insurance against signal unreliability in dynamic environments, 2) forming an honest, multimodal gestalt with which to cross-validate signal honesty, and 3) increasing signal complexity, making the entire signal harder to fake. We then discuss the case of highly cooperative societies, with human language emphasized, and argue that signal honesty is important especially in complex and cooperative societies wherein the need to cooperate and be accepted as part of the group may supersede honesty. Finally, we
propose future directions wherein human and non-human communication research could expand beyond the well trodden realms of competition and mate attraction to investigate the role of multimodality and honesty in cooperative, “cheap” signals, and emphasize the importance of drawing from both the human and non-human literatures in investigating the forces that have shaped the evolution of communication. 

Commentary on this article from an astute MindBlog reader to whom I had sent the manuscript PDF:

What seems most important to me is this: today the problem is not a lack of signals, but their over-complex, recombinant, socially and technically pre-structured excess.

The article still seems to assume that a receiver can construct a reasonably stable basis for communication by integrating several signal channels. Under many older or more localized conditions, that makes sense. But in digital environments this assumption has become fragile. Signals can no longer be clearly assigned to one sender, one intention, or one context. What reaches us is often already a composite: fragments of persons, group styles, algorithmic selection, platform incentives, packaging, emotional cues, and recombined information.

In digital environments, multimodality increasingly loses the very function the article assigns to it. Instead of safeguarding honesty through cross-validation, it can become a vehicle for more persuasive forms of simulation, because the combined signals no longer arise from one coherent communicative source.

What seems necessary today is not just closer attention to signals, but a layered analytical process. At least two loops are needed: one directed at the immediate communicative act — who says what, in what tone, with what apparent intention — and another directed at the conditions that shape this act: group context, platform logic, aesthetic packaging, and algorithmic amplification. These loops cannot be separated cleanly, because the reading of the content changes the reading of the frame, and the reading of the frame changes the meaning of the content. In more complex cases, even a third loop may be needed, one that takes into account the wider circulation and reuse of the signal across the network.

That is why I think a simple theory-of-mind model is no longer enough. It is not sufficient to ask what a person means or wants. We also have to ask how the contribution is shaped before it reaches us, and how its form already prepares its reception.

This does not make the article less valuable. On the contrary, for me it helped clarify how much harder the problem has become. It is no longer only a matter of checking signals across modalities, but of reconstructing who or what is really communicating through them.