Amazing…. Robots learn coordinated behavior from scratch.

Der and Martius suggest that a novel plasticity rule can explain the development of sensorimotor intelligence, without having to postulate higher-level constructs such as intrinsic motivation, curiosity, or a specific reward system.  This seems to me to be groundbreaking and fascinating work. I pass on their overview video, and then some context from their introduction, which I recommend that you read.  Here is their abstract. (I don't even begin to understand the description of their feed-forward controller network and humanoid robot, which follows a “chaining together what changes together” rule. I can send motivated readers a PDF of the whole article with technical details and equations.)

 

Research in neuroscience produces an understanding of the brain on many different levels. At the smallest scale, there is enormous progress in understanding mechanisms of neural signal transmission and processing. At the other end, neuroimaging and related techniques enable the creation of a global understanding of the brain’s functional organization. However, a gap remains in binding these results together, which leaves open the question of how all these complex mechanisms interact. This paper advocates for the role of self-organization in bridging this gap. We focus on the functionality of neural circuits acquired during individual development by processes of self-organization—making complex global behavior emerge from simple local rules.

Donald Hebb’s formula “cells that fire together wire together” may be seen as an early example of such a simple local rule which has proven successful in building associative memories and perceptual functions. However, Hebb’s law and its successors...are restricted to scenarios where the learning is driven passively by an externally generated data stream. However, from the perspective of an autonomous agent, sensory input is mainly determined by its own actions. The challenge of behavioral self-organization requires a new kind of learning that bootstraps novel behavior out of the self-generated past experiences.

This paper introduces a rule which may be expressed as “chaining together what changes together.” This rule takes into account temporal structure and establishes contact to the external world by directly relating the behavioral level to the synaptic dynamics. These features together provide a mechanism for bootstrapping behavioral patterns from scratch.

This synaptic mechanism is neurobiologically plausible and raises the question of whether it is present in living beings. This paper aims to encourage such initiatives by using bioinspired robots as a methodological tool. Admittedly, there is a large gap between biological beings and such robots. However, in the last decade, robotics has seen a change of paradigm from classical AI thinking to embodied AI which recognizes the role of embedding the specific body in its environment. This has moved robotics closer to biological systems and supports their use as a testbed for neuroscientific hypotheses.

We deepen this argument by presenting concrete results showing that the proposed synaptic plasticity rule generates a large number of phenomena which are important for neuroscience. We show that up to the level of sensorimotor contingencies, self-determined behavioral development can be grounded in synaptic dynamics, without having to postulate higher-level constructs such as intrinsic motivation, curiosity, or a specific reward system. This is achieved with a very simple neuronal control structure by outsourcing much of the complexity to the embodiment [the idea of morphological computation].