An animal mind is not born as an empty canvas: Bottlenose dolphins know how to swim and honey bees know how to dance without ever having learned these skills. Little is known about how animals acquire the instincts that enable such innate behavior. Instincts are widely held to be ancestral to learned behavior. Some have been elegantly analyzed at the cellular and molecular levels, but general principles do not exist. Based on recent research, we argue instead that instincts evolve from learning and are therefore served by the same general principles that explain learning.
Tierney first proposed in 1986 that instincts can evolve from behavioral plasticity, but the hypothesis was not widely accepted, perhaps because there was no known mechanism. Now there is a mechanism, namely epigenetics. DNA methylation, histone modifications, and noncoding RNAs all exert profound effects on gene expression without changing DNA sequence. These mechanisms are critical for orchestrating nervous system development and enabling learning-related neural plasticity.
For example, when a mouse has experienced fear of something, changes in DNA methylation and chromatin structure in neurons of the hippocampus help stabilize long-term changes in neural circuits. These changes help the mouse to remember what has been learned and support the establishment of new behavioral responses. Epigenetic mechanisms that support instinct by operating on developmental time scales also support learning by operating on physiological time scales. Evolutionary changes in epigenetic mechanisms may sculpt a learned behavior into an instinct by decreasing its dependence on external stimuli in favor of an internally regulated program of neural development (see the figure).
There is evidence for such epigenetically driven evolutionary changes in behavior. For example, differences in innate aggression levels between races of honey bees can be attributed to evolutionary changes in brain gene expression that also control the onset of aggressive behavior when threatened. These kinds of changes can also explain more contemporary developments, including new innate aspects of mating and foraging behavior in house finches associated with their North American invasion 75 years ago, and new innate changes in the frequency and structure of song communication in populations of several bird species now living in urban environments. We propose that these new instincts have emerged through evolutionary genetic changes that acted on initially plastic behavioral responses.