Remembering past rewards - anterior cingulate cortex needed

Here is the abstract from an interesting paper in Nature Neuroscience by Kennerley et al.

"Learning the value of options in an uncertain environment is central to optimal decision making. The anterior cingulate cortex (ACC) has been implicated in using reinforcement information to control behavior. Here we demonstrate that the ACC's critical role in reinforcement-guided behavior is neither in detecting nor in correcting errors, but in guiding voluntary choices based on the history of actions and outcomes. ACC lesions did not impair the performance of monkeys (Macaca mulatta) immediately after errors, but made them unable to sustain rewarded responses in a reinforcement-guided choice task and to integrate risk and payoff in a dynamic foraging task. These data suggest that the ACC is essential for learning the value of actions."

Here is a graphic (credit: Ann Thomson, Nature Neuroscience) in an accompanying review of this work by Haydn and Platt.

Monkeys with lesions in the anterior cingulate cortex (ACC) correct errors but fail to maintain the new rewarded behavior. (a) Medial view of the right hemisphere in the macaque. Shading, location of the lesion in the ACC. (b) Behavioral responses of monkeys with ACC lesions in experiment 1. After one action—in this case, lift—was rewarded for 25 consecutive trials, the rewarded action was switched—in this case, to turn. Following an unrewarded lift response, lesioned monkeys switched to turning, but could not sustain this response on subsequent trials. Intact monkeys, however, had no difficulty with this task. Similar lesion-induced disruptions in behavioral performance were found in a reward probability matching task in a second experiment.

The Middle East - contribution of basic cognitive errors to cycles of retribution.

Daniel Gilbert, whose book (Stumbling on Happiness) I abstracted in a series of posts on 6/29/06, has a fascinating Op-Ed piece in today's New York Times that seems particularly relevant to the current maelstrom of violence in the middle east. Here are a few clips from his writing:

"He hit me first" provides an acceptable rationale for doing that which is otherwise forbidden. Both civil and religious law provide long lists of behaviors that are illegal or immoral — unless they are responses in kind, in which case they are perfectly fine...a punch thrown second is legally and morally different than a punch thrown first....That’s why participants in every one of the globe’s intractable conflicts — from Ireland to the Middle East — offer the even-numberedness of their punches as grounds for exculpation.

The problem with the principle of even-numberedness is that people count differently...In a study conducted by William Swann and colleagues at the University of Texas, pairs of volunteers played the roles of world leaders who were trying to decide whether to initiate a nuclear strike. The first volunteer was asked to make an opening statement, the second volunteer was asked to respond, the first volunteer was asked to respond to the second, and so on. At the end of the conversation, the volunteers were shown several of the statements that had been made and were asked to recall what had been said just before and just after each of them...The results revealed an intriguing asymmetry: When volunteers were shown one of their own statements, they naturally remembered what had led them to say it. But when they were shown one of their conversation partner’s statements, they naturally remembered how they had responded to it...

What seems like a grossly self-serving pattern of remembering is actually the product of two innocent facts. First, because our senses point outward, we can observe other people’s actions but not our own. Second, because mental life is a private affair, we can observe our own thoughts but not the thoughts of others. Together, these facts suggest that our reasons for punching will always be more salient to us than the punches themselves — but that the opposite will be true of other people’s reasons and other people’s punches...Examples aren’t hard to come by. Shiites seek revenge on Sunnis for the revenge they sought on Shiites; Irish Catholics retaliate against the Protestants who retaliated against them; and since 1948, it’s hard to think of any partisan in the Middle East who has done anything but play defense. In each of these instances, people on one side claim that they are merely responding to provocation and dismiss the other side’s identical claim as disingenuous spin. But research suggests that these claims reflect genuinely different perceptions of the same bloody conversation.

If the first principle of legitimate punching is that punches must be even-numbered, the second principle is that an even-numbered punch may be no more forceful than the odd-numbered punch that preceded it. Legitimate retribution is meant to restore balance, and thus an eye for an eye is fair, but an eye for an eyelash is not. When the European Union condemned Israel for bombing Lebanon in retaliation for the kidnapping of two Israeli soldiers, it did not question Israel’s right to respond, but rather, its “disproportionate use of force.” It is O.K. to hit back, just not too hard.

Research shows that people have as much trouble applying the second principle as the first. In a study conducted by Sukhwinder Shergill and colleagues at University College London, pairs of volunteers were hooked up to a mechanical device that allowed each of them to exert pressure on the other volunteer’s fingers.

The researcher began the game by exerting a fixed amount of pressure on the first volunteer’s finger. The first volunteer was then asked to exert precisely the same amount of pressure on the second volunteer’s finger. The second volunteer was then asked to exert the same amount of pressure on the first volunteer’s finger. And so on. The two volunteers took turns applying equal amounts of pressure to each other’s fingers while the researchers measured the actual amount of pressure they applied.

The results were striking. Although volunteers tried to respond to each other’s touches with equal force, they typically responded with about 40 percent more force than they had just experienced. Each time a volunteer was touched, he touched back harder, which led the other volunteer to touch back even harder. What began as a game of soft touches quickly became a game of moderate pokes and then hard prods, even though both volunteers were doing their level best to respond in kind.

Each volunteer was convinced that he was responding with equal force and that for some reason the other volunteer was escalating. Neither realized that the escalation was the natural byproduct of a neurological quirk that causes the pain we receive to seem more painful than the pain we produce, so we usually give more pain than we have received.

Research teaches us that our reasons and our pains are more palpable, more obvious and real, than are the reasons and pains of others. This leads to the escalation of mutual harm, to the illusion that others are solely responsible for it and to the belief that our actions are justifiable responses to theirs.

None of this is to deny the roles that hatred, intolerance, avarice and deceit play in human conflict. It is simply to say that basic principles of human psychology are important ingredients in this miserable stew. Until we learn to stop trusting everything our brains tell us about others — and to start trusting others themselves — there will continue to be tears and recriminations in the wayback.

A ‘Senior Moment’ or a Self-Fulfilling Prophecy?

This is the title of an article by Benedict Carey in last Tuesday's NYTimes science section (7/18/06) pointing to work reported in the current issue of the journal Social Cognition. Men and women in late middle age (48-62) underperformed on a standard memory test when told they were part of a study including people over age 70. Inclusion with an older group — an indirect reminder of the link between age and memory slippage — was enough to affect their scores, especially for those who were most concerned about getting older. Scores were higher when participants were told they were competing with a group in their 20's. The findings “show how negative images of aging on TV, in other media and in jokes reinforce negative stereotypes that can affect performance even before people reach retirement age."

This self-undermining is a stereotype effect, of the sort that has been documented it in many groups. Other studies have shown that women perform less well on math exams after reading that men tend to perform better on them. Similarly, white men perform less well when they are told that they are competing in math against Asian students.

Chimp playing pacman

This video is spreading like a virus, I'm wanting to pass it on, having seen it on the Cognitive Daily Blog.

The Locus Ceruleus Is Involved in the Retrieval of Emotional Memories in Humans

Emotional events are usually remembered better than neutral ones, and the amygdala is involved in this enhancement. The level of the amygdala's response during encoding of either emotional events or neutral stimuli encoded in an emotional contex is related to the ease of subsequent recall. Retrieving such memories is accompanied by significant amygdalar responses. Sterpenich et. al. have now shown that successful recognition of stimuli encoded in emotional context is accompanied by a significant response not only in the amygdala and parahippocampal gyrus but also in central arousing structures of the brainstem and the diencephalon (centered on the locus ceruleus). They found that locus ceruleus responses at retrieval, as a constituent element of the emotional memory, could be related to the arousal induced during encoding.

A Right amygdala. C Parahippocampus gyrus

A. Locus ceruleus.
Credit: J. Neuroscience

Attention Deficit Hyperactivity Disorder, how Ritalin might be working.

Ruth Williams, in Nature Reviews Neuroscience, points out an article by Drouin et al in The Journal of Neurophysiology that suggests a potential brain mechanism for the action of methylphenidate (Ritalin) in relieving attention deficit hyperactivity disorder (ADHD). ADHD sufferers essentially "pay attention to too many things." They find it difficult to filter out unwanted distractions and focus one issue at a time. Behavioral experiments with rats have shown that Ritalin improves behavior in a sustained attention task, just as it does in humans. Drouin et al found that administering Ritalin to rats (at a dose equivalent to that used in human therapy) caused an increase in the levels of the neurotransmitter noradrenaline in the rat's sensory cortex and reduced a long-latency phase of the brain's response to sensory stimuli. This latter effect may be important in filtering out sensory noise.

Free Will, Free Won't, or Neither? A refinement of Libet's work on the conscious control of spontaneous actions.

In a famous paper published in 1983, Libet et al. showed that the recordable cerebral activity (readiness-potential) that precedes a freely voluntary motor act occurs at least several hundred milliseconds before the reported time of conscious intention to act. The actual movement occurs 200-250 msec after the reported time of intention to act. The data are pretty spooky when you think about it. They say that your brain ("it") has started working on a action well before "you" think you are initiating it! This article has sparked a continuing debate on whether we actually have free will. Libet has suggested that the ~200-250 msec period between awareness of intention and the actual action was sufficient to permit a "veto" of the action if it was judged inappropriate. In this interpretation, we might be said to be "free won't" rather than "free will".

Lau et al. have now done a more nuanced version of LIbet's experiments. In a previous paper they showed that, when participants were required to estimate the onset of their intentions using Libet's procedure, the activity in the presupplementary motor area (pre-SMA) was enhanced ~228 msec before motor execution. In their most recent work they show that when participants were required to estimate the onset of their motor executions (instead of their intentions), the activity in the cingulate motor area was enhanced. This latter condition, judged to be more natural and have less task-demanding instructions. The perceived onset of intention could be as late as ~120 msec before the motor execution . "Together, these findings raise the question of whether the conscious control of spontaneous action can be done within a much shorter time window than we had expected, or whether, as suggested by Wegner (The illusion of conscious will Cambridge, MA: MIT Press, 2002), our impression of conscious control is simply illusory."

I think Wegner has it right. His book, and his interpretation of our sense of agency as an after the fact ' emotion of authorship' is a must-read for anyone interested in issues of conscious volition.

The Dali Lama and evolutionary science. He’s an awesome guy, but……

The Dali Lama deserves great credit for his efforts to integrate the insights of modern science and spiritual traditions, and he deals with this in his recent book, “The Universe in a Single Atom.” However, I really don’t think some of his critiques of the evolutionary theory and the limitations of science and materialism get it quite right....

Take for example, chapter 5, ''Evolution, Karma, and the world of sentience' While it is quite extraordinary that someone in his position has learned so much, it is also not surprising that he seems to be unaware of work that counters many of his perceived shortcomings of "Darwinian evolutionary theory."

pg 104 "that mutations..take place naturally is unproblematic..that they are purely random strikes me as unsatisfying. It leaves open the question of whether this randomness is best understood as an objective feature of reality or better understood as indicating some kind of hidden causality."

This doesn't compute for me. Ionizing cosmic radiation hitting a nucleotide and altering its replication is random, as are a number of low frequency errors made by enzymic processes involved in replication. We at least have a handle on what we mean by random. There are countless examples of how statistically small random changes can lead to complex results (like eyes, or different kinds of hormone and neurotransmitter receptors). "Hidden causality' , on the other hand, is a complete deus ex machina, or wild card, with no presently known means of evaluation within a materialistic scientific world view. If something pops up, great, but until then......

pg 104, "For modern science, at least from a philosophical point of view, the critical divide seems to be between inanimate matter and the origin of living organisms, while for Buddhism the critical divide is between non-sentient matter and the emergence of sentient beings." Aren't we talking about apples and oranges? I'm not understanding the usefulness of trying so hard to unify things, as (on pg. 111) "On the whole, I think the Darwinian theory...gives a fairly coherent account of the evolution of human life on earth. At the same time, I believe that karma can have a central role in understanding the origination of Buddhism calls "sentience", through the media of energy and consciousness."

pg. 115 "I find it [Darwinian account] leaves one crucial area unexamined, the origin of sentience." Virtually all descriptions of the evolution of the nervous system (Dennett's in "Consciousness Explained", for example) view the increasingly complexity of the nervous system - and the gradient of increasing sentience, consciousness, or whatever - as Darwinian adaptations that increase survival fitness of the organism.

pg. 114 "I feel that this inability or unwillingness fully to engage the question of altruism is perhaps the most important drawback of Darwinian...." There are now abundant models of how 'selfish' genes and organisms can generate altruistic behaviors. There is even a recent computer model of simple automata (agents with a limited set of receptors and elementary actions) that evolve various cooperative strategies. (Nature, 20:1041, 2006). See also my 5/26/06 post on "Cooperation, Punishment, and the Evolution of Human Institutions"

Anyway, enough. I won't ramble on. He really is an extraordinary guy. I totally support the idea that buddhist psychological insight offers some correlates with modern cognitive neuroscience, as between Buddha's foundations of mindfulness and steps in the evolution of the brain (mentioned in my "Beast Within" essay at dericbownds.net).

Genes, cognition, and dyslexia.

Learning to read and write places unusual demands on the brain: explicit awareness of the structural elements of language and their relation to arbitrary visual symbols, rapid temporal processing, fine motor control, and visual acuity. Because sophisticated reading and writing systems appeared only a few thousand years ago, it is very unlikely that reading skills were shaped directly by Darwinian selection. Spoken or sign language, on the other hand, is acquired virtually effortlessly during the first few years of life, and is supported by brain specializations that evolved over hundreds of thousands of years.

It is interesting that studies on several European, Canadian, and American families have found genetic changes that correlate with reading disorders, or dyslexia, independent of general cognitive performance. Fisher and Francks, in Trends in Cognitive Sciences, 10:250 (2006), provide an overview of four prominent examples (genes DYX1C1, KIAA0319, DCDC2, and ROBO1). These are not "genes for reading". None are specific to reading-related neuronal circuits, or even to the human brain. They do have intriguing roles in neuronal migration or connectivity. Individual genes do not specify behavioral outputs, cognitive skill, or even particular neural circuits. They "influence brain development and function interactively by affecting processes such as proliferation and migration of neurons, programmed cell-death, axonal pathfinding, connectivity, levels of neurotransmitters/receptors, and so on."

Future and previous goals represented by different neuronal population the prefrontal cortex.

Genovesio et al have measured the activities of a large number of individual prefrontal cortical neurons as monkeys selected a future goal on the basis of events from a previous trial involving visual cues. They found that prefrontal neurons had activity that reflected either previous goals or future goals, but only rarely did individual cells reflect both. Their comment: "This finding suggests that essentially separate neural networks encode these two aspects of spatial information processing. A failure to distinguish previous and future goals could lead to two kinds of maladaptive behavior. First, wrongly representing an accomplished goal as still pending could cause perseveration or compulsive checking, two disorders commonly attributed to dysfunction of the prefrontal cortex. Second, mistaking a pending goal as already accomplished could cause the failures of omission that occur commonly in dementia."

Stoking the Voters' Passions

This is the title of a review in Science Magazine by James Druckman of Ted Brader's recent book "Campaigning for Hearts and Minds - How Emotional Appeals in Political Ads Work." (Univ. of Chicago Press, 2006). The main focus is on television ads. Brader extends the work of others in experiments that show that ads that generate enthusiasm will increase political interest, participation, and confidence, whereas fear-provoking ads will cause people to reevaluate their preferences and potentially change their opinions. His experiments manipulated emotional stimulation entirely by including or excluding certain images and music, having nothing to do with the ads' contents.


CREDIT: COURTESY TED BRADER, graphic from Science Magazine. Cueing emotion: In Brader's enthusiasm experiment, one ad contained hopeful images and music while the other relied on the narration and less evocative imagery. (Both ads used the same positive script, here: "There's good news in your neighborhood. The future looks bright for a generation of young people.") The fear experiment compared the effects of the same unevocative imagery with those of threatening images and dissonant music. (In the negative narration for this pair, "It's happening right now in your neighborhood. A generation of young people is in danger.")

Fraternal birth-order effect on male sexual orientation.

For each additional brother that precedes him, a boy's chance of growing up to be gay increases by a third. A fascinating article and commentary in the current issue of Proc. Nat. Acad. Sci. now make the point "that the social influence of an older brother is irrelevant to whether his younger brother will develop a homosexual orientation. It is the number of older biological brothers the mother carried, not the presence of older brothers while growing up, that makes some boys grow up to be gay. Older stepbrothers in the home have no effect, although older biological brothers raised apart still exert their influence. These data, by elimination, strengthen the notion that the common denominator between biological brothers, the mother, provides a prenatal environment that fosters homosexuality in her younger sons."

One explanation that has been proposed is a maternal immunization hypothesis. "When a mother is carrying her first son, the placental barrier protects each from exposure to the other’s proteins. But inevitable mixing of blood upon delivery will expose the mother for the first time to male-specific proteins..., including those encoded on the Y chromosome. If her immune system produces antibodies to these proteins, then the placenta may actively transport those antibodies (indeed, all IgGs) to subsequent offspring in utero, potentially affecting development of later-born sons, but not later-born daughters."

Brain-machine interfaces

Check out this special WEB FOCUS presented by Nature Magazine. A new study in Nature demonstrates a human with spinal injury manipulating a computer, a television, a screen cursor, and robotic devices by thought alone. Implanted electrodes in his motor cortex recorded neural activity, and translated it into movement commands.

The Deepest Cut - removing one cerebral hemisphere

The right and left hemispheres of our brains have a long list of distinctive specializations, one example being the centering of linguistic functions in the left hemisphere and emotional processing in the right. It is an amazing testament to plasticity of our brains that an entire hemisphere can be removed in children (in a last-ditch effort to stop intractable epilepsy), with the remaining hemisphere learning to take on some functions (such as language) previously handled by the now-missing partner. Several such surgeries (one on an eight year old girl) are described in an interesting article, with the title of this post, by Christine Kenneally in the July 3 issue of The New Yorker magazine.

Mice feel each others' pain.

Recent experiments by Langford et al provide further evidence that simple mammals show 'emotional contagion', a simple precursor to human empathy. Such behavior had previously been clearly documented in more advanced mammals such as chimpanzees, dolphins, and elephants. The experiments demonstrated that a pair of mice injected in view of each other with a chemical solution (acetic acid) that caused a 30-minute stomach ache suffered significantly more distress when they saw a familiar mouse suffering than when they saw the same kind of pain in a stranger. The observer also became more sensitive to other kinds of pain. Thus social manipulation of pain sensitizes the whole pain system.

Credit: Science Magazine and Mona Lisa Chanda
When two mice had their paws injected with different amounts of formalin, which would induce different levels of licking behavior, the high formalin mouse licked less and the low formalin animal licked more, indicating that their behavior was influenced by their neighbor's status bidirectionally. Emotional contagion is an automatic primitive kind of empathy (human babies cry on hearing other babies cry, regardless of cause), it does not require understanding what others are experiencing, and is distinct from altruism. So far there is no evidence for these more advanced behaviors in mice.

Lapses in Attention.....

We frequently make errors during lapses in our attention, and Weissman et al have now looked at what is happening in the brain during this process. They determined that attentional lapses begin with reduced activity in anterior cingulate and right prefrontal regions that act in a top-down fashion to bias posterior sensory regions in favor of processing task relevant information. This reduced activation occurred before the stimuli used to test attention were presented, and was accompanied by decreasing sensory processing in the inferior occipital cortex. These prefrontal regions seem to modulate how well focused people are in the moment just before they have to perform. The same prefrontal areas implicated in focusing attention before stimulus presentation had greater activation during stimulation following a lapse.

Credit: Nature Neuroscience. The figure, from an accompanying review by Hedden and Gabrieli indicates some of the cortical regions associated with focused and lapsed attention. The arrows indicate reciprocal functional connections between prefrontal and parietal regions, and top-down modulation of occipital sensory regions by the prefrontal cortex. IFG, right inferior frontal gyrus; TPJ, temporal-parietal junction.

Lie Detection by MRI spooks ethicists

Helen Pearson writes a good summary of issues and current work on lie detection that catches brains in the act of deception, unlike conventional methods that rely on stress-induced changes in pulse rates or skin conductance. Particular areas of the prefrontal cortex thought to be involved in detecting errors and inhibiting responses become more active during lying. Several groups claim 90% certainty in detecting when individuals are lying in a laboratory setting. There is controversy over statistical methods used in the experiments, and there are no data about whether the technique can be beaten by countermeasures, as the more conventional measures can. Many people who do brain imaging think that claims for the technique may crash and burn as more work is done. There is sentiment for a regulatory scheme that would prevent the use of MRI for lie detection until there was sufficient evidence to conclude that it was proven safe and effective - much as the US Food and Drug Administration bars or approves a drug. There is something deeply unsettling and intrusive about peering into someone's brain, essentially 'mind reading' even before the subject can make a response.

Social Animals Prove their Smarts

This is the title of an interesting brief review of animal intelligence by Pennisi in the June 23 issue of Science Magazine. It points out the remarkable range of behaviors once thought unique to humans but now observed in social animals. The reigning paradigm now is that social living fostered the evolution of intelligence. Crows deceive each other, as do apes; hyenas keep track of social hierarchies. There is no doubt that animals can attribute intentions and motives to others. This ability can evolve quickly. Foxes bred to for 45 years to be comfortable with humans can understand human gestures (like pointing to food). Untamed foxes can not, even after extensive training efforts. What still has not been convincingly shown for any animal is the ability to understand that another individual is thinking something wrong. Human children develop this ability by the age of four.



The roots of social intelligence. Fellowship. Foxes bred to be tame are keenly tuned in to human behavior. From Science Magazine, Credit: Irene Plyusnia, Photo Courtesy Of Brian Hare.

Brain basis of risk-taking behavior in adolescents.

Mesolimbic regions of the brain, particularly the nucleus accumbens are involved in reward, risk taking, and addictive behaviors. Galvan et al have looked at the development of this system by making behavioral and fMRI brain imaging measurements in adults and children in response to tasks manipulating reward values. Accumbens activity in adolescents looked like that of adults in both extent of activity and sensitivity to reward values, although the magnitude of activity was exaggerated. In contrast, the extent of orbital frontal cortex activity in adolescents looked more like that of children than adults, with less focal patterns of activity. Their findings suggest that the accumbens becomes disproportionately activated relative to later maturing top–down control systems in the orbital frontal cortex, biasing the adolescent's action toward immediate over long-term gains.


Localization of nucleus accumbens (A) and orbital frontal cortex (C) activation to reward.
Credit: Journal of Neuroscience

Trust in Fish

A mutually beneficial interaction between two species of fish turns out to involve the careful appraisal of one by the other — and the appropriately virtuous behaviour by the former while being watched. This is yet another example of a complex social behavior once thought to be unique to mammals.


Credit: Nature Magazine

The small (wrasse) fish is a cleaner of its client (bream) fish. It removes parasites, but actually prefers eating the clients mucus, which is not in the client's best inrest (i.e. is non-cooperative). Bshary and Grutter found that eavesdropping clients (who observed cleaners working on other clients) spent more time next to 'cooperative' than 'unknown cooperative level' cleaners, which shows that clients engage in image-scoring behaviour. Furthermore, trained cleaners learned to feed more cooperatively when in an 'image-scoring' than in a 'non-image-scoring' situation.