Wow...after having done several posts uncritically passing on studies by Jaeggi and others claiming that games to improve working memory, such as the n-Back game, increase cognitive skills in other areas, a number of studies have failed to replicate these phenomena. Gareth Cook has done an interesting article on this in The New Yorker, suggesting that claims made by commercial software sites like Cogmen, Lumosity, and CogniFit are bogus.
Over the last year, however, the idea that working-memory training has broad benefits has crumbled. One group of psychologists, lead by a team at Georgia Tech, set out to replicate the Jaeggi findings, but with more careful controls and seventeen different cognitive-skills tests. Their subjects showed no evidence whatsoever for improvement in intelligence. They also identified a pattern of methodological problems with experiments showing positive results, like poor controls and a reliance on a single measure of cognitive improvement. This failed replication was recently published in one of psychology’s top journals, and another, by a group at Case Western Reserve University, has been published since.
The recent meta-analysis, led by Monica Melby-Lervåg, of the University of Oslo, and also published in a top journal, is even more damning. Some studies are more convincing than others, because they include more subjects and show a larger effect. Melby-Lervåg’s paper laboriously accounts for this, incorporating what Jaeggi, Klingberg, and everyone else had reported. The meta-analysis found that the training isn’t doing anyone much good. If anything, the scientific literature tends to overstate effects, because teams that find nothing tend not to publish their papers. (This is known as the “filedrawer” effect.) A null result from meta-analysis, published in a top journal, sends a shudder through the spine of all but the truest of believers. In the meantime, a separate paper by some of the Georgia Tech scientists looked specifically at Cogmed’s training, which has been subjected to more scientific scrutiny than any other program. “The claims made by Cogmed,” they wrote, “are largely unsubstantiated.”