Self-policing by psychologists - many prominent experiments fail replication tests

Benedict Carey offers context, summary, and reactions to the recent Science article reporting work of a consortium of 270 scientists on five continents led by psychologist Brian Nosek at the University of Virginia. (Other commentaries on this work are offered by Bohannon and by The Guardian.) The bottom line is that only 36 percent of the findings from almost 100 studies in the top three psychology journals held up when the original experiments were rigorously redone. This work was an 'inside job,' with psychologists doing the replication attempts communicating with cooperative original researchers. Carey describes a culture change that is taking hold in the psychological research community, towards greater transparency, data sharing, and preregistration of studies with journals that spells out hypotheses and how they are going to be tested. "Doing this upfront is a powerful check against moving the goal posts on a study — that is, analyzing the data and working backward, reverse-engineering the “hypothesis” to fit those findings."

(addendum 9/1/2015:  another useful commentary, "Psychology is Not in Crisis")