TY - JOUR
T1 - The pipeline project
T2 - Pre-publication independent replications of a single laboratory's research pipeline
AU - Schweinsberg, Martin
AU - Madan, Nikhil
AU - Vianello, Michelangelo
AU - Sommer, S. Amy
AU - Jordan, Jennifer
AU - Tierney, Warren
AU - Awtrey, Eli
AU - Zhu, Luke Lei
AU - Diermeier, Daniel
AU - Heinze, Justin E.
AU - Srinivasan, Malavika
AU - Tannenbaum, David
AU - Bivolaru, Eliza
AU - Dana, Jason
AU - Davis-Stober, Clintin P.
AU - du Plessis, Christilene
AU - Gronau, Quentin F.
AU - Hafenbrack, Andrew C.
AU - Liao, Eko Yi
AU - Ly, Alexander
AU - Marsman, Maarten
AU - Murase, Toshio
AU - Qureshi, Israr
AU - Schaerer, Michael
AU - Thornley, Nico
AU - Tworek, Christina M.
AU - Wagenmakers, Eric Jan
AU - Wong, Lynn
AU - Anderson, Tabitha
AU - Bauman, Christopher W.
AU - Bedwell, Wendy L.
AU - Brescoll, Victoria
AU - Canavan, Andrew
AU - Chandler, Jesse J.
AU - Cheries, Erik
AU - Cheryan, Sapna
AU - Cheung, Felix
AU - Cimpian, Andrei
AU - Clark, Mark A.
AU - Cordon, Diana
AU - Cushman, Fiery
AU - Ditto, Peter H.
AU - Donahue, Thomas
AU - Frick, Sarah E.
AU - Gamez-Djokic, Monica
AU - Grady, Rebecca Hofstein
AU - Graham, Jesse
AU - Gu, Jun
AU - Hahn, Adam
AU - Hanson, Brittany E.
AU - Hartwich, Nicole J.
AU - Hein, Kristie
AU - Inbar, Yoel
AU - Jiang, Lily
AU - Kellogg, Tehlyr
AU - Kennedy, Deanna M.
AU - Legate, Nicole
AU - Luoma, Timo P.
AU - Maibuecher, Heidi
AU - Meindl, Peter
AU - Miles, Jennifer
AU - Mislin, Alexandra
AU - Molden, Daniel C.
AU - Motyl, Matt
AU - Newman, George
AU - Ngo, Hoai Huong
AU - Packham, Harvey
AU - Ramsay, Philip S.
AU - Ray, Jennifer L.
AU - Sackett, Aaron M.
AU - Sellier, Anne Laure
AU - Sokolova, Tatiana
AU - Sowden, Walter
AU - Storage, Daniel
AU - Sun, Xiaomin
AU - Van Bavel, Jay J.
AU - Washburn, Anthony N.
AU - Wei, Cong
AU - Wetter, Erik
AU - Wilson, Carlos T.
AU - Darroux, Sophie Charlotte
AU - Uhlmann, Eric Luis
N1 - Funding Information:
The first, second, and last authors contributed equally to the project. Eric Luis Uhlmann designed the Pipeline Project and wrote the initial project proposal. Martin Schweinsberg, Nikhil Madan, Michelangelo Vianello, Amy Sommer, Jennifer Jordan, Warren Tierney, Eli Awtrey, and Luke (Lei) Zhu served as project coordinators. Daniel Diermeier, Justin E. Heinze, Malavika Srinivasan, David Tannenbaum, Eric Luis Uhlmann, and Luke Zhu contributed original studies for replication. Michelangelo Vianello, Jennifer Jordan, Amy Sommer, Eli Awtrey, Eliza Bivolaru, Jason Dana, Clintin P. Davis-Stober, Christilene du Plessis, Quentin F. Gronau, Andrew C. Hafenbrack, Eko Yi Liao, Alexander Ly, Maarten Marsman, Toshio Murase, Israr Qureshi, Michael Schaerer, Nico Thornley, Christina M. Tworek, Eric-Jan Wagenmakers, and Lynn Wong helped analyze the data. Eli Awtrey, Jennifer Jordan, Amy Sommer, Tabitha Anderson, Christopher W. Bauman, Wendy L. Bedwell, Victoria Brescoll, Andrew Canavan, Jesse J. Chandler, Erik Cheries, Sapna Cheryan, Felix Cheung, Andrei Cimpian, Mark A. Clark, Diana Cordon, Fiery Cushman, Peter Ditto, Thomas Donahue, Sarah E. Frick, Monica Gamez-Djokic, Rebecca Hofstein Grady, Jesse Graham, Jun Gu, Adam Hahn, Brittany E. Hanson, Nicole J. Hartwich, Kristie Hein, Yoel Inbar, Lily Jiang, Tehlyr Kellogg, Deanna M. Kennedy, Nicole Legate, Timo P. Luoma, Heidi Maibuecher, Peter Meindl, Jennifer Miles, Alexandra Mislin, Daniel Molden, Matt Motyl, George Newman, Hoai Huong Ngo, Harvey Packham, Philip S. Ramsay, Jennifer Lauren Ray, Aaron M. Sackett, Anne-Laure Sellier, Tatiana Sokolova, Walter Sowden, Daniel Storage, Xiaomin Sun, Christina M. Tworek, Jay J. van Bavel, Anthony N. Washburn, Cong Wei, Erik Wetter, and Carlos T. Wilson carried out the replications. Adam Hahn, Nicole Hartwich, Timo Luoma, Hoai Huong Ngo, and Sophie-Charlotte Darroux translated study materials from English into the local language. Eric Luis Uhlmann and Martin Schweinsberg wrote the first draft of the paper and numerous authors provided feedback, comments, and revisions. Correspondence concerning this paper should be addressed to Martin Schweinsberg, Boulevard de Constance, 77305 Fontainebleau, France, martin.schweinsberg@insead.edu , or to Eric Luis Uhlmann, INSEAD Organizational Behavior Area, 1 Ayer Rajah Avenue, 138676 Singapore, eric.uhlmann@insead.edu . The Pipeline Project was generously supported by an R&D grant from INSEAD.
Publisher Copyright:
© 2015 The Authors
PY - 2016/9/1
Y1 - 2016/9/1
N2 - This crowdsourced project introduces a collaborative approach to improving the reproducibility of scientific research, in which findings are replicated in qualified independent laboratories before (rather than after) they are published. Our goal is to establish a non-adversarial replication process with highly informative final results. To illustrate the Pre-Publication Independent Replication (PPIR) approach, 25 research groups conducted replications of all ten moral judgment effects which the last author and his collaborators had “in the pipeline” as of August 2014. Six findings replicated according to all replication criteria, one finding replicated but with a significantly smaller effect size than the original, one finding replicated consistently in the original culture but not outside of it, and two findings failed to find support. In total, 40% of the original findings failed at least one major replication criterion. Potential ways to implement and incentivize pre-publication independent replication on a large scale are discussed.
AB - This crowdsourced project introduces a collaborative approach to improving the reproducibility of scientific research, in which findings are replicated in qualified independent laboratories before (rather than after) they are published. Our goal is to establish a non-adversarial replication process with highly informative final results. To illustrate the Pre-Publication Independent Replication (PPIR) approach, 25 research groups conducted replications of all ten moral judgment effects which the last author and his collaborators had “in the pipeline” as of August 2014. Six findings replicated according to all replication criteria, one finding replicated but with a significantly smaller effect size than the original, one finding replicated consistently in the original culture but not outside of it, and two findings failed to find support. In total, 40% of the original findings failed at least one major replication criterion. Potential ways to implement and incentivize pre-publication independent replication on a large scale are discussed.
KW - Crowdsourcing science
KW - Meta-science
KW - Methodology
KW - Replication
KW - Reproducibility
KW - Research transparency
UR - http://www.scopus.com/inward/record.url?scp=84977144634&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84977144634&partnerID=8YFLogxK
U2 - 10.1016/j.jesp.2015.10.001
DO - 10.1016/j.jesp.2015.10.001
M3 - Article
AN - SCOPUS:84977144634
SN - 0022-1031
VL - 66
SP - 55
EP - 67
JO - Journal of Experimental Social Psychology
JF - Journal of Experimental Social Psychology
ER -