Sailing From the seas of chaos Into the corridor of stability practical recommendations to increase the informational value of studies

D. Lakens, E.R.K. Evers

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Recent events have led psychologists to acknowledge that the inherent uncertainty encapsulated in an inductive science is amplified by problematic research practices. In this article, we provide a practical introduction to recently developed statistical tools that can be used to deal with these uncertainties when performing and evaluating research. In Part 1, we discuss the importance of accurate and stable effect size estimates as well as how to design studies to reach a corridor of stability around effect size estimates. In Part 2, we explain how, given uncertain effect size estimates, well-powered studies can be designed with sequential analyses. In Part 3, we (a) explain what p values convey about the likelihood that an effect is true, (b) illustrate how the v statistic can be used to evaluate the accuracy of individual studies, and (c) show how the evidential value of multiple studies can be examined with a p-curve analysis. We end by discussing the consequences of incorporating our recommendations in terms of a reduced quantity, but increased quality, of the research output. We hope that the practical recommendations discussed in this article will provide researchers with the tools to make important steps toward a psychological science that allows researchers to differentiate among all possible truths on the basis of their likelihood.
Keywords: induction, p-curve analysis, v statistic, confidence intervals, sequential analyses
Original languageEnglish
Pages (from-to)278-292
JournalPerspectives on Psychological Science
Volume9
Issue number3
DOIs
Publication statusPublished - 2014

Fingerprint

Confidence Intervals

Cite this

@article{a7b4835e6f22432db8ebe33072274066,
title = "Sailing From the seas of chaos Into the corridor of stability practical recommendations to increase the informational value of studies",
abstract = "Recent events have led psychologists to acknowledge that the inherent uncertainty encapsulated in an inductive science is amplified by problematic research practices. In this article, we provide a practical introduction to recently developed statistical tools that can be used to deal with these uncertainties when performing and evaluating research. In Part 1, we discuss the importance of accurate and stable effect size estimates as well as how to design studies to reach a corridor of stability around effect size estimates. In Part 2, we explain how, given uncertain effect size estimates, well-powered studies can be designed with sequential analyses. In Part 3, we (a) explain what p values convey about the likelihood that an effect is true, (b) illustrate how the v statistic can be used to evaluate the accuracy of individual studies, and (c) show how the evidential value of multiple studies can be examined with a p-curve analysis. We end by discussing the consequences of incorporating our recommendations in terms of a reduced quantity, but increased quality, of the research output. We hope that the practical recommendations discussed in this article will provide researchers with the tools to make important steps toward a psychological science that allows researchers to differentiate among all possible truths on the basis of their likelihood.Keywords: induction, p-curve analysis, v statistic, confidence intervals, sequential analyses",
author = "D. Lakens and E.R.K. Evers",
year = "2014",
doi = "10.1177/1745691614528520",
language = "English",
volume = "9",
pages = "278--292",
journal = "Perspectives on Psychological Science",
issn = "1745-6916",
publisher = "SAGE Publications Inc.",
number = "3",

}

Sailing From the seas of chaos Into the corridor of stability practical recommendations to increase the informational value of studies. / Lakens, D.; Evers, E.R.K.

In: Perspectives on Psychological Science, Vol. 9, No. 3, 2014, p. 278-292.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Sailing From the seas of chaos Into the corridor of stability practical recommendations to increase the informational value of studies

AU - Lakens, D.

AU - Evers, E.R.K.

PY - 2014

Y1 - 2014

N2 - Recent events have led psychologists to acknowledge that the inherent uncertainty encapsulated in an inductive science is amplified by problematic research practices. In this article, we provide a practical introduction to recently developed statistical tools that can be used to deal with these uncertainties when performing and evaluating research. In Part 1, we discuss the importance of accurate and stable effect size estimates as well as how to design studies to reach a corridor of stability around effect size estimates. In Part 2, we explain how, given uncertain effect size estimates, well-powered studies can be designed with sequential analyses. In Part 3, we (a) explain what p values convey about the likelihood that an effect is true, (b) illustrate how the v statistic can be used to evaluate the accuracy of individual studies, and (c) show how the evidential value of multiple studies can be examined with a p-curve analysis. We end by discussing the consequences of incorporating our recommendations in terms of a reduced quantity, but increased quality, of the research output. We hope that the practical recommendations discussed in this article will provide researchers with the tools to make important steps toward a psychological science that allows researchers to differentiate among all possible truths on the basis of their likelihood.Keywords: induction, p-curve analysis, v statistic, confidence intervals, sequential analyses

AB - Recent events have led psychologists to acknowledge that the inherent uncertainty encapsulated in an inductive science is amplified by problematic research practices. In this article, we provide a practical introduction to recently developed statistical tools that can be used to deal with these uncertainties when performing and evaluating research. In Part 1, we discuss the importance of accurate and stable effect size estimates as well as how to design studies to reach a corridor of stability around effect size estimates. In Part 2, we explain how, given uncertain effect size estimates, well-powered studies can be designed with sequential analyses. In Part 3, we (a) explain what p values convey about the likelihood that an effect is true, (b) illustrate how the v statistic can be used to evaluate the accuracy of individual studies, and (c) show how the evidential value of multiple studies can be examined with a p-curve analysis. We end by discussing the consequences of incorporating our recommendations in terms of a reduced quantity, but increased quality, of the research output. We hope that the practical recommendations discussed in this article will provide researchers with the tools to make important steps toward a psychological science that allows researchers to differentiate among all possible truths on the basis of their likelihood.Keywords: induction, p-curve analysis, v statistic, confidence intervals, sequential analyses

U2 - 10.1177/1745691614528520

DO - 10.1177/1745691614528520

M3 - Article

VL - 9

SP - 278

EP - 292

JO - Perspectives on Psychological Science

JF - Perspectives on Psychological Science

SN - 1745-6916

IS - 3

ER -