Degrees of freedom in planning, running, analyzing, and reporting psychological studies

A checklist to avoid p-hacking

Research output: Contribution to journalArticleScientificpeer-review

76 Downloads (Pure)

Abstract

The designing, collecting, analyzing, and reporting of psychological studies entail many choices that are often arbitrary. The opportunistic use of these so-called researcher degrees of freedom aimed at obtaining statistically significant results is problematic because it enhances the chances of false positive results and may inflate effect size estimates. In this review article, we present an extensive list of 34 degrees of freedom that researchers have in formulating hypotheses, and in designing, running, analyzing, and reporting of psychological research. The list can be used in research methods education, and as a checklist to assess the quality of preregistrations and to determine the potential for bias due to (arbitrary) choices in unregistered studies.
Original languageEnglish
Article number1832
JournalFrontiers in Psychology
Volume7
DOIs
Publication statusPublished - 2016

Fingerprint

Checklist
Education

Cite this

@article{29a7a64f8ddc4df4b73011ba49c74726,
title = "Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking",
abstract = "The designing, collecting, analyzing, and reporting of psychological studies entail many choices that are often arbitrary. The opportunistic use of these so-called researcher degrees of freedom aimed at obtaining statistically significant results is problematic because it enhances the chances of false positive results and may inflate effect size estimates. In this review article, we present an extensive list of 34 degrees of freedom that researchers have in formulating hypotheses, and in designing, running, analyzing, and reporting of psychological research. The list can be used in research methods education, and as a checklist to assess the quality of preregistrations and to determine the potential for bias due to (arbitrary) choices in unregistered studies.",
author = "J.M. Wicherts and C.L.S. Veldkamp and H.E.M. Augusteijn and M. Bakker and {van Aert}, R.C.M. and {van Assen}, M.A.L.M.",
year = "2016",
doi = "10.3389/fpsyg.2016.01832",
language = "English",
volume = "7",
journal = "Frontiers in Psychology",
issn = "1664-1078",
publisher = "Frontiers Media S.A.",

}

TY - JOUR

T1 - Degrees of freedom in planning, running, analyzing, and reporting psychological studies

T2 - A checklist to avoid p-hacking

AU - Wicherts, J.M.

AU - Veldkamp, C.L.S.

AU - Augusteijn, H.E.M.

AU - Bakker, M.

AU - van Aert, R.C.M.

AU - van Assen, M.A.L.M.

PY - 2016

Y1 - 2016

N2 - The designing, collecting, analyzing, and reporting of psychological studies entail many choices that are often arbitrary. The opportunistic use of these so-called researcher degrees of freedom aimed at obtaining statistically significant results is problematic because it enhances the chances of false positive results and may inflate effect size estimates. In this review article, we present an extensive list of 34 degrees of freedom that researchers have in formulating hypotheses, and in designing, running, analyzing, and reporting of psychological research. The list can be used in research methods education, and as a checklist to assess the quality of preregistrations and to determine the potential for bias due to (arbitrary) choices in unregistered studies.

AB - The designing, collecting, analyzing, and reporting of psychological studies entail many choices that are often arbitrary. The opportunistic use of these so-called researcher degrees of freedom aimed at obtaining statistically significant results is problematic because it enhances the chances of false positive results and may inflate effect size estimates. In this review article, we present an extensive list of 34 degrees of freedom that researchers have in formulating hypotheses, and in designing, running, analyzing, and reporting of psychological research. The list can be used in research methods education, and as a checklist to assess the quality of preregistrations and to determine the potential for bias due to (arbitrary) choices in unregistered studies.

U2 - 10.3389/fpsyg.2016.01832

DO - 10.3389/fpsyg.2016.01832

M3 - Article

VL - 7

JO - Frontiers in Psychology

JF - Frontiers in Psychology

SN - 1664-1078

M1 - 1832

ER -