The (mis)reporting of statistical results in psychology journals

Research output: Contribution to journalArticleScientificpeer-review

Abstract

In order to study the prevalence, nature (direction), and causes of reporting errors in psychology, we checked the consistency of reported test statistics, degrees of freedom, and p values in a random sample of high- and low-impact psychology journals. In a second study, we established the generality of reporting errors in a random sample of recent psychological articles. Our results, on the basis of 281 articles, indicate that around 18% of statistical results in the psychological literature are incorrectly reported. Inconsistencies were more common in low-impact journals than in high-impact journals. Moreover, around 15% of the articles contained at least one statistical conclusion that proved, upon recalculation, to be incorrect; that is, recalculation rendered the previously significant result insignificant, or vice versa. These errors were often in line with researchers' expectations. We classified the most common errors and contacted authors to shed light on the origins of the errors.

Original languageEnglish
Pages (from-to)666-678
JournalBehavior Research Methods
Volume43
Issue number3
DOIs
Publication statusPublished - Sep 2011
Externally publishedYes

Fingerprint

Cross-Sectional Studies
Psychology
Nature
Psychological
Generality
Causes
Psychological Literature
Inconsistency
Statistics

Keywords

  • Humans
  • Periodicals as Topic
  • Psychology
  • Research Design
  • Statistics as Topic

Cite this

@article{9fe2651299894906aee71c4010393677,
title = "The (mis)reporting of statistical results in psychology journals",
abstract = "In order to study the prevalence, nature (direction), and causes of reporting errors in psychology, we checked the consistency of reported test statistics, degrees of freedom, and p values in a random sample of high- and low-impact psychology journals. In a second study, we established the generality of reporting errors in a random sample of recent psychological articles. Our results, on the basis of 281 articles, indicate that around 18{\%} of statistical results in the psychological literature are incorrectly reported. Inconsistencies were more common in low-impact journals than in high-impact journals. Moreover, around 15{\%} of the articles contained at least one statistical conclusion that proved, upon recalculation, to be incorrect; that is, recalculation rendered the previously significant result insignificant, or vice versa. These errors were often in line with researchers' expectations. We classified the most common errors and contacted authors to shed light on the origins of the errors.",
keywords = "Humans, Periodicals as Topic, Psychology, Research Design, Statistics as Topic",
author = "M. Bakker and J.M. Wicherts",
year = "2011",
month = "9",
doi = "10.3758/s13428-011-0089-5",
language = "English",
volume = "43",
pages = "666--678",
journal = "Behavior Research Methods",
issn = "1554-351X",
publisher = "Springer",
number = "3",

}

The (mis)reporting of statistical results in psychology journals. / Bakker, M.; Wicherts, J.M.

In: Behavior Research Methods, Vol. 43, No. 3, 09.2011, p. 666-678.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - The (mis)reporting of statistical results in psychology journals

AU - Bakker, M.

AU - Wicherts, J.M.

PY - 2011/9

Y1 - 2011/9

N2 - In order to study the prevalence, nature (direction), and causes of reporting errors in psychology, we checked the consistency of reported test statistics, degrees of freedom, and p values in a random sample of high- and low-impact psychology journals. In a second study, we established the generality of reporting errors in a random sample of recent psychological articles. Our results, on the basis of 281 articles, indicate that around 18% of statistical results in the psychological literature are incorrectly reported. Inconsistencies were more common in low-impact journals than in high-impact journals. Moreover, around 15% of the articles contained at least one statistical conclusion that proved, upon recalculation, to be incorrect; that is, recalculation rendered the previously significant result insignificant, or vice versa. These errors were often in line with researchers' expectations. We classified the most common errors and contacted authors to shed light on the origins of the errors.

AB - In order to study the prevalence, nature (direction), and causes of reporting errors in psychology, we checked the consistency of reported test statistics, degrees of freedom, and p values in a random sample of high- and low-impact psychology journals. In a second study, we established the generality of reporting errors in a random sample of recent psychological articles. Our results, on the basis of 281 articles, indicate that around 18% of statistical results in the psychological literature are incorrectly reported. Inconsistencies were more common in low-impact journals than in high-impact journals. Moreover, around 15% of the articles contained at least one statistical conclusion that proved, upon recalculation, to be incorrect; that is, recalculation rendered the previously significant result insignificant, or vice versa. These errors were often in line with researchers' expectations. We classified the most common errors and contacted authors to shed light on the origins of the errors.

KW - Humans

KW - Periodicals as Topic

KW - Psychology

KW - Research Design

KW - Statistics as Topic

U2 - 10.3758/s13428-011-0089-5

DO - 10.3758/s13428-011-0089-5

M3 - Article

VL - 43

SP - 666

EP - 678

JO - Behavior Research Methods

JF - Behavior Research Methods

SN - 1554-351X

IS - 3

ER -