The prevalence of marginally significant results in psychology over time

Anton Olsson-Collentine*, Marcel A. L. M. Van Assen, Chris H. J. Hartgerink

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

Abstract

We examined the percentage of p values (.05 < p ≤ .10) reported as marginally significant in 44,200 articles, across nine psychology disciplines, published in 70 journals belonging to the American Psychological Association between 1985 and 2016. Using regular expressions, we extracted 42,504 p values between .05 and .10. Almost 40% of p values in this range were reported as marginally significant, although there were considerable differences between disciplines. The practice is most common in organizational psychology (45.4%) and least common in clinical psychology (30.1%). Contrary to what was reported by previous researchers, our results showed no evidence of an increasing trend in any discipline; in all disciplines, the percentage of p values reported as marginally significant was decreasing or constant over time. We recommend against reporting these results as marginally significant because of the low evidential value of p values between .05 and .10.
Original languageEnglish
Pages (from-to)576-586
JournalPsychological Science
Volume30
Issue number4
DOIs
Publication statusPublished - 2019

Keywords

  • APA
  • marginal significance
  • null-hypothesis significance testing
  • open data
  • open materials
  • over time
  • p values

Cite this

@article{c6d9d82ac56c4e6facb425d5550accc1,
title = "The prevalence of marginally significant results in psychology over time",
abstract = "We examined the percentage of p values (.05 < p ≤ .10) reported as marginally significant in 44,200 articles, across nine psychology disciplines, published in 70 journals belonging to the American Psychological Association between 1985 and 2016. Using regular expressions, we extracted 42,504 p values between .05 and .10. Almost 40{\%} of p values in this range were reported as marginally significant, although there were considerable differences between disciplines. The practice is most common in organizational psychology (45.4{\%}) and least common in clinical psychology (30.1{\%}). Contrary to what was reported by previous researchers, our results showed no evidence of an increasing trend in any discipline; in all disciplines, the percentage of p values reported as marginally significant was decreasing or constant over time. We recommend against reporting these results as marginally significant because of the low evidential value of p values between .05 and .10.",
keywords = "APA, marginal significance, null-hypothesis significance testing, open data, open materials, over time, p values",
author = "Anton Olsson-Collentine and {Van Assen}, {Marcel A. L. M.} and Hartgerink, {Chris H. J.}",
year = "2019",
doi = "10.1177/0956797619830326",
language = "English",
volume = "30",
pages = "576--586",
journal = "Psychological Science",
issn = "0956-7976",
publisher = "Sage Publications, Inc.",
number = "4",

}

The prevalence of marginally significant results in psychology over time. / Olsson-Collentine, Anton; Van Assen, Marcel A. L. M.; Hartgerink, Chris H. J.

In: Psychological Science, Vol. 30, No. 4, 2019, p. 576-586.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - The prevalence of marginally significant results in psychology over time

AU - Olsson-Collentine, Anton

AU - Van Assen, Marcel A. L. M.

AU - Hartgerink, Chris H. J.

PY - 2019

Y1 - 2019

N2 - We examined the percentage of p values (.05 < p ≤ .10) reported as marginally significant in 44,200 articles, across nine psychology disciplines, published in 70 journals belonging to the American Psychological Association between 1985 and 2016. Using regular expressions, we extracted 42,504 p values between .05 and .10. Almost 40% of p values in this range were reported as marginally significant, although there were considerable differences between disciplines. The practice is most common in organizational psychology (45.4%) and least common in clinical psychology (30.1%). Contrary to what was reported by previous researchers, our results showed no evidence of an increasing trend in any discipline; in all disciplines, the percentage of p values reported as marginally significant was decreasing or constant over time. We recommend against reporting these results as marginally significant because of the low evidential value of p values between .05 and .10.

AB - We examined the percentage of p values (.05 < p ≤ .10) reported as marginally significant in 44,200 articles, across nine psychology disciplines, published in 70 journals belonging to the American Psychological Association between 1985 and 2016. Using regular expressions, we extracted 42,504 p values between .05 and .10. Almost 40% of p values in this range were reported as marginally significant, although there were considerable differences between disciplines. The practice is most common in organizational psychology (45.4%) and least common in clinical psychology (30.1%). Contrary to what was reported by previous researchers, our results showed no evidence of an increasing trend in any discipline; in all disciplines, the percentage of p values reported as marginally significant was decreasing or constant over time. We recommend against reporting these results as marginally significant because of the low evidential value of p values between .05 and .10.

KW - APA

KW - marginal significance

KW - null-hypothesis significance testing

KW - open data

KW - open materials

KW - over time

KW - p values

U2 - 10.1177/0956797619830326

DO - 10.1177/0956797619830326

M3 - Article

VL - 30

SP - 576

EP - 586

JO - Psychological Science

JF - Psychological Science

SN - 0956-7976

IS - 4

ER -