Conducting meta-analyses based on p values: Reservations and recommendations for applying p-uniform and p-curve

Research output: Contribution to journalArticleScientificpeer-review

51 Downloads (Pure)

Abstract

Because of overwhelming evidence of publication bias in psychology, techniques to correct meta-analytic estimates for such bias are greatly needed. The methodology on which the p-uniform and p-curve methods are based has great promise for providing accurate meta-analytic estimates in the presence of publication bias. However, in this article, we show that in some situations, p-curve behaves erratically, whereas p-uniform may yield implausible estimates of negative effect size. Moreover, we show that (and explain why) p-curve and p-uniform result in overestimation of effect size under moderate-to-large heterogeneity and may yield unpredictable bias when researchers employ p-hacking. We offer hands-on recommendations on applying and interpreting results of meta-analyses in general and p-uniform and p-curve in particular. Both methods as well as traditional methods are applied to a meta-analysis on the effect of weight on judgments of importance. We offer guidance for applying p-uniform or p-curve using R and a user-friendly web application for applying p-uniform.
Original languageEnglish
Pages (from-to)713-729
JournalPerspectives on Psychological Science
Volume11
Issue number5
DOIs
Publication statusPublished - 1 Sep 2016

Fingerprint

Publication Bias
Weights and Measures

Cite this

@article{0c407795b1b740a3906dfb82e7419a7b,
title = "Conducting meta-analyses based on p values: Reservations and recommendations for applying p-uniform and p-curve",
abstract = "Because of overwhelming evidence of publication bias in psychology, techniques to correct meta-analytic estimates for such bias are greatly needed. The methodology on which the p-uniform and p-curve methods are based has great promise for providing accurate meta-analytic estimates in the presence of publication bias. However, in this article, we show that in some situations, p-curve behaves erratically, whereas p-uniform may yield implausible estimates of negative effect size. Moreover, we show that (and explain why) p-curve and p-uniform result in overestimation of effect size under moderate-to-large heterogeneity and may yield unpredictable bias when researchers employ p-hacking. We offer hands-on recommendations on applying and interpreting results of meta-analyses in general and p-uniform and p-curve in particular. Both methods as well as traditional methods are applied to a meta-analysis on the effect of weight on judgments of importance. We offer guidance for applying p-uniform or p-curve using R and a user-friendly web application for applying p-uniform.",
author = "{Van Aert}, R.C.M. and J.M. Wicherts and {Van Assen}, M.A.L.M.",
year = "2016",
month = "9",
day = "1",
doi = "10.1177/1745691616650874",
language = "English",
volume = "11",
pages = "713--729",
journal = "Perspectives on Psychological Science",
issn = "1745-6916",
publisher = "SAGE Publications Inc.",
number = "5",

}

TY - JOUR

T1 - Conducting meta-analyses based on p values

T2 - Reservations and recommendations for applying p-uniform and p-curve

AU - Van Aert, R.C.M.

AU - Wicherts, J.M.

AU - Van Assen, M.A.L.M.

PY - 2016/9/1

Y1 - 2016/9/1

N2 - Because of overwhelming evidence of publication bias in psychology, techniques to correct meta-analytic estimates for such bias are greatly needed. The methodology on which the p-uniform and p-curve methods are based has great promise for providing accurate meta-analytic estimates in the presence of publication bias. However, in this article, we show that in some situations, p-curve behaves erratically, whereas p-uniform may yield implausible estimates of negative effect size. Moreover, we show that (and explain why) p-curve and p-uniform result in overestimation of effect size under moderate-to-large heterogeneity and may yield unpredictable bias when researchers employ p-hacking. We offer hands-on recommendations on applying and interpreting results of meta-analyses in general and p-uniform and p-curve in particular. Both methods as well as traditional methods are applied to a meta-analysis on the effect of weight on judgments of importance. We offer guidance for applying p-uniform or p-curve using R and a user-friendly web application for applying p-uniform.

AB - Because of overwhelming evidence of publication bias in psychology, techniques to correct meta-analytic estimates for such bias are greatly needed. The methodology on which the p-uniform and p-curve methods are based has great promise for providing accurate meta-analytic estimates in the presence of publication bias. However, in this article, we show that in some situations, p-curve behaves erratically, whereas p-uniform may yield implausible estimates of negative effect size. Moreover, we show that (and explain why) p-curve and p-uniform result in overestimation of effect size under moderate-to-large heterogeneity and may yield unpredictable bias when researchers employ p-hacking. We offer hands-on recommendations on applying and interpreting results of meta-analyses in general and p-uniform and p-curve in particular. Both methods as well as traditional methods are applied to a meta-analysis on the effect of weight on judgments of importance. We offer guidance for applying p-uniform or p-curve using R and a user-friendly web application for applying p-uniform.

U2 - 10.1177/1745691616650874

DO - 10.1177/1745691616650874

M3 - Article

VL - 11

SP - 713

EP - 729

JO - Perspectives on Psychological Science

JF - Perspectives on Psychological Science

SN - 1745-6916

IS - 5

ER -