Journal data sharing policies and statistical reporting inconsistencies in psychology

M.B. Nuijten, J. Borghuis, C.L.S. Veldkamp, L. Dominguez Alvarez, M.A.L.M. van Assen, J.M. Wicherts

Research output: Contribution to journalArticleScientificpeer-review

61 Downloads (Pure)

Abstract

In this paper, we present three retrospective observational studies that investigate the relation between data sharing and statistical reporting inconsistencies. Previous research found that reluctance to share data was related to a higher prevalence of statistical errors, often in the direction of statistical significance (Wicherts, Bakker, & Molenaar, 2011). We therefore hypothesized that journal policies about data sharing and data sharing itself would reduce these inconsistencies. In Study 1, we compared the prevalence of reporting inconsistencies in two similar journals on decision making with different data sharing policies. In Study 2, we compared reporting inconsistencies in psychology articles published in PLOS journals (with a data sharing policy) and Frontiers in Psychology (without a stipulated data sharing policy). In Study 3, we looked at papers published in the journal Psychological Science to check whether papers with or without an Open Practice Badge differed in the prevalence of reporting errors. Overall, we found no relationship between data sharing and reporting inconsistencies. We did find that journal policies on data sharing are extremely effective in promoting data sharing. We argue that open data is essential in improving the quality of psychological science, and we discuss ways to detect and reduce reporting inconsistencies in the literature.
Original languageEnglish
Article number31
JournalCollabra: Psychology
Volume3
Issue number1
DOIs
Publication statusPublished - 2017

Fingerprint

Information Dissemination

Keywords

  • data sharing
  • journal policy
  • meta-research
  • statistical errors

Cite this

@article{06a3a0d6bfaf49afa602954731f56149,
title = "Journal data sharing policies and statistical reporting inconsistencies in psychology",
abstract = "In this paper, we present three retrospective observational studies that investigate the relation between data sharing and statistical reporting inconsistencies. Previous research found that reluctance to share data was related to a higher prevalence of statistical errors, often in the direction of statistical significance (Wicherts, Bakker, & Molenaar, 2011). We therefore hypothesized that journal policies about data sharing and data sharing itself would reduce these inconsistencies. In Study 1, we compared the prevalence of reporting inconsistencies in two similar journals on decision making with different data sharing policies. In Study 2, we compared reporting inconsistencies in psychology articles published in PLOS journals (with a data sharing policy) and Frontiers in Psychology (without a stipulated data sharing policy). In Study 3, we looked at papers published in the journal Psychological Science to check whether papers with or without an Open Practice Badge differed in the prevalence of reporting errors. Overall, we found no relationship between data sharing and reporting inconsistencies. We did find that journal policies on data sharing are extremely effective in promoting data sharing. We argue that open data is essential in improving the quality of psychological science, and we discuss ways to detect and reduce reporting inconsistencies in the literature.",
keywords = "data sharing, journal policy, meta-research, statistical errors",
author = "M.B. Nuijten and J. Borghuis and C.L.S. Veldkamp and {Dominguez Alvarez}, L. and {van Assen}, M.A.L.M. and J.M. Wicherts",
note = "Retrieved from psyarxiv.com/sgbta",
year = "2017",
doi = "10.17605/OSF.IO/SGBTA",
language = "English",
volume = "3",
journal = "Collabra: Psychology",
number = "1",

}

Journal data sharing policies and statistical reporting inconsistencies in psychology. / Nuijten, M.B.; Borghuis, J.; Veldkamp, C.L.S.; Dominguez Alvarez, L.; van Assen, M.A.L.M.; Wicherts, J.M.

In: Collabra: Psychology, Vol. 3, No. 1, 31, 2017.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Journal data sharing policies and statistical reporting inconsistencies in psychology

AU - Nuijten, M.B.

AU - Borghuis, J.

AU - Veldkamp, C.L.S.

AU - Dominguez Alvarez, L.

AU - van Assen, M.A.L.M.

AU - Wicherts, J.M.

N1 - Retrieved from psyarxiv.com/sgbta

PY - 2017

Y1 - 2017

N2 - In this paper, we present three retrospective observational studies that investigate the relation between data sharing and statistical reporting inconsistencies. Previous research found that reluctance to share data was related to a higher prevalence of statistical errors, often in the direction of statistical significance (Wicherts, Bakker, & Molenaar, 2011). We therefore hypothesized that journal policies about data sharing and data sharing itself would reduce these inconsistencies. In Study 1, we compared the prevalence of reporting inconsistencies in two similar journals on decision making with different data sharing policies. In Study 2, we compared reporting inconsistencies in psychology articles published in PLOS journals (with a data sharing policy) and Frontiers in Psychology (without a stipulated data sharing policy). In Study 3, we looked at papers published in the journal Psychological Science to check whether papers with or without an Open Practice Badge differed in the prevalence of reporting errors. Overall, we found no relationship between data sharing and reporting inconsistencies. We did find that journal policies on data sharing are extremely effective in promoting data sharing. We argue that open data is essential in improving the quality of psychological science, and we discuss ways to detect and reduce reporting inconsistencies in the literature.

AB - In this paper, we present three retrospective observational studies that investigate the relation between data sharing and statistical reporting inconsistencies. Previous research found that reluctance to share data was related to a higher prevalence of statistical errors, often in the direction of statistical significance (Wicherts, Bakker, & Molenaar, 2011). We therefore hypothesized that journal policies about data sharing and data sharing itself would reduce these inconsistencies. In Study 1, we compared the prevalence of reporting inconsistencies in two similar journals on decision making with different data sharing policies. In Study 2, we compared reporting inconsistencies in psychology articles published in PLOS journals (with a data sharing policy) and Frontiers in Psychology (without a stipulated data sharing policy). In Study 3, we looked at papers published in the journal Psychological Science to check whether papers with or without an Open Practice Badge differed in the prevalence of reporting errors. Overall, we found no relationship between data sharing and reporting inconsistencies. We did find that journal policies on data sharing are extremely effective in promoting data sharing. We argue that open data is essential in improving the quality of psychological science, and we discuss ways to detect and reduce reporting inconsistencies in the literature.

KW - data sharing

KW - journal policy

KW - meta-research

KW - statistical errors

U2 - 10.17605/OSF.IO/SGBTA

DO - 10.17605/OSF.IO/SGBTA

M3 - Article

VL - 3

JO - Collabra: Psychology

JF - Collabra: Psychology

IS - 1

M1 - 31

ER -