A tutorial on Bayesian single-test reliability analysis with JASP

Julius M. Pfadt*, Don Van Den Bergh, Klaas Sijtsma, Eric-Jan Wagenmakers

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

4 Citations (Scopus)
127 Downloads (Pure)

Abstract

The current practice of reliability analysis is both uniform and troublesome: most reports consider only Cronbach’s α, and almost all reports focus exclusively on a point estimate, disregarding the impact of sampling error. In an attempt to improve the status quo we have implemented Bayesian estimation routines for five popular single-test reliability coefficients in the open-source statistical software program JASP. Using JASP, researchers can easily obtain Bayesian credible intervals to indicate a range of plausible values and thereby quantify the precision of the point estimate. In addition, researchers may use the posterior distribution of the reliability coefficients to address practically relevant questions such as “What is the probability that the reliability of my test is larger than a threshold value of .80?”. In this tutorial article, we outline how to conduct a Bayesian reliability analysis in JASP and correctly interpret the results. By making available a computationally complex procedure in an easy-to-use software package, we hope to motivate researchers to include uncertainty estimates whenever reporting the results of a single-test reliability analysis.

Original languageEnglish
Pages (from-to)1069-1078
JournalBehavior Research Methods
Volume55
Issue number3
DOIs
Publication statusPublished - 2023

Keywords

  • COEFFICIENT ALPHA
  • Credible interval
  • McDonald's omega
  • OMEGA

Fingerprint

Dive into the research topics of 'A tutorial on Bayesian single-test reliability analysis with JASP'. Together they form a unique fingerprint.

Cite this