Using confidence intervals for assessing reliability of real tests

P.R. Oosterwijk, L.A. van der Ark*, K. Sijtsma

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

1 Downloads (Pure)

Abstract

Test authors report sample reliability values but rarely consider the sampling error and related confidence intervals. This study investigated the truth of this conjecture for 116 tests with 1,024 reliability estimates (105 pertaining to test batteries and 919 to tests measuring a single attribute) obtained from an online database. Based on 90% confidence intervals, approximately 20% of the initial quality assessments had to be downgraded. For 95% confidence intervals, the percentage was approximately 23%. The results demonstrated that reported reliability values cannot be trusted without considering their estimation precision.
Keywords: confidence intervals for reliability; precision of reported reliability; quality assessment of reliability; test-score reliability
Original languageEnglish
Pages (from-to)1207-1216
JournalAssessment
Volume26
Issue number7
DOIs
Publication statusPublished - 2019

    Fingerprint

Keywords

  • COEFFICIENT ALPHA
  • HISTORY
  • LOWER BOUNDS
  • SCORE
  • confidence intervals for reliability
  • precision of reported reliability
  • quality assessment of reliability
  • test-score reliability

Cite this

Oosterwijk, P. R., van der Ark, L. A., & Sijtsma, K. (2019). Using confidence intervals for assessing reliability of real tests. Assessment, 26(7), 1207-1216. https://doi.org/10.1177/1073191117737375