Methods for estimating item-score reliability

E.A.O. Zijlmans, L.A. van der Ark, J. Tijmstra, K. Sijtsma

Research output: Contribution to journalArticleScientificpeer-review

8 Downloads (Pure)

Abstract

Reliability is usually estimated for a test score, but it can also be estimated for item scores. Item-score reliability can be useful to assess the item’s contribution to the test score’s reliability, for identifying unreliable scores in aberrant item-score patterns in person-fit analysis, and for selecting the most reliable item from a test to use as a single-item measure. Four methods were discussed for estimating item-score reliability: the Molenaar–Sijtsma method (method MS), Guttman’s method λ6, the latent class reliability coefficient (method LCRC), and the correction for attenuation (method CA). A simulation study was used to compare the methods with respect to median bias, variability (interquartile range [IQR]), and percentage of outliers. The simulation study consisted of six conditions: standard, polytomous items, unequal α parameters, two-dimensional data, long test, and small sample size. Methods MS and CA were the most accurate. Method LCRC showed almost unbiased results, but large variability. Method λ6 consistently underestimated item-score reliabilty, but showed a smaller IQR than the other methods.
Original languageEnglish
Pages (from-to)553-570
JournalApplied Psychological Measurement
Volume42
Issue number7
DOIs
Publication statusPublished - 2018

    Fingerprint

Keywords

  • Guttman's method (6)
  • JOB-SATISFACTION
  • MODEL
  • SCALE
  • SINGLE-ITEM
  • VALIDITY
  • correction for attenuation
  • item-score reliability
  • latent class reliability coefficient
  • method MS

Cite this