A causal theory of error scores

Riet Van Bork*, Mijke Rhemtulla, Klaas Sijtsma, Denny Borsboom

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

8 Citations (Scopus)

Abstract

In modern test theory, response variables are a function of a common latent variable that represents the measured attribute, and error variables that are unique to the response variables. While considerable thought goes into the interpretation of latent variables in these models (e.g., validity research), the interpretation of error variables is typically left implicit (e.g., describing error variables as residuals). Yet, many psychometric assumptions are essentially assumptions about error and thus being able to reason about psychometric models requires the ability to reason about errors. We propose a causal theory of error as a framework that enables researchers to reason about errors in terms of the data-generating mechanism. In this framework, the error variable reflects myriad causes that are specific to an item and, together with the latent variable, determine the scores on that item. We distinguish two types of item-specific causes: characteristic variables that differ between people (e.g., familiarity with words used in the item), and circumstance variables that vary over occasions in which the item is administered (e.g., a distracting noise). We show that different assumptions about these unique causes (a) imply different psychometric models; (b) have different implications for the chance experiment that makes these models probabilistic models; and (c) have different consequences for item bias, local homogeneity, and reliability coefficient α and the test-retest correlation. The ability to reason about the causes that produce error variance puts researchers in a better position to motivate modeling choices.
Original languageEnglish
Number of pages21
JournalPsychological Methods
DOIs
Publication statusE-pub ahead of print - 2023

Keywords

  • COEFFICIENT ALPHA
  • GENERALIZABILITY
  • INVARIANCE
  • ITEM RESPONSE THEORY
  • LATENT
  • PREDICTION
  • PSYCHOLOGY
  • RELIABILITY
  • STATE-TRAIT MODEL
  • causal theory of error
  • error definition of IRT models
  • item response theory
  • latent variable models

Fingerprint

Dive into the research topics of 'A causal theory of error scores'. Together they form a unique fingerprint.

Cite this