Comparison of classical test theory and item response theory in individual change assessment

R. Jabrayilov, W.H.M. Emons, K. Sijtsma

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Clinical psychologists are advised to assess clinical and statistical significance when assessing change in individual patients. Individual change assessment can be conducted using either the methodologies of classical test theory (CTT) or item response theory (IRT). Researchers have been optimistic about the possible advantages of using IRT rather than CTT in change assessment. However, little empirical evidence is available to support the alleged superiority of IRT in the context of individual change assessment. In this study, the authors compared the CTT and IRT methods with respect to their Type I error and detection rates. Preliminary results revealed that IRT is indeed superior to CTT in individual change detection, provided that the tests consist of at least 20 items. For shorter tests, however, CTT is generally better at correctly detecting change in individuals. The results and their implications are discussed.
Original languageEnglish
Pages (from-to)559-572
JournalApplied Psychological Measurement
Volume40
Issue number8
DOIs
Publication statusPublished - 2016

Fingerprint

test theory
statistical significance
psychologist
methodology
evidence

Cite this

Jabrayilov, R. ; Emons, W.H.M. ; Sijtsma, K. / Comparison of classical test theory and item response theory in individual change assessment. In: Applied Psychological Measurement. 2016 ; Vol. 40, No. 8. pp. 559-572.
@article{6948d01b997a463d95bb7e1fb34e3b0a,
title = "Comparison of classical test theory and item response theory in individual change assessment",
abstract = "Clinical psychologists are advised to assess clinical and statistical significance when assessing change in individual patients. Individual change assessment can be conducted using either the methodologies of classical test theory (CTT) or item response theory (IRT). Researchers have been optimistic about the possible advantages of using IRT rather than CTT in change assessment. However, little empirical evidence is available to support the alleged superiority of IRT in the context of individual change assessment. In this study, the authors compared the CTT and IRT methods with respect to their Type I error and detection rates. Preliminary results revealed that IRT is indeed superior to CTT in individual change detection, provided that the tests consist of at least 20 items. For shorter tests, however, CTT is generally better at correctly detecting change in individuals. The results and their implications are discussed.",
author = "R. Jabrayilov and W.H.M. Emons and K. Sijtsma",
year = "2016",
doi = "10.1177/0146621616664046",
language = "English",
volume = "40",
pages = "559--572",
journal = "Applied Psychological Measurement",
issn = "0146-6216",
publisher = "SAGE Publications Inc.",
number = "8",

}

Comparison of classical test theory and item response theory in individual change assessment. / Jabrayilov, R.; Emons, W.H.M.; Sijtsma, K.

In: Applied Psychological Measurement, Vol. 40, No. 8, 2016, p. 559-572.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Comparison of classical test theory and item response theory in individual change assessment

AU - Jabrayilov, R.

AU - Emons, W.H.M.

AU - Sijtsma, K.

PY - 2016

Y1 - 2016

N2 - Clinical psychologists are advised to assess clinical and statistical significance when assessing change in individual patients. Individual change assessment can be conducted using either the methodologies of classical test theory (CTT) or item response theory (IRT). Researchers have been optimistic about the possible advantages of using IRT rather than CTT in change assessment. However, little empirical evidence is available to support the alleged superiority of IRT in the context of individual change assessment. In this study, the authors compared the CTT and IRT methods with respect to their Type I error and detection rates. Preliminary results revealed that IRT is indeed superior to CTT in individual change detection, provided that the tests consist of at least 20 items. For shorter tests, however, CTT is generally better at correctly detecting change in individuals. The results and their implications are discussed.

AB - Clinical psychologists are advised to assess clinical and statistical significance when assessing change in individual patients. Individual change assessment can be conducted using either the methodologies of classical test theory (CTT) or item response theory (IRT). Researchers have been optimistic about the possible advantages of using IRT rather than CTT in change assessment. However, little empirical evidence is available to support the alleged superiority of IRT in the context of individual change assessment. In this study, the authors compared the CTT and IRT methods with respect to their Type I error and detection rates. Preliminary results revealed that IRT is indeed superior to CTT in individual change detection, provided that the tests consist of at least 20 items. For shorter tests, however, CTT is generally better at correctly detecting change in individuals. The results and their implications are discussed.

U2 - 10.1177/0146621616664046

DO - 10.1177/0146621616664046

M3 - Article

VL - 40

SP - 559

EP - 572

JO - Applied Psychological Measurement

JF - Applied Psychological Measurement

SN - 0146-6216

IS - 8

ER -