Assessing the item response theory with covariate (IRT-C) procedure for ascertaining differential item functioning

L. Tay, J.K. Vermunt, C. Wang

Research output: Contribution to journalArticleScientificpeer-review

Abstract

We evaluate the item response theory with covariates (IRT-C) procedure for assessing differential item functioning (DIF) without preknowledge of anchor items (Tay, Newman, & Vermunt, 2011). This procedure begins with a fully constrained baseline model, and candidate items are tested for uniform and/or nonuniform DIF using the Wald statistic. Candidate items are selected in turn based on high unconditional bivariate residual (UBVR) values. This iterative process continues until no further DIF is detected or the Bayes information criterion (BIC) increases. We expanded on the procedure and examined the use of conditional bivariate residuals (CBVR) to flag for DIF; aside from the BIC, alternative stopping criteria were also considered. Simulation results showed that the IRT-C approach for assessing DIF performed well, with the use of CBVR yielding slightly better power and Type I error rates than UBVR. Additionally, using no information criterion yielded higher power than using the BIC, although Type I error rates were generally well controlled in both cases. Across the simulation conditions, the IRT-C procedure produced results similar to the Mantel-Haenszel and MIMIC procedures. Keywords: differential item functioning, item response theory, multiple covariates, simulation
Original languageEnglish
Pages (from-to)201-222
JournalInternational Journal of Testing
Volume13
Issue number3
DOIs
Publication statusPublished - 2013

Fingerprint

Differential Item Functioning
Covariates
Bayes Information Criterion
Anchors
Type I Error Rate
simulation
Statistics
candidacy
Wald Statistic
Simulation
Stopping Criterion
Information Criterion
Iterative Process
High Power
statistics
Baseline
Continue
Evaluate
Alternatives
Values

Cite this

@article{72ebcdda287348739b85dbe2e1f2d86e,
title = "Assessing the item response theory with covariate (IRT-C) procedure for ascertaining differential item functioning",
abstract = "We evaluate the item response theory with covariates (IRT-C) procedure for assessing differential item functioning (DIF) without preknowledge of anchor items (Tay, Newman, & Vermunt, 2011). This procedure begins with a fully constrained baseline model, and candidate items are tested for uniform and/or nonuniform DIF using the Wald statistic. Candidate items are selected in turn based on high unconditional bivariate residual (UBVR) values. This iterative process continues until no further DIF is detected or the Bayes information criterion (BIC) increases. We expanded on the procedure and examined the use of conditional bivariate residuals (CBVR) to flag for DIF; aside from the BIC, alternative stopping criteria were also considered. Simulation results showed that the IRT-C approach for assessing DIF performed well, with the use of CBVR yielding slightly better power and Type I error rates than UBVR. Additionally, using no information criterion yielded higher power than using the BIC, although Type I error rates were generally well controlled in both cases. Across the simulation conditions, the IRT-C procedure produced results similar to the Mantel-Haenszel and MIMIC procedures. Keywords: differential item functioning, item response theory, multiple covariates, simulation",
author = "L. Tay and J.K. Vermunt and C. Wang",
year = "2013",
doi = "10.1080/15305058.2012.692415",
language = "English",
volume = "13",
pages = "201--222",
journal = "International Journal of Testing",
issn = "1530-5058",
publisher = "Routledge",
number = "3",

}

Assessing the item response theory with covariate (IRT-C) procedure for ascertaining differential item functioning. / Tay, L.; Vermunt, J.K.; Wang, C.

In: International Journal of Testing, Vol. 13, No. 3, 2013, p. 201-222.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Assessing the item response theory with covariate (IRT-C) procedure for ascertaining differential item functioning

AU - Tay, L.

AU - Vermunt, J.K.

AU - Wang, C.

PY - 2013

Y1 - 2013

N2 - We evaluate the item response theory with covariates (IRT-C) procedure for assessing differential item functioning (DIF) without preknowledge of anchor items (Tay, Newman, & Vermunt, 2011). This procedure begins with a fully constrained baseline model, and candidate items are tested for uniform and/or nonuniform DIF using the Wald statistic. Candidate items are selected in turn based on high unconditional bivariate residual (UBVR) values. This iterative process continues until no further DIF is detected or the Bayes information criterion (BIC) increases. We expanded on the procedure and examined the use of conditional bivariate residuals (CBVR) to flag for DIF; aside from the BIC, alternative stopping criteria were also considered. Simulation results showed that the IRT-C approach for assessing DIF performed well, with the use of CBVR yielding slightly better power and Type I error rates than UBVR. Additionally, using no information criterion yielded higher power than using the BIC, although Type I error rates were generally well controlled in both cases. Across the simulation conditions, the IRT-C procedure produced results similar to the Mantel-Haenszel and MIMIC procedures. Keywords: differential item functioning, item response theory, multiple covariates, simulation

AB - We evaluate the item response theory with covariates (IRT-C) procedure for assessing differential item functioning (DIF) without preknowledge of anchor items (Tay, Newman, & Vermunt, 2011). This procedure begins with a fully constrained baseline model, and candidate items are tested for uniform and/or nonuniform DIF using the Wald statistic. Candidate items are selected in turn based on high unconditional bivariate residual (UBVR) values. This iterative process continues until no further DIF is detected or the Bayes information criterion (BIC) increases. We expanded on the procedure and examined the use of conditional bivariate residuals (CBVR) to flag for DIF; aside from the BIC, alternative stopping criteria were also considered. Simulation results showed that the IRT-C approach for assessing DIF performed well, with the use of CBVR yielding slightly better power and Type I error rates than UBVR. Additionally, using no information criterion yielded higher power than using the BIC, although Type I error rates were generally well controlled in both cases. Across the simulation conditions, the IRT-C procedure produced results similar to the Mantel-Haenszel and MIMIC procedures. Keywords: differential item functioning, item response theory, multiple covariates, simulation

U2 - 10.1080/15305058.2012.692415

DO - 10.1080/15305058.2012.692415

M3 - Article

VL - 13

SP - 201

EP - 222

JO - International Journal of Testing

JF - International Journal of Testing

SN - 1530-5058

IS - 3

ER -