Modeling conditional dependence between response time and accuracy

Maria Bolsinova, Paul de Boeck, J. Tijmstra

Research output: Contribution to journalArticleScientificpeer-review

Abstract

The assumption of conditional independence between response time and accuracy given speed and ability is commonly made in response time modelling. However, this assumption might be violated in some cases, meaning that the relationship between the response time and the response accuracy of the same item cannot be fully explained by the correlation between the overall speed and ability. We propose to explicitly model the residual dependence between time and accuracy by incorporating the effects of the residual response time on the intercept and the slope parameter of the IRT model for response accuracy. We present an empirical example of a violation of conditional independence from a low-stakes educational test and show that our new model reveals interesting phenomena about the dependence of the item properties on whether the response is relatively fast or slow. For more difficult items responding slowly is associated with a higher probability of a correct response, whereas for the easier items responding slower is associated with a lower probability of a correct response. Moreover, for many of the items slower responses were less informative for the ability because their discrimination parameters decrease with residual response time.
Original languageEnglish
Pages (from-to)1126-1148
JournalPsychometrika
Volume82
Issue number4
DOIs
Publication statusPublished - 2017

Fingerprint

Response Time
Modeling
Conditional Independence
Intercept
Discrimination
Slope
Model
Decrease

Cite this

Bolsinova, Maria ; de Boeck, Paul ; Tijmstra, J. / Modeling conditional dependence between response time and accuracy. In: Psychometrika. 2017 ; Vol. 82, No. 4. pp. 1126-1148.
@article{ab1cc20f89fa4b0dbda60f33d82abb17,
title = "Modeling conditional dependence between response time and accuracy",
abstract = "The assumption of conditional independence between response time and accuracy given speed and ability is commonly made in response time modelling. However, this assumption might be violated in some cases, meaning that the relationship between the response time and the response accuracy of the same item cannot be fully explained by the correlation between the overall speed and ability. We propose to explicitly model the residual dependence between time and accuracy by incorporating the effects of the residual response time on the intercept and the slope parameter of the IRT model for response accuracy. We present an empirical example of a violation of conditional independence from a low-stakes educational test and show that our new model reveals interesting phenomena about the dependence of the item properties on whether the response is relatively fast or slow. For more difficult items responding slowly is associated with a higher probability of a correct response, whereas for the easier items responding slower is associated with a lower probability of a correct response. Moreover, for many of the items slower responses were less informative for the ability because their discrimination parameters decrease with residual response time.",
author = "Maria Bolsinova and {de Boeck}, Paul and J. Tijmstra",
year = "2017",
doi = "10.1007/s11336-016-9537-6",
language = "English",
volume = "82",
pages = "1126--1148",
journal = "Psychometrika",
issn = "0033-3123",
publisher = "Springer",
number = "4",

}

Modeling conditional dependence between response time and accuracy. / Bolsinova, Maria; de Boeck, Paul; Tijmstra, J.

In: Psychometrika, Vol. 82, No. 4, 2017, p. 1126-1148.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Modeling conditional dependence between response time and accuracy

AU - Bolsinova, Maria

AU - de Boeck, Paul

AU - Tijmstra, J.

PY - 2017

Y1 - 2017

N2 - The assumption of conditional independence between response time and accuracy given speed and ability is commonly made in response time modelling. However, this assumption might be violated in some cases, meaning that the relationship between the response time and the response accuracy of the same item cannot be fully explained by the correlation between the overall speed and ability. We propose to explicitly model the residual dependence between time and accuracy by incorporating the effects of the residual response time on the intercept and the slope parameter of the IRT model for response accuracy. We present an empirical example of a violation of conditional independence from a low-stakes educational test and show that our new model reveals interesting phenomena about the dependence of the item properties on whether the response is relatively fast or slow. For more difficult items responding slowly is associated with a higher probability of a correct response, whereas for the easier items responding slower is associated with a lower probability of a correct response. Moreover, for many of the items slower responses were less informative for the ability because their discrimination parameters decrease with residual response time.

AB - The assumption of conditional independence between response time and accuracy given speed and ability is commonly made in response time modelling. However, this assumption might be violated in some cases, meaning that the relationship between the response time and the response accuracy of the same item cannot be fully explained by the correlation between the overall speed and ability. We propose to explicitly model the residual dependence between time and accuracy by incorporating the effects of the residual response time on the intercept and the slope parameter of the IRT model for response accuracy. We present an empirical example of a violation of conditional independence from a low-stakes educational test and show that our new model reveals interesting phenomena about the dependence of the item properties on whether the response is relatively fast or slow. For more difficult items responding slowly is associated with a higher probability of a correct response, whereas for the easier items responding slower is associated with a lower probability of a correct response. Moreover, for many of the items slower responses were less informative for the ability because their discrimination parameters decrease with residual response time.

U2 - 10.1007/s11336-016-9537-6

DO - 10.1007/s11336-016-9537-6

M3 - Article

VL - 82

SP - 1126

EP - 1148

JO - Psychometrika

JF - Psychometrika

SN - 0033-3123

IS - 4

ER -