Testing for aberrant behavior in response time modeling

Sukaesi Marianti, Jean-Paul Fox, Marianna Avetisyan, Bernard P. Veldkamp, J. Tijmstra

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Many standardized tests are now administered via computer rather than paper-and-pencil format. In a computer-based testing environment, it is possible to record not only the test taker’s response to each question (item) but also the amount of time spent by the test taker in considering and answering each item. Response times (RTs) provide information not only about the test taker’s ability and response behavior but also about item and test characteristics. This study focuses on the use of RTs to detect aberrant test-taker responses. An example of such aberrance is a correct answer with a short RT on a difficult question. Such aberrance may be displayed when a test taker or test takers have preknowledge of the items. Another example is rapid guessing, wherein the test taker displays unusually short RTs for a series of items. When rapid guessing occurs at the end of a timed test, it often indicates that the test taker has run out of time before completing the test. In this study, Bayesian tests of significance for detecting various types of aberrant RT patterns are proposed and evaluated. In a simulation study, the tests were successful in identifying aberrant response patterns. A real data example is given to illustrate the use of the proposed person-fit tests for RTs
Original languageEnglish
Pages (from-to)426-451
JournalJournal of Educational and Behavioral Statistics
Volume39
Issue number6
DOIs
Publication statusPublished - Dec 2014

Keywords

  • response times
  • aberrant behavior
  • person fit

Cite this

Marianti, Sukaesi ; Fox, Jean-Paul ; Avetisyan, Marianna ; Veldkamp, Bernard P. ; Tijmstra, J. / Testing for aberrant behavior in response time modeling. In: Journal of Educational and Behavioral Statistics. 2014 ; Vol. 39, No. 6. pp. 426-451.
@article{fd623f38bf854f08a06df8a6bdc7e86d,
title = "Testing for aberrant behavior in response time modeling",
abstract = "Many standardized tests are now administered via computer rather than paper-and-pencil format. In a computer-based testing environment, it is possible to record not only the test taker’s response to each question (item) but also the amount of time spent by the test taker in considering and answering each item. Response times (RTs) provide information not only about the test taker’s ability and response behavior but also about item and test characteristics. This study focuses on the use of RTs to detect aberrant test-taker responses. An example of such aberrance is a correct answer with a short RT on a difficult question. Such aberrance may be displayed when a test taker or test takers have preknowledge of the items. Another example is rapid guessing, wherein the test taker displays unusually short RTs for a series of items. When rapid guessing occurs at the end of a timed test, it often indicates that the test taker has run out of time before completing the test. In this study, Bayesian tests of significance for detecting various types of aberrant RT patterns are proposed and evaluated. In a simulation study, the tests were successful in identifying aberrant response patterns. A real data example is given to illustrate the use of the proposed person-fit tests for RTs",
keywords = "response times, aberrant behavior, person fit",
author = "Sukaesi Marianti and Jean-Paul Fox and Marianna Avetisyan and Veldkamp, {Bernard P.} and J. Tijmstra",
year = "2014",
month = "12",
doi = "10.3102/1076998614559412",
language = "English",
volume = "39",
pages = "426--451",
journal = "Journal of Educational and Behavioral Statistics",
issn = "1076-9986",
publisher = "Sage Publications, Inc.",
number = "6",

}

Testing for aberrant behavior in response time modeling. / Marianti, Sukaesi; Fox, Jean-Paul; Avetisyan, Marianna; Veldkamp, Bernard P.; Tijmstra, J.

In: Journal of Educational and Behavioral Statistics, Vol. 39, No. 6, 12.2014, p. 426-451.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Testing for aberrant behavior in response time modeling

AU - Marianti, Sukaesi

AU - Fox, Jean-Paul

AU - Avetisyan, Marianna

AU - Veldkamp, Bernard P.

AU - Tijmstra, J.

PY - 2014/12

Y1 - 2014/12

N2 - Many standardized tests are now administered via computer rather than paper-and-pencil format. In a computer-based testing environment, it is possible to record not only the test taker’s response to each question (item) but also the amount of time spent by the test taker in considering and answering each item. Response times (RTs) provide information not only about the test taker’s ability and response behavior but also about item and test characteristics. This study focuses on the use of RTs to detect aberrant test-taker responses. An example of such aberrance is a correct answer with a short RT on a difficult question. Such aberrance may be displayed when a test taker or test takers have preknowledge of the items. Another example is rapid guessing, wherein the test taker displays unusually short RTs for a series of items. When rapid guessing occurs at the end of a timed test, it often indicates that the test taker has run out of time before completing the test. In this study, Bayesian tests of significance for detecting various types of aberrant RT patterns are proposed and evaluated. In a simulation study, the tests were successful in identifying aberrant response patterns. A real data example is given to illustrate the use of the proposed person-fit tests for RTs

AB - Many standardized tests are now administered via computer rather than paper-and-pencil format. In a computer-based testing environment, it is possible to record not only the test taker’s response to each question (item) but also the amount of time spent by the test taker in considering and answering each item. Response times (RTs) provide information not only about the test taker’s ability and response behavior but also about item and test characteristics. This study focuses on the use of RTs to detect aberrant test-taker responses. An example of such aberrance is a correct answer with a short RT on a difficult question. Such aberrance may be displayed when a test taker or test takers have preknowledge of the items. Another example is rapid guessing, wherein the test taker displays unusually short RTs for a series of items. When rapid guessing occurs at the end of a timed test, it often indicates that the test taker has run out of time before completing the test. In this study, Bayesian tests of significance for detecting various types of aberrant RT patterns are proposed and evaluated. In a simulation study, the tests were successful in identifying aberrant response patterns. A real data example is given to illustrate the use of the proposed person-fit tests for RTs

KW - response times

KW - aberrant behavior

KW - person fit

U2 - 10.3102/1076998614559412

DO - 10.3102/1076998614559412

M3 - Article

VL - 39

SP - 426

EP - 451

JO - Journal of Educational and Behavioral Statistics

JF - Journal of Educational and Behavioral Statistics

SN - 1076-9986

IS - 6

ER -