Assessing the equivalence of web-based and paper-and-pencil questionnaires using differential item and test functioning (DIF and DTF) analysis: A case of the Four-Dimensional Symptom Questionnaire (4DSQ)

B. Terluin, E.P.M. Brouwers, M.A.G. Marchand, H.C. De Vet

Research output: Contribution to journalArticleScientificpeer-review

29 Downloads (Pure)

Abstract

Purpose: 
Many paper-and-pencil (P&P) questionnaires have been migrated to electronic platforms. Differential item and test functioning (DIF and DTF) analysis constitutes a superior research design to assess measurement equivalence across modes of administration. The purpose of this study was to demonstrate an item response theory (IRT)-based DIF and DTF analysis to assess the measurement equivalence of a Web-based version and the original P&P format of the Four-Dimensional Symptom Questionnaire (4DSQ), measuring distress, depression, anxiety, and somatization.
Methods: 
The P&P group (n=2031) and the Web group (n=958) consisted of primary care psychology clients. Unidimensionality and local independence of the 4DSQ scales were examined using IRT and Yen’s Q3. Bifactor modeling was used to assess the scales’ essential unidimensionality. Measurement equivalence was assessed using IRT-based DIF analysis using a 3-stage approach: linking on the latent mean and variance, selection of anchor items, and DIF testing using the Wald test. DTF was evaluated by comparing expected scale scores as a function of the latent trait.
Results: 
The 4DSQ scales proved to be essentially unidimensional in both modalities. Five items, belonging to the distress and somatization scales, displayed small amounts of DIF. DTF analysis revealed that the impact of DIF on the scale levelwas negligible.
Conclusions: 
IRT-based DIF and DTF analysis is demonstrated as a way to assess the equivalence of Web-based and P&P questionnaire modalities. Data obtained with the Web-based 4DSQ are equivalent to data obtained with the P&P version.
Original languageEnglish
Pages (from-to)1191-1200
JournalQuality of Life Research
Volume27
Issue number5
DOIs
Publication statusPublished - 2018

    Fingerprint

Keywords

  • Bifactor model
  • DEPRESSION
  • Differential item functioning
  • Differential test functioning
  • FIT
  • INDEXES
  • IRT
  • MEASUREMENT INVARIANCE
  • MODELING APPROACH
  • MULTIDIMENSIONALITY
  • Measurement equivalence
  • POPULATION
  • Paper-and-pencil questionnaire
  • REPORTED OUTCOME MEASURES
  • RESPONSE THEORY
  • Web-based questionnaire

Cite this