The assessment and impact of careless responding in routine outcome monitoring within mental Health care

Judith M. Conijn*, Gunhild Franz, Wilco H. M. Emons, Edwin De Beurs, Ingrid V. E. Carlier

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Careless responding by mental health patients on self-report assessments is rarely investigated in routine care despite the potential for serious consequences such as faulty clinical decisions. We investigated validity indices most appropriate for detecting careless responding in routine outcome monitoring (ROM) in mental health-care. First, we reviewed indices proposed in previous research for their suitability in ROM. Next, we evaluated six selected indices using data of the Brief Symptom Inventory and the Mood and Anxiety Symptom Questionnaire from 3,483 outpatients. Simulations showed that for typical ROM scales the Lmax index, Mahalanobis distance, and inter-item standard deviation may be too strongly confounded with the latent trait value to compare careless responding across patients with different symptom severity. Application of two different classification methods to the validity indices did not converge in similar prevalence estimates of careless responding. Finally, results suggest that careless responding does not have a substantial biasing effect on scale-score statistics. We recommend the lpz person-fit index to screen for random careless responding in large ROM data sets. However, additional research should further investigate methods for detecting repetitive responding in typical ROM data and assess whether there are specific circumstances in which simpler validity statistics or direct screening methods perform similarly as the lpz index.
Original languageEnglish
Pages (from-to)593-611
JournalMultivariate Behavioral Research
Volume54
Issue number4
DOIs
Publication statusPublished - 2019

Fingerprint

Healthcare
Mental Health
Monitoring
Delivery of Health Care
Validity Index
Mentally Ill Persons
Self Report
Latent Trait
Outpatients
Score Statistic
Mood
Mahalanobis Distance
Anxiety
Equipment and Supplies
Questionnaire
Standard deviation
Screening
Person
Health
Statistics

Keywords

  • ACQUIESCENCE
  • ANXIETY
  • BRIEF SYMPTOM INVENTORY
  • Careless responding
  • DEPRESSION
  • MMPI-2-RF
  • PERSON-FIT
  • QUESTIONNAIRE
  • RESPONSES
  • TRIPARTITE MODEL
  • VALIDITY
  • mental health care
  • person-fit indices
  • routine outcome monitoring
  • validity indices

Cite this

Conijn, Judith M. ; Franz, Gunhild ; Emons, Wilco H. M. ; De Beurs, Edwin ; Carlier, Ingrid V. E. / The assessment and impact of careless responding in routine outcome monitoring within mental Health care. In: Multivariate Behavioral Research. 2019 ; Vol. 54, No. 4. pp. 593-611.
@article{23f72262928c4f4498617e3a6a4f7e92,
title = "The assessment and impact of careless responding in routine outcome monitoring within mental Health care",
abstract = "Careless responding by mental health patients on self-report assessments is rarely investigated in routine care despite the potential for serious consequences such as faulty clinical decisions. We investigated validity indices most appropriate for detecting careless responding in routine outcome monitoring (ROM) in mental health-care. First, we reviewed indices proposed in previous research for their suitability in ROM. Next, we evaluated six selected indices using data of the Brief Symptom Inventory and the Mood and Anxiety Symptom Questionnaire from 3,483 outpatients. Simulations showed that for typical ROM scales the Lmax index, Mahalanobis distance, and inter-item standard deviation may be too strongly confounded with the latent trait value to compare careless responding across patients with different symptom severity. Application of two different classification methods to the validity indices did not converge in similar prevalence estimates of careless responding. Finally, results suggest that careless responding does not have a substantial biasing effect on scale-score statistics. We recommend the lpz person-fit index to screen for random careless responding in large ROM data sets. However, additional research should further investigate methods for detecting repetitive responding in typical ROM data and assess whether there are specific circumstances in which simpler validity statistics or direct screening methods perform similarly as the lpz index.",
keywords = "ACQUIESCENCE, ANXIETY, BRIEF SYMPTOM INVENTORY, Careless responding, DEPRESSION, MMPI-2-RF, PERSON-FIT, QUESTIONNAIRE, RESPONSES, TRIPARTITE MODEL, VALIDITY, mental health care, person-fit indices, routine outcome monitoring, validity indices",
author = "Conijn, {Judith M.} and Gunhild Franz and Emons, {Wilco H. M.} and {De Beurs}, Edwin and Carlier, {Ingrid V. E.}",
year = "2019",
doi = "10.1080/00273171.2018.1563520",
language = "English",
volume = "54",
pages = "593--611",
journal = "Multivariate Behavioral Research",
issn = "0027-3171",
publisher = "ROUTLEDGE JOURNALS, TAYLOR & FRANCIS LTD",
number = "4",

}

The assessment and impact of careless responding in routine outcome monitoring within mental Health care. / Conijn, Judith M.; Franz, Gunhild; Emons, Wilco H. M.; De Beurs, Edwin; Carlier, Ingrid V. E.

In: Multivariate Behavioral Research, Vol. 54, No. 4, 2019, p. 593-611.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - The assessment and impact of careless responding in routine outcome monitoring within mental Health care

AU - Conijn, Judith M.

AU - Franz, Gunhild

AU - Emons, Wilco H. M.

AU - De Beurs, Edwin

AU - Carlier, Ingrid V. E.

PY - 2019

Y1 - 2019

N2 - Careless responding by mental health patients on self-report assessments is rarely investigated in routine care despite the potential for serious consequences such as faulty clinical decisions. We investigated validity indices most appropriate for detecting careless responding in routine outcome monitoring (ROM) in mental health-care. First, we reviewed indices proposed in previous research for their suitability in ROM. Next, we evaluated six selected indices using data of the Brief Symptom Inventory and the Mood and Anxiety Symptom Questionnaire from 3,483 outpatients. Simulations showed that for typical ROM scales the Lmax index, Mahalanobis distance, and inter-item standard deviation may be too strongly confounded with the latent trait value to compare careless responding across patients with different symptom severity. Application of two different classification methods to the validity indices did not converge in similar prevalence estimates of careless responding. Finally, results suggest that careless responding does not have a substantial biasing effect on scale-score statistics. We recommend the lpz person-fit index to screen for random careless responding in large ROM data sets. However, additional research should further investigate methods for detecting repetitive responding in typical ROM data and assess whether there are specific circumstances in which simpler validity statistics or direct screening methods perform similarly as the lpz index.

AB - Careless responding by mental health patients on self-report assessments is rarely investigated in routine care despite the potential for serious consequences such as faulty clinical decisions. We investigated validity indices most appropriate for detecting careless responding in routine outcome monitoring (ROM) in mental health-care. First, we reviewed indices proposed in previous research for their suitability in ROM. Next, we evaluated six selected indices using data of the Brief Symptom Inventory and the Mood and Anxiety Symptom Questionnaire from 3,483 outpatients. Simulations showed that for typical ROM scales the Lmax index, Mahalanobis distance, and inter-item standard deviation may be too strongly confounded with the latent trait value to compare careless responding across patients with different symptom severity. Application of two different classification methods to the validity indices did not converge in similar prevalence estimates of careless responding. Finally, results suggest that careless responding does not have a substantial biasing effect on scale-score statistics. We recommend the lpz person-fit index to screen for random careless responding in large ROM data sets. However, additional research should further investigate methods for detecting repetitive responding in typical ROM data and assess whether there are specific circumstances in which simpler validity statistics or direct screening methods perform similarly as the lpz index.

KW - ACQUIESCENCE

KW - ANXIETY

KW - BRIEF SYMPTOM INVENTORY

KW - Careless responding

KW - DEPRESSION

KW - MMPI-2-RF

KW - PERSON-FIT

KW - QUESTIONNAIRE

KW - RESPONSES

KW - TRIPARTITE MODEL

KW - VALIDITY

KW - mental health care

KW - person-fit indices

KW - routine outcome monitoring

KW - validity indices

UR - https://app-eu.readspeaker.com/cgi-bin/rsent?customerid=10118&lang=en_us&readclass=rs_readArea&url=https%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Ffull%2F10.1080%2F00273171.2018.1563520

U2 - 10.1080/00273171.2018.1563520

DO - 10.1080/00273171.2018.1563520

M3 - Article

VL - 54

SP - 593

EP - 611

JO - Multivariate Behavioral Research

JF - Multivariate Behavioral Research

SN - 0027-3171

IS - 4

ER -