Predicting undesired treatment outcome in mental healthcare: Machine learning study

K. van Mens*, J. Lokkerbol, R. T. J. M. Janssen, R. P. J. de Lange, B. Tiemens

*Corresponding author for this work

Research output: Contribution to journalArticleScientific

Abstract

Background:
It remains a challenge to predict which treatment will work for which patient in mental healthcare.

Objective:
In this study we compare machine algorithms to predict during treatment which patients will not benefit from brief mental health treatment and present trade-offs that must be considered before an algorithm can be used in clinical practice.

Methods:
Using an anonymized dataset containing routine outcome monitoring data from a mental healthcare organization in the Netherlands (n = 2,655), we applied three machine learning algorithms to predict treatment outcome. The algorithms were internally validated with cross-validation on a training sample (n = 1,860) and externally validated on an unseen test sample (n = 795).

Results:
The performance of the three algorithms did not significantly differ on the test set. With a default classification cut-off at 0.5 predicted probability, the extreme gradient boosting algorithm showed the highest positive predictive value (ppv) of 0.71(0.61 – 0.77) with a sensitivity of 0.35 (0.29 – 0.41) and area under the curve of 0.78. A trade-off can be made between ppv and sensitivity by choosing different cut-off probabilities. With a cut-off at 0.63, the ppv increased to 0.87 and the sensitivity dropped to 0.17. With a cut-off of at 0.38, the ppv decreased to 0.61 and the sensitivity increased to 0.57.

Conclusions:
Machine learning can be used to predict treatment outcomes based on routine monitoring data.This allows practitioners to choose their own trade-off between being selective and more certain versus inclusive and less certain.
Original languageEnglish
Number of pages24
JournalJMIR Medical Informatics
Publication statusAccepted/In press - 2020

Fingerprint Dive into the research topics of 'Predicting undesired treatment outcome in mental healthcare: Machine learning study'. Together they form a unique fingerprint.

Cite this