Automatic detection of confusion in elderly users of a web-based health instruction video

Research output: Contribution to journalArticleScientificpeer-review

Abstract

BACKGROUND: Because of cognitive limitations and lower health literacy, many elderly patients have difficulty understanding verbal medical instructions. Automatic detection of facial movements provides a nonintrusive basis for building technological tools supporting confusion detection in healthcare delivery applications on the Internet.

MATERIALS AND METHODS: Twenty-four elderly participants (70-90 years old) were recorded while watching Web-based health instruction videos involving easy and complex medical terminology. Relevant fragments of the participants' facial expressions were rated by 40 medical students for perceived level of confusion and analyzed with automatic software for facial movement recognition.

RESULTS: A computer classification of the automatically detected facial features performed more accurately and with a higher sensitivity than the human observers (automatic detection and classification, 64% accuracy, 0.64 sensitivity; human observers, 41% accuracy, 0.43 sensitivity). A drill-down analysis of cues to confusion indicated the importance of the eye and eyebrow region.

CONCLUSIONS: Confusion caused by misunderstanding of medical terminology is signaled by facial cues that can be automatically detected with currently available facial expression detection technology. The findings are relevant for the development of Web-based services for healthcare consumers.

Original languageEnglish
Pages (from-to)514-519
Number of pages6
JournalTelemedicine and e-Health
Volume21
Issue number6
DOIs
Publication statusPublished - Jun 2015

Fingerprint

Facial Expression
Terminology
Cues
Eyebrows
Delivery of Health Care
Mandrillus
Recognition (Psychology)

Cite this

@article{1f7631bc7f1b490ba475607b4ff86e3a,
title = "Automatic detection of confusion in elderly users of a web-based health instruction video",
abstract = "BACKGROUND: Because of cognitive limitations and lower health literacy, many elderly patients have difficulty understanding verbal medical instructions. Automatic detection of facial movements provides a nonintrusive basis for building technological tools supporting confusion detection in healthcare delivery applications on the Internet.MATERIALS AND METHODS: Twenty-four elderly participants (70-90 years old) were recorded while watching Web-based health instruction videos involving easy and complex medical terminology. Relevant fragments of the participants' facial expressions were rated by 40 medical students for perceived level of confusion and analyzed with automatic software for facial movement recognition.RESULTS: A computer classification of the automatically detected facial features performed more accurately and with a higher sensitivity than the human observers (automatic detection and classification, 64{\%} accuracy, 0.64 sensitivity; human observers, 41{\%} accuracy, 0.43 sensitivity). A drill-down analysis of cues to confusion indicated the importance of the eye and eyebrow region.CONCLUSIONS: Confusion caused by misunderstanding of medical terminology is signaled by facial cues that can be automatically detected with currently available facial expression detection technology. The findings are relevant for the development of Web-based services for healthcare consumers.",
author = "Marie Postma-Nilsenov{\'a} and Eric Postma and Kiek Tates",
year = "2015",
month = "6",
doi = "10.1089/tmj.2014.0061",
language = "English",
volume = "21",
pages = "514--519",
journal = "Telemedicine and e-Health",
issn = "1530-5627",
publisher = "Mary Ann Liebert Inc.",
number = "6",

}

Automatic detection of confusion in elderly users of a web-based health instruction video. / Postma-Nilsenová, Marie; Postma, Eric; Tates, Kiek.

In: Telemedicine and e-Health, Vol. 21, No. 6, 06.2015, p. 514-519.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Automatic detection of confusion in elderly users of a web-based health instruction video

AU - Postma-Nilsenová, Marie

AU - Postma, Eric

AU - Tates, Kiek

PY - 2015/6

Y1 - 2015/6

N2 - BACKGROUND: Because of cognitive limitations and lower health literacy, many elderly patients have difficulty understanding verbal medical instructions. Automatic detection of facial movements provides a nonintrusive basis for building technological tools supporting confusion detection in healthcare delivery applications on the Internet.MATERIALS AND METHODS: Twenty-four elderly participants (70-90 years old) were recorded while watching Web-based health instruction videos involving easy and complex medical terminology. Relevant fragments of the participants' facial expressions were rated by 40 medical students for perceived level of confusion and analyzed with automatic software for facial movement recognition.RESULTS: A computer classification of the automatically detected facial features performed more accurately and with a higher sensitivity than the human observers (automatic detection and classification, 64% accuracy, 0.64 sensitivity; human observers, 41% accuracy, 0.43 sensitivity). A drill-down analysis of cues to confusion indicated the importance of the eye and eyebrow region.CONCLUSIONS: Confusion caused by misunderstanding of medical terminology is signaled by facial cues that can be automatically detected with currently available facial expression detection technology. The findings are relevant for the development of Web-based services for healthcare consumers.

AB - BACKGROUND: Because of cognitive limitations and lower health literacy, many elderly patients have difficulty understanding verbal medical instructions. Automatic detection of facial movements provides a nonintrusive basis for building technological tools supporting confusion detection in healthcare delivery applications on the Internet.MATERIALS AND METHODS: Twenty-four elderly participants (70-90 years old) were recorded while watching Web-based health instruction videos involving easy and complex medical terminology. Relevant fragments of the participants' facial expressions were rated by 40 medical students for perceived level of confusion and analyzed with automatic software for facial movement recognition.RESULTS: A computer classification of the automatically detected facial features performed more accurately and with a higher sensitivity than the human observers (automatic detection and classification, 64% accuracy, 0.64 sensitivity; human observers, 41% accuracy, 0.43 sensitivity). A drill-down analysis of cues to confusion indicated the importance of the eye and eyebrow region.CONCLUSIONS: Confusion caused by misunderstanding of medical terminology is signaled by facial cues that can be automatically detected with currently available facial expression detection technology. The findings are relevant for the development of Web-based services for healthcare consumers.

U2 - 10.1089/tmj.2014.0061

DO - 10.1089/tmj.2014.0061

M3 - Article

VL - 21

SP - 514

EP - 519

JO - Telemedicine and e-Health

JF - Telemedicine and e-Health

SN - 1530-5627

IS - 6

ER -