How humans impair automated deception detection performance

Bennett Kleinberg*, Bruno Verschuere

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

5 Downloads (Pure)

Abstract

Background: 

Deception detection is a prevalent problem for security practitioners. With a need for more large-scale approaches, automated methods using machine learning have gained traction. However, detection performance still implies considerable error rates. Findings from different domains suggest that hybrid human-machine integrations could offer a viable path in detection tasks. 

Method: 

We collected a corpus of truthful and deceptive answers about participants' autobiographical intentions (n = 1640) and tested whether a combination of supervised machine learning and human judgment could improve deception detection accuracy. Human judges were presented with the outcome of the automated credibility judgment of truthful or deceptive statements. They could either fully overrule it (hybrid-overrule condition) or adjust it within a given boundary (hybrid-adjust condition). 

Results: 

The data suggest that in neither of the hybrid conditions did the human judgment add a meaningful contribution. Machine learning in isolation identified truth-tellers and liars with an overall accuracy of 69%. Human involvement through hybrid-overrule decisions brought the accuracy back to chance level. The hybrid-adjust condition did not improve deception detection performance. The decision-making strategies of humans suggest that the truth bias - the tendency to assume the other is telling the truth - could explain the detrimental effect. 

Conclusions: 

The current study does not support the notion that humans can meaningfully add the deception detection performance of a machine learning system. All data are available at https://osf.io/45z7e/.

Original languageEnglish
Article number103250
Number of pages9
JournalActa Psychologica
Volume213
DOIs
Publication statusPublished - 2021

Keywords

  • Deception detection
  • Deceptive intentions
  • Decision-making
  • Machine learning
  • Truth bias

Fingerprint

Dive into the research topics of 'How humans impair automated deception detection performance'. Together they form a unique fingerprint.

Cite this