Wizard-of-Oz vs. GPT-4: A Comparative Study of Perceived Social Intelligence in HRI Brainstorming

Anita Vrins*, Ethel Pruss, Caterina Ceccato, Jos Prinsen, Alwin de Rooij, Maryam Alimardani, Jan de Wit

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

2 Citations (Scopus)
19 Downloads (Pure)

Abstract

Human-robot interaction often employs the Wizard-of-Oz (WoZ) paradigm, where a human controls the robot. However, this approach has limitations, such as a lack of autonomy that impedes real-world applications. Large language models (LLMs) can replace WoZ in conversational tasks, such as brainstorming. We propose that, in such application domains, LLM-controlled robots can achieve comparable perceived social intelligence to WoZ-controlled robots. An experiment (n=27, within-subject design) tested this by having participants brainstorm with an LLM- and WoZ-controlled Furhat robot. Bayesian analyses revealed substantial evidence for the null model for perceived social intelligence, social presentation, and social information processing, indicating similar perceptions of social intelligence for WoZ- and LLM-controlled robots. Participants tentatively preferred the LLM-controlled robot, and reliably identified when the robot was WoZ- or LLM-controlled. This study highlights the potential of LLMs to replace the WoZ paradigm and transform HRI in various research and application domains.
Original languageEnglish
Title of host publicationHRI '24: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction
PublisherACM
Pages1090–1094
Number of pages5
ISBN (Electronic)9798400703232
DOIs
Publication statusPublished - 11 Mar 2024

Publication series

NameACM/IEEE International Conference on Human-Robot Interaction
ISSN (Electronic)2167-2148

Keywords

  • Human-robot interaction
  • Wizard-of-oz
  • Large Language Models
  • ChatGPT
  • Brainstorming
  • Social Intelligence

Fingerprint

Dive into the research topics of 'Wizard-of-Oz vs. GPT-4: A Comparative Study of Perceived Social Intelligence in HRI Brainstorming'. Together they form a unique fingerprint.

Cite this