TY - GEN
T1 - Wizard-of-Oz vs. GPT-4: A Comparative Study of Perceived Social Intelligence in HRI Brainstorming
AU - Vrins, Anita
AU - Pruss, Ethel
AU - Ceccato, Caterina
AU - Prinsen, Jos
AU - de Rooij, Alwin
AU - Alimardani, Maryam
AU - de Wit, Jan
N1 - Publisher Copyright:
© 2024 Copyright held by the owner/author(s)
PY - 2024/3/11
Y1 - 2024/3/11
N2 - Human-robot interaction often employs the Wizard-of-Oz (WoZ) paradigm, where a human controls the robot. However, this approach has limitations, such as a lack of autonomy that impedes real-world applications. Large language models (LLMs) can replace WoZ in conversational tasks, such as brainstorming. We propose that, in such application domains, LLM-controlled robots can achieve comparable perceived social intelligence to WoZ-controlled robots. An experiment (n=27, within-subject design) tested this by having participants brainstorm with an LLM- and WoZ-controlled Furhat robot. Bayesian analyses revealed substantial evidence for the null model for perceived social intelligence, social presentation, and social information processing, indicating similar perceptions of social intelligence for WoZ- and LLM-controlled robots. Participants tentatively preferred the LLM-controlled robot, and reliably identified when the robot was WoZ- or LLM-controlled. This study highlights the potential of LLMs to replace the WoZ paradigm and transform HRI in various research and application domains.
AB - Human-robot interaction often employs the Wizard-of-Oz (WoZ) paradigm, where a human controls the robot. However, this approach has limitations, such as a lack of autonomy that impedes real-world applications. Large language models (LLMs) can replace WoZ in conversational tasks, such as brainstorming. We propose that, in such application domains, LLM-controlled robots can achieve comparable perceived social intelligence to WoZ-controlled robots. An experiment (n=27, within-subject design) tested this by having participants brainstorm with an LLM- and WoZ-controlled Furhat robot. Bayesian analyses revealed substantial evidence for the null model for perceived social intelligence, social presentation, and social information processing, indicating similar perceptions of social intelligence for WoZ- and LLM-controlled robots. Participants tentatively preferred the LLM-controlled robot, and reliably identified when the robot was WoZ- or LLM-controlled. This study highlights the potential of LLMs to replace the WoZ paradigm and transform HRI in various research and application domains.
KW - Human-robot interaction
KW - Wizard-of-oz
KW - Large Language Models
KW - ChatGPT
KW - Brainstorming
KW - Social Intelligence
UR - http://www.scopus.com/inward/record.url?scp=85188136366&partnerID=8YFLogxK
U2 - 10.1145/3610978.3640755
DO - 10.1145/3610978.3640755
M3 - Conference contribution
T3 - ACM/IEEE International Conference on Human-Robot Interaction
SP - 1090
EP - 1094
BT - HRI '24: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction
PB - ACM
ER -