Social Robots as Creative Partners: Comparing Large Language Models with Wizard-of-Oz in Human-Robot Brainstorming

Social robots can enhance brainstorming. The frequent reliance on Wizard-of-Oz (WoZ) methods hinders the development of autonomous human-robot brainstorming interactions. Large Language Models (LLM) may help address this issue. To compare WoZ- and LLM-controlled robots, a mixed methods experiment (within-subjects, n = 27) was conducted, in which human participants brainstormed with WoZ- and LLM-controlled Furhat robots. Quantitative analysis showed substantial evidence for equality between the two conditions regarding perceived robot creativity and social intelligence; and very strong evidence for a positive relationship between participants’ self-rated creativity and perceived robot creativity and social intelligence, but only when brainstorming with the LLM-controlled robot. Qualitative analysis supported these findings and contributed areas of improvement, most notably, regarding utilizing conversational turn-taking, adaptability, and non-verbal behavior. The findings highlight the potential of LLMs to advance social robots as autonomous creative partners in real-world applications.