Enhancing Voter Decision-Making: The Impact of Proactive and Reactive Chatbots in Voting Advice Applications

Previous research has shown that voters often struggle to understand political statements in Voting Advice Applications (VAAs) and tend to make minimal efforts to resolve these comprehension difficulties. Conversational Agent Voting Advice Applications (CAVAAs) present a promising solution by incorporating chatbots that provide users with additional information at low cognitive cost. This study investigates an experiment conducted in the lead-up to the Dutch National Elections on November 22, 2023, comparing a standard VAA without supplementary information to two CAVAA versions: one with a proactive chatbot that initiates interactions and one with a reactive chatbot that responds to user queries.

<br><br>

A total of 148 university students were randomly assigned to one of the three conditions. Results revealed that information options in the two CAVAA conditions were used extensively, with information being requested in about 45% of the cases. Participants primarily sought opinion-based information, followed by details on the status quo, and lastly, semantic clarification of terms. This suggests that voters are not just searching for objective facts but also for argumentative insights.<br>
Comparing the two CAVAA conditions to the regular VAA, both CAVAA versions showed a reduced proportion of non-directional responses, indicating that chatbots effectively help users resolve comprehension issues. <br>Furthermore, while all tools were rated similarly in terms of ease of use and playfulness, CAVAA users found the tool more useful and they also felt better informed. These improvements were largely consistent across both proactive and reactive chatbot versions, suggesting that CAVAAs can enhance the quality of voter decision-making, regardless of the chatbot’s engagement style.

Please ask permission from the first author before accessing the data.





Data files




SPSS

Qualtrics_surveydata_Dataverse.sav
Cleaned dataset of the relevant survey data.
Chatbot_gedrag.sav

The registration of the clicks in the CAVAA


MLWIN

Model_nietdirectioneleantwoorden.wsz en Model_semantisch_Infomratie.wsz

File with the model used to analyzed the % of non-directional responses in MLWIN and the % of semantic information requested (please keep this file in the Dataverse as it contains examples of the exact statistical models reported; the information in these files is also available in chatbot_gedrag.sav so contains no new information)


Supplemental material





Materials

A Word-version of the experimental materials used
Clip

Three video clips demonstrating the chatbot can be found at this SURF Research Drive folder.


Structure of the data package



In the package we have included the experimental materials (pdf file), a video in which the tools are demonstrated (included in the metadata), the survey data and the data related to the behavior in the tool itself (.sav files).

We also like to include wsz-files including the statistical models we constructed in MLWIN. These files offer the same information as the .sav-files but researchers with an MLWIN account can open the statistical models used for analyzing the response data.

Method: Data were collected through an experimental study

Universe: The population consisted of theoretically educated students from Tilburg University

Data sources: Qualtrics and the data of the conversations have been collected through the chatbot developer (genius voice)

Country / Nation: the Netherlands