I don’t get it: Response difficulties in answering political attitude statements in Voting Advice Applications

Naomi Kamoen, Bregje Holleman

Research output: Contribution to conferenceAbstractOther research output

Abstract

Political knowledge and feelings of political competence are essential for political participation. Voting Advice Applications (VAAs) aim to contribute to that. In a VAA, users respond to a set of attitude statements about political issues. Based on a comparison between the users’ answers and the parties’ issue positions, the VAA subsequently provides a voting advice.
VAAs have become a central source of political information in many European countries. As there is no body of knowledge on how political opinion questions in these popular tools are understood, research into VAA usage is urgently called for. The current research is the first to investigate to what extent VAA users understand the VAA statements that lead to the voting advice, and what search and response behaviour they expose in case of comprehension problems. These questions are investigated using both qualitative (Study 1) and quantitative (Study 2) methodologies.

Method and Results Study 1
In Study 1, 60 participants (55% female; M age 28.7) verbalized their thoughts while filling out a VAA during the Utrecht Municipal Elections of 2014. These verbalizations were scored for a set of comprehension problems by two coders (e.g., political jargon; Kappa/ Kappa Max between 0.58 and 0.98).
Results show that users encounter comprehension problems for about 1 in every 5 questions. About two-thirds of these relate to the semantic question meaning, covering difficulties with political jargon (e.g., referendum), tax names (e.g., tax on waste) geographical locations (e.g., in the polder Rijnenburg). In addition, one-third of the problems relate to the pragmatic question meaning. In these cases, VAA users lack contextual knowledge necessary for providing an answer. Such pragmatic problems are often triggered by vague quantifying term in the question, for example “more money should be spent on culture” leads to verbalizations like “I don’t know how much money is currently spend”.
In case of comprehension problems, VAA users often fill in the gaps of missing information by making assumptions about the question meaning, rather than by looking for factual information on the web. Failing to look for additional information, VAA users proceed by supplying a non-substantive No Opinion answer (about 25% of the times) or by supplying a substantive answer on the ‘agree’–‘disagree’ dimension (75% of the cases). Most of these substantive answers are Neutral answers, which indicates that VAA users wrongly assume that the Neutral category is meant to express comprehension problems.
Original languageEnglish
Publication statusPublished - 2017
EventEtmaal van de Communicatiewetenschap: Innovative Methods in Communication Research - Tilburg University, Tilburg, Netherlands
Duration: 26 Jan 201727 Jan 2017

Conference

ConferenceEtmaal van de Communicatiewetenschap
CountryNetherlands
CityTilburg
Period26/01/1727/01/17

Fingerprint

political attitude
voting
comprehension
taxes
pragmatics
money
political opinion
response behavior
referendum
political participation
know how

Cite this

Kamoen, N., & Holleman, B. (2017). I don’t get it: Response difficulties in answering political attitude statements in Voting Advice Applications. Abstract from Etmaal van de Communicatiewetenschap, Tilburg, Netherlands.
Kamoen, Naomi ; Holleman, Bregje. / I don’t get it : Response difficulties in answering political attitude statements in Voting Advice Applications. Abstract from Etmaal van de Communicatiewetenschap, Tilburg, Netherlands.
@conference{e5ea0a47264a496ebd297cebd53f40b5,
title = "I don’t get it: Response difficulties in answering political attitude statements in Voting Advice Applications",
abstract = "Political knowledge and feelings of political competence are essential for political participation. Voting Advice Applications (VAAs) aim to contribute to that. In a VAA, users respond to a set of attitude statements about political issues. Based on a comparison between the users’ answers and the parties’ issue positions, the VAA subsequently provides a voting advice. VAAs have become a central source of political information in many European countries. As there is no body of knowledge on how political opinion questions in these popular tools are understood, research into VAA usage is urgently called for. The current research is the first to investigate to what extent VAA users understand the VAA statements that lead to the voting advice, and what search and response behaviour they expose in case of comprehension problems. These questions are investigated using both qualitative (Study 1) and quantitative (Study 2) methodologies.Method and Results Study 1In Study 1, 60 participants (55{\%} female; M age 28.7) verbalized their thoughts while filling out a VAA during the Utrecht Municipal Elections of 2014. These verbalizations were scored for a set of comprehension problems by two coders (e.g., political jargon; Kappa/ Kappa Max between 0.58 and 0.98). Results show that users encounter comprehension problems for about 1 in every 5 questions. About two-thirds of these relate to the semantic question meaning, covering difficulties with political jargon (e.g., referendum), tax names (e.g., tax on waste) geographical locations (e.g., in the polder Rijnenburg). In addition, one-third of the problems relate to the pragmatic question meaning. In these cases, VAA users lack contextual knowledge necessary for providing an answer. Such pragmatic problems are often triggered by vague quantifying term in the question, for example “more money should be spent on culture” leads to verbalizations like “I don’t know how much money is currently spend”. In case of comprehension problems, VAA users often fill in the gaps of missing information by making assumptions about the question meaning, rather than by looking for factual information on the web. Failing to look for additional information, VAA users proceed by supplying a non-substantive No Opinion answer (about 25{\%} of the times) or by supplying a substantive answer on the ‘agree’–‘disagree’ dimension (75{\%} of the cases). Most of these substantive answers are Neutral answers, which indicates that VAA users wrongly assume that the Neutral category is meant to express comprehension problems.",
author = "Naomi Kamoen and Bregje Holleman",
year = "2017",
language = "English",
note = "Etmaal van de Communicatiewetenschap : Innovative Methods in Communication Research ; Conference date: 26-01-2017 Through 27-01-2017",

}

Kamoen, N & Holleman, B 2017, 'I don’t get it: Response difficulties in answering political attitude statements in Voting Advice Applications' Etmaal van de Communicatiewetenschap, Tilburg, Netherlands, 26/01/17 - 27/01/17, .

I don’t get it : Response difficulties in answering political attitude statements in Voting Advice Applications. / Kamoen, Naomi; Holleman, Bregje.

2017. Abstract from Etmaal van de Communicatiewetenschap, Tilburg, Netherlands.

Research output: Contribution to conferenceAbstractOther research output

TY - CONF

T1 - I don’t get it

T2 - Response difficulties in answering political attitude statements in Voting Advice Applications

AU - Kamoen, Naomi

AU - Holleman, Bregje

PY - 2017

Y1 - 2017

N2 - Political knowledge and feelings of political competence are essential for political participation. Voting Advice Applications (VAAs) aim to contribute to that. In a VAA, users respond to a set of attitude statements about political issues. Based on a comparison between the users’ answers and the parties’ issue positions, the VAA subsequently provides a voting advice. VAAs have become a central source of political information in many European countries. As there is no body of knowledge on how political opinion questions in these popular tools are understood, research into VAA usage is urgently called for. The current research is the first to investigate to what extent VAA users understand the VAA statements that lead to the voting advice, and what search and response behaviour they expose in case of comprehension problems. These questions are investigated using both qualitative (Study 1) and quantitative (Study 2) methodologies.Method and Results Study 1In Study 1, 60 participants (55% female; M age 28.7) verbalized their thoughts while filling out a VAA during the Utrecht Municipal Elections of 2014. These verbalizations were scored for a set of comprehension problems by two coders (e.g., political jargon; Kappa/ Kappa Max between 0.58 and 0.98). Results show that users encounter comprehension problems for about 1 in every 5 questions. About two-thirds of these relate to the semantic question meaning, covering difficulties with political jargon (e.g., referendum), tax names (e.g., tax on waste) geographical locations (e.g., in the polder Rijnenburg). In addition, one-third of the problems relate to the pragmatic question meaning. In these cases, VAA users lack contextual knowledge necessary for providing an answer. Such pragmatic problems are often triggered by vague quantifying term in the question, for example “more money should be spent on culture” leads to verbalizations like “I don’t know how much money is currently spend”. In case of comprehension problems, VAA users often fill in the gaps of missing information by making assumptions about the question meaning, rather than by looking for factual information on the web. Failing to look for additional information, VAA users proceed by supplying a non-substantive No Opinion answer (about 25% of the times) or by supplying a substantive answer on the ‘agree’–‘disagree’ dimension (75% of the cases). Most of these substantive answers are Neutral answers, which indicates that VAA users wrongly assume that the Neutral category is meant to express comprehension problems.

AB - Political knowledge and feelings of political competence are essential for political participation. Voting Advice Applications (VAAs) aim to contribute to that. In a VAA, users respond to a set of attitude statements about political issues. Based on a comparison between the users’ answers and the parties’ issue positions, the VAA subsequently provides a voting advice. VAAs have become a central source of political information in many European countries. As there is no body of knowledge on how political opinion questions in these popular tools are understood, research into VAA usage is urgently called for. The current research is the first to investigate to what extent VAA users understand the VAA statements that lead to the voting advice, and what search and response behaviour they expose in case of comprehension problems. These questions are investigated using both qualitative (Study 1) and quantitative (Study 2) methodologies.Method and Results Study 1In Study 1, 60 participants (55% female; M age 28.7) verbalized their thoughts while filling out a VAA during the Utrecht Municipal Elections of 2014. These verbalizations were scored for a set of comprehension problems by two coders (e.g., political jargon; Kappa/ Kappa Max between 0.58 and 0.98). Results show that users encounter comprehension problems for about 1 in every 5 questions. About two-thirds of these relate to the semantic question meaning, covering difficulties with political jargon (e.g., referendum), tax names (e.g., tax on waste) geographical locations (e.g., in the polder Rijnenburg). In addition, one-third of the problems relate to the pragmatic question meaning. In these cases, VAA users lack contextual knowledge necessary for providing an answer. Such pragmatic problems are often triggered by vague quantifying term in the question, for example “more money should be spent on culture” leads to verbalizations like “I don’t know how much money is currently spend”. In case of comprehension problems, VAA users often fill in the gaps of missing information by making assumptions about the question meaning, rather than by looking for factual information on the web. Failing to look for additional information, VAA users proceed by supplying a non-substantive No Opinion answer (about 25% of the times) or by supplying a substantive answer on the ‘agree’–‘disagree’ dimension (75% of the cases). Most of these substantive answers are Neutral answers, which indicates that VAA users wrongly assume that the Neutral category is meant to express comprehension problems.

M3 - Abstract

ER -

Kamoen N, Holleman B. I don’t get it: Response difficulties in answering political attitude statements in Voting Advice Applications. 2017. Abstract from Etmaal van de Communicatiewetenschap, Tilburg, Netherlands.