I don't get it: Response difficulties in answering political attitude questions in Voting Advice Applications

Naomi Kamoen, Bregje Holleman

Research output: Contribution to journalArticleScientificpeer-review

34 Downloads (Pure)

Abstract

Voting Advice Applications are online tools that provide users with a voting advice based on their answers to a set of political attitude questions. This study investigated to what extent VAA users understand the questions that lead to the voting advice, and what search and response behaviour they expose in case of comprehension difficulties. Two studies were conducted to investigate these issues: a cognitive interviewing study among 60 VAA users during the Dutch municipal elections in the city of Utrecht, and a statistical analysis of all answers provided by 357,858 users who accessed one of the 34 municipal VAAs during these same elections. Results of the two studies show a coherent picture: difficult concepts (e.g., tax names or municipal jargon), geographical locations (e.g., reference to a specific street), and vague quantifying terms (e.g., “more”) all complicate the question. In case of comprehension difficulties, Study 1 shows that VAA users make little effort to solve their problems, for example by looking up difficult terms on the Internet. Instead, they draw inferences about what the question might mean and proceed to answer nonetheless. These are often neutral or no opinion answers, which seems to suggest that the meanings of those options are confounded. In Study 2, however, we found that the choice for either a neutral or no opinion response is not accidental: semantic meaning problems often result in no opinion answers, whereas pragmatic problems are related to neutral responses. We discuss the implications of these findings for survey theory and practice.
Original languageEnglish
Pages (from-to)125-140
Number of pages15
JournalSurvey Research Methods
Volume11
Issue number2
DOIs
Publication statusPublished - 2017

Fingerprint

political attitude
voting
comprehension
election
response behavior
taxes
statistical analysis
pragmatics
semantics
Internet

Cite this

@article{3c3ea721f8a64fd196618383e3dfc7b5,
title = "I don't get it: Response difficulties in answering political attitude questions in Voting Advice Applications",
abstract = "Voting Advice Applications are online tools that provide users with a voting advice based on their answers to a set of political attitude questions. This study investigated to what extent VAA users understand the questions that lead to the voting advice, and what search and response behaviour they expose in case of comprehension difficulties. Two studies were conducted to investigate these issues: a cognitive interviewing study among 60 VAA users during the Dutch municipal elections in the city of Utrecht, and a statistical analysis of all answers provided by 357,858 users who accessed one of the 34 municipal VAAs during these same elections. Results of the two studies show a coherent picture: difficult concepts (e.g., tax names or municipal jargon), geographical locations (e.g., reference to a specific street), and vague quantifying terms (e.g., “more”) all complicate the question. In case of comprehension difficulties, Study 1 shows that VAA users make little effort to solve their problems, for example by looking up difficult terms on the Internet. Instead, they draw inferences about what the question might mean and proceed to answer nonetheless. These are often neutral or no opinion answers, which seems to suggest that the meanings of those options are confounded. In Study 2, however, we found that the choice for either a neutral or no opinion response is not accidental: semantic meaning problems often result in no opinion answers, whereas pragmatic problems are related to neutral responses. We discuss the implications of these findings for survey theory and practice.",
author = "Naomi Kamoen and Bregje Holleman",
year = "2017",
doi = "10.18148/srm/2017.v11i2.6728",
language = "English",
volume = "11",
pages = "125--140",
journal = "Survey Research Methods",
issn = "1864-3361",
publisher = "European Survey Research Association",
number = "2",

}

I don't get it : Response difficulties in answering political attitude questions in Voting Advice Applications. / Kamoen, Naomi; Holleman, Bregje.

In: Survey Research Methods, Vol. 11, No. 2, 2017, p. 125-140.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - I don't get it

T2 - Response difficulties in answering political attitude questions in Voting Advice Applications

AU - Kamoen, Naomi

AU - Holleman, Bregje

PY - 2017

Y1 - 2017

N2 - Voting Advice Applications are online tools that provide users with a voting advice based on their answers to a set of political attitude questions. This study investigated to what extent VAA users understand the questions that lead to the voting advice, and what search and response behaviour they expose in case of comprehension difficulties. Two studies were conducted to investigate these issues: a cognitive interviewing study among 60 VAA users during the Dutch municipal elections in the city of Utrecht, and a statistical analysis of all answers provided by 357,858 users who accessed one of the 34 municipal VAAs during these same elections. Results of the two studies show a coherent picture: difficult concepts (e.g., tax names or municipal jargon), geographical locations (e.g., reference to a specific street), and vague quantifying terms (e.g., “more”) all complicate the question. In case of comprehension difficulties, Study 1 shows that VAA users make little effort to solve their problems, for example by looking up difficult terms on the Internet. Instead, they draw inferences about what the question might mean and proceed to answer nonetheless. These are often neutral or no opinion answers, which seems to suggest that the meanings of those options are confounded. In Study 2, however, we found that the choice for either a neutral or no opinion response is not accidental: semantic meaning problems often result in no opinion answers, whereas pragmatic problems are related to neutral responses. We discuss the implications of these findings for survey theory and practice.

AB - Voting Advice Applications are online tools that provide users with a voting advice based on their answers to a set of political attitude questions. This study investigated to what extent VAA users understand the questions that lead to the voting advice, and what search and response behaviour they expose in case of comprehension difficulties. Two studies were conducted to investigate these issues: a cognitive interviewing study among 60 VAA users during the Dutch municipal elections in the city of Utrecht, and a statistical analysis of all answers provided by 357,858 users who accessed one of the 34 municipal VAAs during these same elections. Results of the two studies show a coherent picture: difficult concepts (e.g., tax names or municipal jargon), geographical locations (e.g., reference to a specific street), and vague quantifying terms (e.g., “more”) all complicate the question. In case of comprehension difficulties, Study 1 shows that VAA users make little effort to solve their problems, for example by looking up difficult terms on the Internet. Instead, they draw inferences about what the question might mean and proceed to answer nonetheless. These are often neutral or no opinion answers, which seems to suggest that the meanings of those options are confounded. In Study 2, however, we found that the choice for either a neutral or no opinion response is not accidental: semantic meaning problems often result in no opinion answers, whereas pragmatic problems are related to neutral responses. We discuss the implications of these findings for survey theory and practice.

U2 - 10.18148/srm/2017.v11i2.6728

DO - 10.18148/srm/2017.v11i2.6728

M3 - Article

VL - 11

SP - 125

EP - 140

JO - Survey Research Methods

JF - Survey Research Methods

SN - 1864-3361

IS - 2

ER -