Words, Numbers and Visual Heuristics in Web Surveys: Is there a Hierarchy of Importance?

In interpreting questions, respondents extract meaning from how the information in a questionnaire is shaped, spaced, and shaded. This makes it important to pay close attention to the arrangement of visual information on a questionnaire. Respondents follow simple heuristics in interpreting the visual features of questions. We carried out five experiments to investigate how the effect of visual heuristics affected the answers to survey questions. We varied verbal, numerical, and other visual cues such as color. In some instances the use of words helps overcome visual layout effects. In at least one instance, a fundamental difference in visual layout (violating the 'left and top means first' heuristic) influenced answers on top of word labels. This suggests that both visual and verbal languages are important. Yet sometimes one can override the other. To reduce the effect of visual cues, it is better to use fully labeled scales in survey questions.