Design of web questionnaires: The effect of layout in rating scales

This article shows that respondents gain meaning from verbal cues (words) as well as nonverbal cues (layout; numbers) in a web survey. We manipulated the layout of a five-point rating scale in two experiments. In the first experiment, we compared answers for different presentations of the responses: in one column with separate rows for each answer (“linear”), in three columns and two rows (“nonlinear”) in various orders, and after adding numerical labels to each response option. Our results show significant differences between a linear and nonlinear layout of response options. In the second experiment we looked at effects of verbal, graphical, and numerical language. We compared two linear vertical layouts with reverse orderings (from positive to negative and from negative to positive), a horizontal layout, and layouts with various numerical labels (1 to 5, 5 to 1, and 2 to 22). We found effects of verbal and graphical language. The effect of numerical language was only apparent when the numbers 2 to 22 were added to the verbal labels. We also examined whether the effects of design vary with personal characteristics. Elderly respondents appeared to be more sensitive to verbal, graphical, and numerical language.