Exploring response differences between face-to-face and web surveys: A qualitative comparative analysis of the Dutch European Values Survey 2008

Can existing longitudinal surveys profit from the (financial) advantages of web surveying by switching survey mode from face-to-face interviews to web surveys? Before such a radical change in data collection procedure can be undertaken, it needs to be established that mode effects cannot confound the responses to the survey items. To this end, the responses of the Dutch European Values Study of 2008 were compared to the responses of a time parallel web survey. The responses on 163 of the 256 items differed significantly across modes. To explain these response differences between modes, an exploratory crisp set qualitative comparative analysis approach was used. Five sufficient conditions—combinations of survey mode characteristics—but no necessary conditions for response differences between survey modes were found. Two survey characteristics were neither necessary nor sufficient to produce the outcome. Results suggest that switching modes may affect comparability between waves in a longitudinal survey.
Keywords: mode effects, face-to-face surveys, web surveys, internet surveys, crisp set qualitative comparative analysis