Establishing the accuracy of online panels for survey research

Many surveys being conducted today for academic research, government policy-making, and marketing collect data via the Internet from groups of respondents who volunteered to answer questions regularly, rather than from random samples of individuals who were selected using the scientific methods that have dominated survey research for decades. This paper compares the accuracy of results obtained from 18 such opt-in online “panels” with the results obtained from respondents selected randomly from the population who answered questions either via the Internet or via face-to-face interviewing. The nonprobability samples yielded less accurate estimates of proportions and notably different relations between variables than did the probability samples, and these differences were not eliminated by weighting. These findings reinforce the value of scientific, random sampling to permit generalizing research findings to a larger population. These findings suggest that the marketing community should pay more attention to and provide elaborate and honest descriptions of the nature of survey samples, to allow consumers of the data to assess their likely accuracy.