Towards an Integrated Framework of Bias in Noncognitive Assessment in International Large-Scale Studies: Challenges and Prospects

A conceptual framework of measurement bias in cross-cultural comparisons, distinguishing between construct, method, and item bias (differential item functioning), is used to describe a methodological framework addressing assessment of noncognitive variables in international large-scale studies. It is argued that the treatment of bias, coming from constructs, measurement procedures, items, or any combination, is often piecemeal and that the quality of studies would be enhanced by an integrated approach to all kinds of bias in all stages of a study. The extensive preparations of data collections involving feedback from all participating countries make it likely that constructs to be measured apply in most or all of the participating countries (thereby reducing the likelihood of construct bias). Large-scale studies tend to use up-to-date procedures for item bias (differential item functioning). Sources of method bias are examined much less. Particular attention is paid to response styles and possible ways to address such styles in large-scale studies. Finally, ways to enhance the integrated treatment of bias are described.