Analysis of cross-cultural comparability of PISA 2009 scores

M. Kankaras, G.B.D. Moors

Research output: Contribution to journalArticleScientificpeer-review

22 Citations (Scopus)
244 Downloads (Pure)

Abstract

The Program for International Student Assessment (PISA) is a large-scale cross-national study that measures academic competencies of 15-year-old students in mathematics, reading, and science from more than 50 countries/economies around the world. PISA results are usually aggregated and presented in so-called “league tables,” in which countries are compared and ranked in each of the three scales. However, to compare results obtained from different groups/countries, one must first be sure that the tests measure the same competencies in all cultures. In this paper, this is tested by examining the level of measurement equivalence in the 2009 PISA data set using an item response theory approach (IRT) and analyzing differential item functioning (DIF). Measurement in-equivalence was found in the form of uniform DIF. In-equivalence occurred in a majority of test questions in all three scales researched and is, on average, of moderate size. It varies considerably both across items and across countries. When this uniform DIF is accounted for in the in-equivalent model, the resulting country scores change considerably in the cases of the “Mathematics,” “Science,” and especially, “Reading” scale. These changes tend to occur simultaneously and in the same direction in groups of regional countries. The most affected seems to be Southeast Asian countries/territories whose scores, although among the highest in the initial, homogeneous model, additionally increase when accounting for in-equivalence in the scales.
Keywords: measurement equivalence, PISA, differential item functioning, cross-cultural research, educational measurement
Original languageEnglish
Pages (from-to)381-399
JournalJournal of Cross-Cultural Psychology
Volume45
Issue number3
DOIs
Publication statusPublished - 2014

Fingerprint Dive into the research topics of 'Analysis of cross-cultural comparability of PISA 2009 scores'. Together they form a unique fingerprint.

Cite this