Differential Item Functioning and educational risk factors in Guatemalan reading assessment

We examined Differential Item Functioning (DIF) indicators for four variables that repeatedly have been demonstrated to constitute risk factors in primary school achievement in Guatemala. These factors are over-age of enrollment, urban/rural area of residence, ethnicity, and gender. We used scores from national reading assessments in third-grade for this study. Given the instability often reported in DIF literature, we employed three different approaches (chi-square, Rasch, and logistic regression) and checked for their consistency with data from three calendar years. We found substantial evidence of DIF. However, removal of DIF items did not influence differences in test scores between groups.
Findings suggest that educational risk factors act in concert in this Guatemalan population and, that that at least to some degree, they interact to create bias. We conclude that DIF analysis and test-writing would benefit from taking into account multiple background risk variables simultaneously.
Keywords: differential item functioning, reading assessment, multiple risk factors, Guatemala.