TY - JOUR
T1 - Bias-variance trade-off in continuous test norming
AU - Voncken, Lieke
AU - Albers, Casper
AU - Timmerman, Marieke
N1 - Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the Dutch Research Council (NWO) within research programme ‘Graduate Programme 2013’ with project number 022.005.003.
PY - 2021
Y1 - 2021
N2 - In continuous test norming, the test score distribution is estimated as a continuous function of predictor(s). A flexible approach for norm estimation is the use of generalized additive models for location, scale, and shape. It is unknown how sensitive their estimates are to model flexibility and sample size. Generally, a flexible model that fits at the population level has smaller bias than its restricted nonfitting version, yet it has larger sampling variability. We investigated how model flexibility relates to bias, variance, and total variability in estimates of normalized z scores under empirically relevant conditions, involving the skew Student t and normal distributions as population distributions. We considered both transversal and longitudinal assumption violations. We found that models with too strict distributional assumptions yield biased estimates, whereas too flexible models yield increased variance. The skew Student t distribution, unlike the Box–Cox Power Exponential distribution, appeared problematic to estimate for normally distributed data. Recommendations for empirical norming practice are provided.
AB - In continuous test norming, the test score distribution is estimated as a continuous function of predictor(s). A flexible approach for norm estimation is the use of generalized additive models for location, scale, and shape. It is unknown how sensitive their estimates are to model flexibility and sample size. Generally, a flexible model that fits at the population level has smaller bias than its restricted nonfitting version, yet it has larger sampling variability. We investigated how model flexibility relates to bias, variance, and total variability in estimates of normalized z scores under empirically relevant conditions, involving the skew Student t and normal distributions as population distributions. We considered both transversal and longitudinal assumption violations. We found that models with too strict distributional assumptions yield biased estimates, whereas too flexible models yield increased variance. The skew Student t distribution, unlike the Box–Cox Power Exponential distribution, appeared problematic to estimate for normally distributed data. Recommendations for empirical norming practice are provided.
KW - ABILITIES
KW - GAMLSS
KW - GROWTH
KW - REGRESSION
KW - assumption violations
KW - model assumptions
KW - model flexibility
KW - skew Studenttdistribution
KW - standard linear regression model
UR - http://www.scopus.com/inward/record.url?scp=85087861130&partnerID=8YFLogxK
U2 - 10.1177/1073191120939155
DO - 10.1177/1073191120939155
M3 - Article
C2 - 32659111
SN - 1073-1911
VL - 28
SP - 1932
EP - 1948
JO - Assessment
JF - Assessment
IS - 8
ER -