Improving confidence intervals for normed test scores: Include uncertainty due to sampling variability

Lieke Voncken*, Casper J. Albers, Marieke E. Timmerman

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Test publishers usually provide confidence intervals (CIs) for normed test scores that reflect the uncertainty due to the unreliability of the tests. The uncertainty due to sampling variability in the norming phase is ignored. To express uncertainty due to norming, we propose a flexible method that is applicable in continuous norming and allows for a variety of score distributions, using Generalized Additive Models for Location, Scale, and Shape (GAMLSS; Rigby & Stasinopoulos, 2005). We assessed the performance of this method in a simulation study, by examining the quality of the resulting CIs. We varied the population model, procedure of estimating the CI, confidence level, sample size, value of the predictor, extremity of the test score, and type of variance-covariance matrix. The results showed that good quality of the CIs could be achieved in most conditions. The method is illustrated using normative data of the SON-R 6-40 test. We recommend test developers to use this approach to arrive at CIs, and thus properly express the uncertainty due to norm sampling fluctuations, in the context of continuous norming. Adopting this approach will help (e.g., clinical) practitioners to obtain a fair picture of the person assessed.
Original languageEnglish
Pages (from-to)826-839
JournalBehavior Research Methods
Volume51
Issue number2
DOIs
Publication statusPublished - 2019
Externally publishedYes

Fingerprint

Confidence Intervals
Sampling
Test Scores
Uncertainty
Confidence Interval

Keywords

  • continuous norming
  • GAMLSS
  • Box-Cox power exponential distribution
  • posterior simulation
  • psychological tests
  • Confidence Intervals
  • Data Interpretation, Statistical
  • Sample Size
  • Uncertainty
  • Humans
  • Psychological Tests
  • Continuous norming
  • Posterior simulation
  • Psychological tests
  • CORRELATION MATRIX

Cite this

@article{4ee0980c3af84a73b3ef6012a4b9d91d,
title = "Improving confidence intervals for normed test scores: Include uncertainty due to sampling variability",
abstract = "Test publishers usually provide confidence intervals (CIs) for normed test scores that reflect the uncertainty due to the unreliability of the tests. The uncertainty due to sampling variability in the norming phase is ignored. To express uncertainty due to norming, we propose a flexible method that is applicable in continuous norming and allows for a variety of score distributions, using Generalized Additive Models for Location, Scale, and Shape (GAMLSS; Rigby & Stasinopoulos, 2005). We assessed the performance of this method in a simulation study, by examining the quality of the resulting CIs. We varied the population model, procedure of estimating the CI, confidence level, sample size, value of the predictor, extremity of the test score, and type of variance-covariance matrix. The results showed that good quality of the CIs could be achieved in most conditions. The method is illustrated using normative data of the SON-R 6-40 test. We recommend test developers to use this approach to arrive at CIs, and thus properly express the uncertainty due to norm sampling fluctuations, in the context of continuous norming. Adopting this approach will help (e.g., clinical) practitioners to obtain a fair picture of the person assessed.",
keywords = "continuous norming, GAMLSS, Box-Cox power exponential distribution, posterior simulation, psychological tests, Confidence Intervals, Data Interpretation, Statistical, Sample Size, Uncertainty, Humans, Psychological Tests, Continuous norming, Posterior simulation, Psychological tests, CORRELATION MATRIX",
author = "Lieke Voncken and Albers, {Casper J.} and Timmerman, {Marieke E.}",
year = "2019",
doi = "10.3758/s13428-018-1122-8",
language = "English",
volume = "51",
pages = "826--839",
journal = "Behavior Research Methods",
issn = "1554-351X",
publisher = "Springer",
number = "2",

}

Improving confidence intervals for normed test scores : Include uncertainty due to sampling variability. / Voncken, Lieke; Albers, Casper J.; Timmerman, Marieke E.

In: Behavior Research Methods, Vol. 51, No. 2, 2019, p. 826-839.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Improving confidence intervals for normed test scores

T2 - Include uncertainty due to sampling variability

AU - Voncken, Lieke

AU - Albers, Casper J.

AU - Timmerman, Marieke E.

PY - 2019

Y1 - 2019

N2 - Test publishers usually provide confidence intervals (CIs) for normed test scores that reflect the uncertainty due to the unreliability of the tests. The uncertainty due to sampling variability in the norming phase is ignored. To express uncertainty due to norming, we propose a flexible method that is applicable in continuous norming and allows for a variety of score distributions, using Generalized Additive Models for Location, Scale, and Shape (GAMLSS; Rigby & Stasinopoulos, 2005). We assessed the performance of this method in a simulation study, by examining the quality of the resulting CIs. We varied the population model, procedure of estimating the CI, confidence level, sample size, value of the predictor, extremity of the test score, and type of variance-covariance matrix. The results showed that good quality of the CIs could be achieved in most conditions. The method is illustrated using normative data of the SON-R 6-40 test. We recommend test developers to use this approach to arrive at CIs, and thus properly express the uncertainty due to norm sampling fluctuations, in the context of continuous norming. Adopting this approach will help (e.g., clinical) practitioners to obtain a fair picture of the person assessed.

AB - Test publishers usually provide confidence intervals (CIs) for normed test scores that reflect the uncertainty due to the unreliability of the tests. The uncertainty due to sampling variability in the norming phase is ignored. To express uncertainty due to norming, we propose a flexible method that is applicable in continuous norming and allows for a variety of score distributions, using Generalized Additive Models for Location, Scale, and Shape (GAMLSS; Rigby & Stasinopoulos, 2005). We assessed the performance of this method in a simulation study, by examining the quality of the resulting CIs. We varied the population model, procedure of estimating the CI, confidence level, sample size, value of the predictor, extremity of the test score, and type of variance-covariance matrix. The results showed that good quality of the CIs could be achieved in most conditions. The method is illustrated using normative data of the SON-R 6-40 test. We recommend test developers to use this approach to arrive at CIs, and thus properly express the uncertainty due to norm sampling fluctuations, in the context of continuous norming. Adopting this approach will help (e.g., clinical) practitioners to obtain a fair picture of the person assessed.

KW - continuous norming

KW - GAMLSS

KW - Box-Cox power exponential distribution

KW - posterior simulation

KW - psychological tests

KW - Confidence Intervals

KW - Data Interpretation, Statistical

KW - Sample Size

KW - Uncertainty

KW - Humans

KW - Psychological Tests

KW - Continuous norming

KW - Posterior simulation

KW - Psychological tests

KW - CORRELATION MATRIX

U2 - 10.3758/s13428-018-1122-8

DO - 10.3758/s13428-018-1122-8

M3 - Article

C2 - 30402815

VL - 51

SP - 826

EP - 839

JO - Behavior Research Methods

JF - Behavior Research Methods

SN - 1554-351X

IS - 2

ER -