Item-score reliability in empirical-data sets and its relationship with other item indices

E.A.O. Zijlmans, J. Tijmstra, L.A. van der Ark, K. Sijtsma

Research output: Contribution to journalArticleScientificpeer-review

4 Downloads (Pure)

Abstract

Reliability is usually estimated for a total score, but it can also be estimated for item scores. Item-score reliability can be useful to assess the repeatability of an individual item score in a group. Three methods to estimate item-score reliability are discussed, known as method MS, method λ6, and method CA. The item-score reliability methods are compared with four well-known and widely accepted item indices, which are the item-rest correlation, the item-factor loading, the item scalability, and the item discrimination. Realistic values for item-score reliability in empirical-data sets are monitored to obtain an impression of the values to be expected in other empirical-data sets. The relation between the three item-score reliability methods and the four well-known item indices are investigated. Tentatively, a minimum value for the item-score reliability methods to be used in item analysis is recommended.
Keywords Coefficient λ6, correction for attenuation, item discrimination, item-factor loading, item-rest correlation, item scalability, item-score reliability
Original languageEnglish
Article number998-1020
JournalEducational and Psychological Measurement
Volume78
Issue number6
DOIs
Publication statusPublished - 2018

Fingerprint

Scalability
Discrimination
discrimination
Relationships
Values
item analysis
Repeatability
Attenuation
Coefficient
Estimate
Group

Keywords

  • ASSOCIATION
  • Coefficient lambda(6)
  • IRT
  • JOB-SATISFACTION
  • MODEL
  • MOKKEN SCALE ANALYSIS
  • PERSONALITY
  • SINGLE-ITEM
  • VALIDITY
  • correction for attenuation
  • item discrimination
  • item scalability
  • item-factor loading
  • item-rest correlation
  • item-score reliability

Cite this

@article{59698864fab9414ba05704402c4feccc,
title = "Item-score reliability in empirical-data sets and its relationship with other item indices",
abstract = "Reliability is usually estimated for a total score, but it can also be estimated for item scores. Item-score reliability can be useful to assess the repeatability of an individual item score in a group. Three methods to estimate item-score reliability are discussed, known as method MS, method λ6, and method CA. The item-score reliability methods are compared with four well-known and widely accepted item indices, which are the item-rest correlation, the item-factor loading, the item scalability, and the item discrimination. Realistic values for item-score reliability in empirical-data sets are monitored to obtain an impression of the values to be expected in other empirical-data sets. The relation between the three item-score reliability methods and the four well-known item indices are investigated. Tentatively, a minimum value for the item-score reliability methods to be used in item analysis is recommended.Keywords Coefficient λ6, correction for attenuation, item discrimination, item-factor loading, item-rest correlation, item scalability, item-score reliability",
keywords = "ASSOCIATION, Coefficient lambda(6), IRT, JOB-SATISFACTION, MODEL, MOKKEN SCALE ANALYSIS, PERSONALITY, SINGLE-ITEM, VALIDITY, correction for attenuation, item discrimination, item scalability, item-factor loading, item-rest correlation, item-score reliability",
author = "E.A.O. Zijlmans and J. Tijmstra and {van der Ark}, L.A. and K. Sijtsma",
year = "2018",
doi = "10.1177/0013164417728358",
language = "English",
volume = "78",
journal = "Educational and Psychological Measurement",
issn = "0013-1644",
publisher = "Sage Publications, Inc.",
number = "6",

}

Item-score reliability in empirical-data sets and its relationship with other item indices. / Zijlmans, E.A.O.; Tijmstra, J.; van der Ark, L.A.; Sijtsma, K.

In: Educational and Psychological Measurement, Vol. 78, No. 6, 998-1020, 2018.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Item-score reliability in empirical-data sets and its relationship with other item indices

AU - Zijlmans, E.A.O.

AU - Tijmstra, J.

AU - van der Ark, L.A.

AU - Sijtsma, K.

PY - 2018

Y1 - 2018

N2 - Reliability is usually estimated for a total score, but it can also be estimated for item scores. Item-score reliability can be useful to assess the repeatability of an individual item score in a group. Three methods to estimate item-score reliability are discussed, known as method MS, method λ6, and method CA. The item-score reliability methods are compared with four well-known and widely accepted item indices, which are the item-rest correlation, the item-factor loading, the item scalability, and the item discrimination. Realistic values for item-score reliability in empirical-data sets are monitored to obtain an impression of the values to be expected in other empirical-data sets. The relation between the three item-score reliability methods and the four well-known item indices are investigated. Tentatively, a minimum value for the item-score reliability methods to be used in item analysis is recommended.Keywords Coefficient λ6, correction for attenuation, item discrimination, item-factor loading, item-rest correlation, item scalability, item-score reliability

AB - Reliability is usually estimated for a total score, but it can also be estimated for item scores. Item-score reliability can be useful to assess the repeatability of an individual item score in a group. Three methods to estimate item-score reliability are discussed, known as method MS, method λ6, and method CA. The item-score reliability methods are compared with four well-known and widely accepted item indices, which are the item-rest correlation, the item-factor loading, the item scalability, and the item discrimination. Realistic values for item-score reliability in empirical-data sets are monitored to obtain an impression of the values to be expected in other empirical-data sets. The relation between the three item-score reliability methods and the four well-known item indices are investigated. Tentatively, a minimum value for the item-score reliability methods to be used in item analysis is recommended.Keywords Coefficient λ6, correction for attenuation, item discrimination, item-factor loading, item-rest correlation, item scalability, item-score reliability

KW - ASSOCIATION

KW - Coefficient lambda(6)

KW - IRT

KW - JOB-SATISFACTION

KW - MODEL

KW - MOKKEN SCALE ANALYSIS

KW - PERSONALITY

KW - SINGLE-ITEM

KW - VALIDITY

KW - correction for attenuation

KW - item discrimination

KW - item scalability

KW - item-factor loading

KW - item-rest correlation

KW - item-score reliability

U2 - 10.1177/0013164417728358

DO - 10.1177/0013164417728358

M3 - Article

VL - 78

JO - Educational and Psychological Measurement

JF - Educational and Psychological Measurement

SN - 0013-1644

IS - 6

M1 - 998-1020

ER -