Psychometric problems with the method of correlated vectors applied to item scores (including some nonsensical results)

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Spearman's hypothesis stating that ethnic group differences on cognitive tests are most pronounced on the most highly g loaded tests has been commonly tested with Jensen's method of correlated vectors (MCV). This paper illustrates and explains why MCV applied to item-level data does not provide a test of measurement invariance and fails to provide accurate information about the role of g in group differences in test scores. I focus on studies that applied MCV to study group differences on items of Raven's Standard Progressive Matrices (SPM). In an empirical illustration of the psychometric problems with this method, I show that MCV applied to 60 SPM items incorrectly yields support for Spearman's hypothesis (so-called Jensen Effects suggesting that the group difference is on g) even when the items in the second group are not from the SPM but rather from a test composed of 60 items measuring either anxiety and anger or the big five personality traits. This shows that MCV applied to item level data does not accurately reflect the degree to which item bias or g plays a role in group differences. I conclude that MCV applied to items lacks both sensitivity and specificity.
Original languageEnglish
Pages (from-to)26–38
JournalIntelligence
Volume60
DOIs
Publication statusPublished - 2017

Fingerprint

Crows
Psychometrics
Group Differences
Anger
Specificity
Big Five
Anxiety
Personality Traits
Test Scores
Invariance
Ethnic Groups

Cite this

@article{32f26f9f9d0a441da7f68c96b2669d25,
title = "Psychometric problems with the method of correlated vectors applied to item scores (including some nonsensical results)",
abstract = "Spearman's hypothesis stating that ethnic group differences on cognitive tests are most pronounced on the most highly g loaded tests has been commonly tested with Jensen's method of correlated vectors (MCV). This paper illustrates and explains why MCV applied to item-level data does not provide a test of measurement invariance and fails to provide accurate information about the role of g in group differences in test scores. I focus on studies that applied MCV to study group differences on items of Raven's Standard Progressive Matrices (SPM). In an empirical illustration of the psychometric problems with this method, I show that MCV applied to 60 SPM items incorrectly yields support for Spearman's hypothesis (so-called Jensen Effects suggesting that the group difference is on g) even when the items in the second group are not from the SPM but rather from a test composed of 60 items measuring either anxiety and anger or the big five personality traits. This shows that MCV applied to item level data does not accurately reflect the degree to which item bias or g plays a role in group differences. I conclude that MCV applied to items lacks both sensitivity and specificity.",
author = "J.M. Wicherts",
year = "2017",
doi = "10.1016/j.intell.2016.11.002",
language = "English",
volume = "60",
pages = "26–38",
journal = "Intelligence",
issn = "0160-2896",
publisher = "Elsevier Limited",

}

Psychometric problems with the method of correlated vectors applied to item scores (including some nonsensical results). / Wicherts, J.M.

In: Intelligence, Vol. 60, 2017, p. 26–38.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Psychometric problems with the method of correlated vectors applied to item scores (including some nonsensical results)

AU - Wicherts, J.M.

PY - 2017

Y1 - 2017

N2 - Spearman's hypothesis stating that ethnic group differences on cognitive tests are most pronounced on the most highly g loaded tests has been commonly tested with Jensen's method of correlated vectors (MCV). This paper illustrates and explains why MCV applied to item-level data does not provide a test of measurement invariance and fails to provide accurate information about the role of g in group differences in test scores. I focus on studies that applied MCV to study group differences on items of Raven's Standard Progressive Matrices (SPM). In an empirical illustration of the psychometric problems with this method, I show that MCV applied to 60 SPM items incorrectly yields support for Spearman's hypothesis (so-called Jensen Effects suggesting that the group difference is on g) even when the items in the second group are not from the SPM but rather from a test composed of 60 items measuring either anxiety and anger or the big five personality traits. This shows that MCV applied to item level data does not accurately reflect the degree to which item bias or g plays a role in group differences. I conclude that MCV applied to items lacks both sensitivity and specificity.

AB - Spearman's hypothesis stating that ethnic group differences on cognitive tests are most pronounced on the most highly g loaded tests has been commonly tested with Jensen's method of correlated vectors (MCV). This paper illustrates and explains why MCV applied to item-level data does not provide a test of measurement invariance and fails to provide accurate information about the role of g in group differences in test scores. I focus on studies that applied MCV to study group differences on items of Raven's Standard Progressive Matrices (SPM). In an empirical illustration of the psychometric problems with this method, I show that MCV applied to 60 SPM items incorrectly yields support for Spearman's hypothesis (so-called Jensen Effects suggesting that the group difference is on g) even when the items in the second group are not from the SPM but rather from a test composed of 60 items measuring either anxiety and anger or the big five personality traits. This shows that MCV applied to item level data does not accurately reflect the degree to which item bias or g plays a role in group differences. I conclude that MCV applied to items lacks both sensitivity and specificity.

U2 - 10.1016/j.intell.2016.11.002

DO - 10.1016/j.intell.2016.11.002

M3 - Article

VL - 60

SP - 26

EP - 38

JO - Intelligence

JF - Intelligence

SN - 0160-2896

ER -