Detecting which variables alter component interpretation across multiple groups

A resampling-based method

Sopiko Gvaladze*, Kim De Roover, Francis Tuerlinckx, Eva Ceulemans

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

Abstract

In psychology, many studies measure the same variables in different groups. In the case of a large number of variables when a strong a priori idea about the underlying latent construct is lacking, researchers often start by reducing the variables to a few principal components in an exploratory way. Herewith, one often wants to evaluate whether the components represent the same construct in the different groups. To this end, it makes sense to remove outlying variables that have significantly different loadings on the extracted components across the groups, hampering equivalent interpretations of the components. Moreover, identifying such outlying variables is important when testing theories about which variables behave similarly or differently across groups. In this article, we first scrutinize the lower bound congruence method (LBCM; De Roover, Timmerman, & Ceulemans in Behavior Research Methods, 49, 216-229, 2017), which was recently proposed for solving the outlying-variable detection problem. LBCM investigates how Tucker's congruence between the loadings of the obtained cluster-loading matrices improves when specific variables are discarded. We show that LBCM has the tendency to output outlying variables that either are false positives or concern very small, and thus practically insignificant, loading differences. To address this issue, we present a new heuristic: the lower and resampled upper bound congruence method (LRUBCM). This method uses a resampling technique to obtain a sampling distribution for the congruence coefficient, under the hypothesis that no outlying variable is present. In a simulation study, we show that LRUBCM outperforms LBCM. Finally, we illustrate the use of the method by means of empirical data.

Original languageEnglish
JournalBehavior Research Methods
DOIs
Publication statusE-pub ahead of print - 2019

Fingerprint

Resampling
Congruence
Simulation
Heuristics
Principal Components
Sampling
Empirical Data
Psychology
Research Methods
Testing

Cite this

@article{e38fb9e8e6cb4cfcbd83910c459a7a7d,
title = "Detecting which variables alter component interpretation across multiple groups: A resampling-based method",
abstract = "In psychology, many studies measure the same variables in different groups. In the case of a large number of variables when a strong a priori idea about the underlying latent construct is lacking, researchers often start by reducing the variables to a few principal components in an exploratory way. Herewith, one often wants to evaluate whether the components represent the same construct in the different groups. To this end, it makes sense to remove outlying variables that have significantly different loadings on the extracted components across the groups, hampering equivalent interpretations of the components. Moreover, identifying such outlying variables is important when testing theories about which variables behave similarly or differently across groups. In this article, we first scrutinize the lower bound congruence method (LBCM; De Roover, Timmerman, & Ceulemans in Behavior Research Methods, 49, 216-229, 2017), which was recently proposed for solving the outlying-variable detection problem. LBCM investigates how Tucker's congruence between the loadings of the obtained cluster-loading matrices improves when specific variables are discarded. We show that LBCM has the tendency to output outlying variables that either are false positives or concern very small, and thus practically insignificant, loading differences. To address this issue, we present a new heuristic: the lower and resampled upper bound congruence method (LRUBCM). This method uses a resampling technique to obtain a sampling distribution for the congruence coefficient, under the hypothesis that no outlying variable is present. In a simulation study, we show that LRUBCM outperforms LBCM. Finally, we illustrate the use of the method by means of empirical data.",
author = "Sopiko Gvaladze and {De Roover}, Kim and Francis Tuerlinckx and Eva Ceulemans",
year = "2019",
doi = "10.3758/s13428-019-01222-4",
language = "English",
journal = "Behavior Research Methods",
issn = "1554-351X",
publisher = "Springer",

}

Detecting which variables alter component interpretation across multiple groups : A resampling-based method. / Gvaladze, Sopiko; De Roover, Kim; Tuerlinckx, Francis; Ceulemans, Eva.

In: Behavior Research Methods, 2019.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Detecting which variables alter component interpretation across multiple groups

T2 - A resampling-based method

AU - Gvaladze, Sopiko

AU - De Roover, Kim

AU - Tuerlinckx, Francis

AU - Ceulemans, Eva

PY - 2019

Y1 - 2019

N2 - In psychology, many studies measure the same variables in different groups. In the case of a large number of variables when a strong a priori idea about the underlying latent construct is lacking, researchers often start by reducing the variables to a few principal components in an exploratory way. Herewith, one often wants to evaluate whether the components represent the same construct in the different groups. To this end, it makes sense to remove outlying variables that have significantly different loadings on the extracted components across the groups, hampering equivalent interpretations of the components. Moreover, identifying such outlying variables is important when testing theories about which variables behave similarly or differently across groups. In this article, we first scrutinize the lower bound congruence method (LBCM; De Roover, Timmerman, & Ceulemans in Behavior Research Methods, 49, 216-229, 2017), which was recently proposed for solving the outlying-variable detection problem. LBCM investigates how Tucker's congruence between the loadings of the obtained cluster-loading matrices improves when specific variables are discarded. We show that LBCM has the tendency to output outlying variables that either are false positives or concern very small, and thus practically insignificant, loading differences. To address this issue, we present a new heuristic: the lower and resampled upper bound congruence method (LRUBCM). This method uses a resampling technique to obtain a sampling distribution for the congruence coefficient, under the hypothesis that no outlying variable is present. In a simulation study, we show that LRUBCM outperforms LBCM. Finally, we illustrate the use of the method by means of empirical data.

AB - In psychology, many studies measure the same variables in different groups. In the case of a large number of variables when a strong a priori idea about the underlying latent construct is lacking, researchers often start by reducing the variables to a few principal components in an exploratory way. Herewith, one often wants to evaluate whether the components represent the same construct in the different groups. To this end, it makes sense to remove outlying variables that have significantly different loadings on the extracted components across the groups, hampering equivalent interpretations of the components. Moreover, identifying such outlying variables is important when testing theories about which variables behave similarly or differently across groups. In this article, we first scrutinize the lower bound congruence method (LBCM; De Roover, Timmerman, & Ceulemans in Behavior Research Methods, 49, 216-229, 2017), which was recently proposed for solving the outlying-variable detection problem. LBCM investigates how Tucker's congruence between the loadings of the obtained cluster-loading matrices improves when specific variables are discarded. We show that LBCM has the tendency to output outlying variables that either are false positives or concern very small, and thus practically insignificant, loading differences. To address this issue, we present a new heuristic: the lower and resampled upper bound congruence method (LRUBCM). This method uses a resampling technique to obtain a sampling distribution for the congruence coefficient, under the hypothesis that no outlying variable is present. In a simulation study, we show that LRUBCM outperforms LBCM. Finally, we illustrate the use of the method by means of empirical data.

U2 - 10.3758/s13428-019-01222-4

DO - 10.3758/s13428-019-01222-4

M3 - Article

JO - Behavior Research Methods

JF - Behavior Research Methods

SN - 1554-351X

ER -