Investigating the utility of fixed-margin sampling in network psychometrics

Sacha Epskamp*, Eiko Fried, Claudia van Borkulo, Don Robinaugh, Maarten Marsman, Jonas Dalege, Mijke Rhemtulla, A.O.J. Cramer

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

2 Downloads (Pure)

Abstract

Steinley, Hoffman, Brusco, and Sher (2017) proposed a new method for evaluating the performance of psychological network models: fixed-margin sampling. The authors investigated LASSO regularized Ising models (eLasso) by generating random datasets with the same margins as the original binary dataset, and concluded that many estimated eLasso parameters are not distinguishable from those that would be expected if the data were generated by chance. We argue that fixed-margin sampling cannot be used for this purpose, as it generates data under a particular null-hypothesis: a unidimensional factor model with interchangeable indicators (i.e., the Rasch model). We show this by discussing relevant psychometric literature and by performing simulation studies. Results indicate that while eLasso correctly estimated network models and estimated almost no edges due to chance, fixed-margin sampling performed poorly in classifying true effects as “interesting” (Steinley et al. 2017, p. 1004). Further simulation studies indicate that fixed-margin sampling offers a powerful method for highlighting local misfit from the Rasch model, but performs only moderately in identifying global departures from the Rasch model. We conclude that fixed-margin sampling is not up to the task of assessing if results from estimated Ising models or other multivariate psychometric models are due to chance.
Original languageEnglish
JournalMultivariate Behavioral Research
DOIs
Publication statusAccepted/In press - 2019

Fingerprint

Psychometrics
Margin
Psychological Models
Rasch Model
Ising Model
Network Model
Simulation Study
Factor Models
Null hypothesis
Sampling
Binary

Cite this

Epskamp, Sacha ; Fried, Eiko ; van Borkulo, Claudia ; Robinaugh, Don ; Marsman, Maarten ; Dalege, Jonas ; Rhemtulla, Mijke ; Cramer, A.O.J. / Investigating the utility of fixed-margin sampling in network psychometrics. In: Multivariate Behavioral Research. 2019.
@article{66022c9f09344cd3894696a1a343f153,
title = "Investigating the utility of fixed-margin sampling in network psychometrics",
abstract = "Steinley, Hoffman, Brusco, and Sher (2017) proposed a new method for evaluating the performance of psychological network models: fixed-margin sampling. The authors investigated LASSO regularized Ising models (eLasso) by generating random datasets with the same margins as the original binary dataset, and concluded that many estimated eLasso parameters are not distinguishable from those that would be expected if the data were generated by chance. We argue that fixed-margin sampling cannot be used for this purpose, as it generates data under a particular null-hypothesis: a unidimensional factor model with interchangeable indicators (i.e., the Rasch model). We show this by discussing relevant psychometric literature and by performing simulation studies. Results indicate that while eLasso correctly estimated network models and estimated almost no edges due to chance, fixed-margin sampling performed poorly in classifying true effects as “interesting” (Steinley et al. 2017, p. 1004). Further simulation studies indicate that fixed-margin sampling offers a powerful method for highlighting local misfit from the Rasch model, but performs only moderately in identifying global departures from the Rasch model. We conclude that fixed-margin sampling is not up to the task of assessing if results from estimated Ising models or other multivariate psychometric models are due to chance.",
author = "Sacha Epskamp and Eiko Fried and {van Borkulo}, Claudia and Don Robinaugh and Maarten Marsman and Jonas Dalege and Mijke Rhemtulla and A.O.J. Cramer",
year = "2019",
doi = "10.1080/00273171.2018.1489771",
language = "English",
journal = "Multivariate Behavioral Research",
issn = "0027-3171",
publisher = "ROUTLEDGE JOURNALS, TAYLOR & FRANCIS LTD",

}

Investigating the utility of fixed-margin sampling in network psychometrics. / Epskamp, Sacha; Fried, Eiko; van Borkulo, Claudia; Robinaugh, Don; Marsman, Maarten; Dalege, Jonas; Rhemtulla, Mijke; Cramer, A.O.J.

In: Multivariate Behavioral Research, 2019.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Investigating the utility of fixed-margin sampling in network psychometrics

AU - Epskamp, Sacha

AU - Fried, Eiko

AU - van Borkulo, Claudia

AU - Robinaugh, Don

AU - Marsman, Maarten

AU - Dalege, Jonas

AU - Rhemtulla, Mijke

AU - Cramer, A.O.J.

PY - 2019

Y1 - 2019

N2 - Steinley, Hoffman, Brusco, and Sher (2017) proposed a new method for evaluating the performance of psychological network models: fixed-margin sampling. The authors investigated LASSO regularized Ising models (eLasso) by generating random datasets with the same margins as the original binary dataset, and concluded that many estimated eLasso parameters are not distinguishable from those that would be expected if the data were generated by chance. We argue that fixed-margin sampling cannot be used for this purpose, as it generates data under a particular null-hypothesis: a unidimensional factor model with interchangeable indicators (i.e., the Rasch model). We show this by discussing relevant psychometric literature and by performing simulation studies. Results indicate that while eLasso correctly estimated network models and estimated almost no edges due to chance, fixed-margin sampling performed poorly in classifying true effects as “interesting” (Steinley et al. 2017, p. 1004). Further simulation studies indicate that fixed-margin sampling offers a powerful method for highlighting local misfit from the Rasch model, but performs only moderately in identifying global departures from the Rasch model. We conclude that fixed-margin sampling is not up to the task of assessing if results from estimated Ising models or other multivariate psychometric models are due to chance.

AB - Steinley, Hoffman, Brusco, and Sher (2017) proposed a new method for evaluating the performance of psychological network models: fixed-margin sampling. The authors investigated LASSO regularized Ising models (eLasso) by generating random datasets with the same margins as the original binary dataset, and concluded that many estimated eLasso parameters are not distinguishable from those that would be expected if the data were generated by chance. We argue that fixed-margin sampling cannot be used for this purpose, as it generates data under a particular null-hypothesis: a unidimensional factor model with interchangeable indicators (i.e., the Rasch model). We show this by discussing relevant psychometric literature and by performing simulation studies. Results indicate that while eLasso correctly estimated network models and estimated almost no edges due to chance, fixed-margin sampling performed poorly in classifying true effects as “interesting” (Steinley et al. 2017, p. 1004). Further simulation studies indicate that fixed-margin sampling offers a powerful method for highlighting local misfit from the Rasch model, but performs only moderately in identifying global departures from the Rasch model. We conclude that fixed-margin sampling is not up to the task of assessing if results from estimated Ising models or other multivariate psychometric models are due to chance.

UR - https://app-eu.readspeaker.com/cgi-bin/rsent?customerid=10118&lang=en_us&readclass=rs_readArea&url=https%3A%2F%2Fwww.tandfonline.com%2Fdoi%2Ffull%2F10.1080%2F00273171.2018.1489771

U2 - 10.1080/00273171.2018.1489771

DO - 10.1080/00273171.2018.1489771

M3 - Article

JO - Multivariate Behavioral Research

JF - Multivariate Behavioral Research

SN - 0027-3171

ER -