Investigating the utility of fixed-margin sampling in network psychometrics

Sacha Epskamp*, Eiko Fried, Claudia van Borkulo, Don Robinaugh, Maarten Marsman, Jonas Dalege, Mijke Rhemtulla, A.O.J. Cramer

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

5 Citations (Scopus)
4 Downloads (Pure)

Abstract

Steinley, Hoffman, Brusco, and Sher (2017) proposed a new method for evaluating the performance of psychological network models: fixed-margin sampling. The authors investigated LASSO regularized Ising models (eLasso) by generating random datasets with the same margins as the original binary dataset, and concluded that many estimated eLasso parameters are not distinguishable from those that would be expected if the data were generated by chance. We argue that fixed-margin sampling cannot be used for this purpose, as it generates data under a particular null-hypothesis: a unidimensional factor model with interchangeable indicators (i.e., the Rasch model). We show this by discussing relevant psychometric literature and by performing simulation studies. Results indicate that while eLasso correctly estimated network models and estimated almost no edges due to chance, fixed-margin sampling performed poorly in classifying true effects as “interesting” (Steinley et al. 2017, p. 1004). Further simulation studies indicate that fixed-margin sampling offers a powerful method for highlighting local misfit from the Rasch model, but performs only moderately in identifying global departures from the Rasch model. We conclude that fixed-margin sampling is not up to the task of assessing if results from estimated Ising models or other multivariate psychometric models are due to chance.
Original languageEnglish
JournalMultivariate Behavioral Research
DOIs
Publication statusAccepted/In press - 2020

Keywords

  • IRT
  • Ising models
  • exploratory data analysis
  • fixed-margin sampling
  • network Psychometrics

Fingerprint Dive into the research topics of 'Investigating the utility of fixed-margin sampling in network psychometrics'. Together they form a unique fingerprint.

  • Cite this

    Epskamp, S., Fried, E., van Borkulo, C., Robinaugh, D., Marsman, M., Dalege, J., Rhemtulla, M., & Cramer, A. O. J. (Accepted/In press). Investigating the utility of fixed-margin sampling in network psychometrics. Multivariate Behavioral Research. https://doi.org/10.1080/00273171.2018.1489771