On enhancing the cross-cultural comparability of Likert-Scale Personality and Value Measures: A comparison of common procedures: Enhancing comparability

J. He, Fons J.r. Van De Vijver, Velichko H. Fetvadjiev, Alejandra De Carmen Dominguez Espinosa, B.G. Adams, Itziar Alonso-arbiol, Arzu Aydinli-karakulak, Carmen Buzea, Radosveta Dimitrova, Alvaro Fortin, Given Hapunda, Sang Ma, Ruta Sargautyte, Samantha Sim, Maja K. Schachner, Angela Suryani, Pia Zeinoun, Rui Zhang, René Mõttus (Editor)

Research output: Contribution to journalArticleScientificpeer-review

Abstract

This study aims to evaluate a number of procedures that have been proposed to enhance cross-cultural comparability of personality and value data. A priori procedures (anchoring vignettes and direct measures of response styles (i.e. acquiescence, extremity, midpoint responding, and social desirability), a posteriori procedures focusing on data transformations prior to analysis (ipsatization and item parcelling), and two data modelling procedures (treating data as continuous vs as ordered categories) were compared using data collected from university students in 16 countries. We found that (i) anchoring vignettes showed lack of invariance, so they were not bias-free; (ii) anchoring vignettes showed higher internal consistencies than raw scores where all other correction procedures, notably ipsatization, showed lower internal consistencies; (iii) in measurement invariance testing, no procedure yielded scalar invariance; anchoring vignettes and item parcelling slightly improved comparability, response style correction did not affect it, and ipsatization resulted in lower comparability; (iv) treating Likert-scale data as categorical resulted in higher levels of comparability; (v) factor scores of scales extracted from different procedures showed similar correlational patterning; and (vi) response style correction was the only procedure that suggested improvement in external validity of country-level conscientiousness. We conclude that, although no procedure resolves all comparability issues, anchoring vignettes, parcelling, and treating data as ordered categories seem promising to alleviate incomparability. We advise caution in uncritically applying any of these procedures.
Original languageEnglish
Pages (from-to)642-657
JournalEuropean Journal of Personality
Volume31
Issue number6
DOIs
Publication statusPublished - 1 Nov 2017

Cite this

He, J., Van De Vijver, F. J. R., Fetvadjiev, V. H., De Carmen Dominguez Espinosa, A., Adams, B. G., Alonso-arbiol, I., ... Mõttus, R. (Ed.) (2017). On enhancing the cross-cultural comparability of Likert-Scale Personality and Value Measures: A comparison of common procedures: Enhancing comparability. European Journal of Personality, 31(6), 642-657. https://doi.org/10.1002/per.2132
He, J. ; Van De Vijver, Fons J.r. ; Fetvadjiev, Velichko H. ; De Carmen Dominguez Espinosa, Alejandra ; Adams, B.G. ; Alonso-arbiol, Itziar ; Aydinli-karakulak, Arzu ; Buzea, Carmen ; Dimitrova, Radosveta ; Fortin, Alvaro ; Hapunda, Given ; Ma, Sang ; Sargautyte, Ruta ; Sim, Samantha ; Schachner, Maja K. ; Suryani, Angela ; Zeinoun, Pia ; Zhang, Rui ; Mõttus, René (Editor). / On enhancing the cross-cultural comparability of Likert-Scale Personality and Value Measures: A comparison of common procedures : Enhancing comparability. In: European Journal of Personality. 2017 ; Vol. 31, No. 6. pp. 642-657.
@article{1fcc923a2ee94f0480e3cafe02682006,
title = "On enhancing the cross-cultural comparability of Likert-Scale Personality and Value Measures: A comparison of common procedures: Enhancing comparability",
abstract = "This study aims to evaluate a number of procedures that have been proposed to enhance cross-cultural comparability of personality and value data. A priori procedures (anchoring vignettes and direct measures of response styles (i.e. acquiescence, extremity, midpoint responding, and social desirability), a posteriori procedures focusing on data transformations prior to analysis (ipsatization and item parcelling), and two data modelling procedures (treating data as continuous vs as ordered categories) were compared using data collected from university students in 16 countries. We found that (i) anchoring vignettes showed lack of invariance, so they were not bias-free; (ii) anchoring vignettes showed higher internal consistencies than raw scores where all other correction procedures, notably ipsatization, showed lower internal consistencies; (iii) in measurement invariance testing, no procedure yielded scalar invariance; anchoring vignettes and item parcelling slightly improved comparability, response style correction did not affect it, and ipsatization resulted in lower comparability; (iv) treating Likert-scale data as categorical resulted in higher levels of comparability; (v) factor scores of scales extracted from different procedures showed similar correlational patterning; and (vi) response style correction was the only procedure that suggested improvement in external validity of country-level conscientiousness. We conclude that, although no procedure resolves all comparability issues, anchoring vignettes, parcelling, and treating data as ordered categories seem promising to alleviate incomparability. We advise caution in uncritically applying any of these procedures.",
author = "J. He and {Van De Vijver}, {Fons J.r.} and Fetvadjiev, {Velichko H.} and {De Carmen Dominguez Espinosa}, Alejandra and B.G. Adams and Itziar Alonso-arbiol and Arzu Aydinli-karakulak and Carmen Buzea and Radosveta Dimitrova and Alvaro Fortin and Given Hapunda and Sang Ma and Ruta Sargautyte and Samantha Sim and Schachner, {Maja K.} and Angela Suryani and Pia Zeinoun and Rui Zhang and Ren{\'e} M{\~o}ttus",
year = "2017",
month = "11",
day = "1",
doi = "10.1002/per.2132",
language = "English",
volume = "31",
pages = "642--657",
journal = "European Journal of Personality",
issn = "0890-2070",
publisher = "Wiley-Blackwell",
number = "6",

}

He, J, Van De Vijver, FJR, Fetvadjiev, VH, De Carmen Dominguez Espinosa, A, Adams, BG, Alonso-arbiol, I, Aydinli-karakulak, A, Buzea, C, Dimitrova, R, Fortin, A, Hapunda, G, Ma, S, Sargautyte, R, Sim, S, Schachner, MK, Suryani, A, Zeinoun, P, Zhang, R & Mõttus, R (ed.) 2017, 'On enhancing the cross-cultural comparability of Likert-Scale Personality and Value Measures: A comparison of common procedures: Enhancing comparability', European Journal of Personality, vol. 31, no. 6, pp. 642-657. https://doi.org/10.1002/per.2132

On enhancing the cross-cultural comparability of Likert-Scale Personality and Value Measures: A comparison of common procedures : Enhancing comparability. / He, J.; Van De Vijver, Fons J.r.; Fetvadjiev, Velichko H.; De Carmen Dominguez Espinosa, Alejandra; Adams, B.G.; Alonso-arbiol, Itziar; Aydinli-karakulak, Arzu; Buzea, Carmen; Dimitrova, Radosveta; Fortin, Alvaro; Hapunda, Given; Ma, Sang; Sargautyte, Ruta; Sim, Samantha; Schachner, Maja K.; Suryani, Angela; Zeinoun, Pia; Zhang, Rui; Mõttus, René (Editor).

In: European Journal of Personality, Vol. 31, No. 6, 01.11.2017, p. 642-657.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - On enhancing the cross-cultural comparability of Likert-Scale Personality and Value Measures: A comparison of common procedures

T2 - Enhancing comparability

AU - He, J.

AU - Van De Vijver, Fons J.r.

AU - Fetvadjiev, Velichko H.

AU - De Carmen Dominguez Espinosa, Alejandra

AU - Adams, B.G.

AU - Alonso-arbiol, Itziar

AU - Aydinli-karakulak, Arzu

AU - Buzea, Carmen

AU - Dimitrova, Radosveta

AU - Fortin, Alvaro

AU - Hapunda, Given

AU - Ma, Sang

AU - Sargautyte, Ruta

AU - Sim, Samantha

AU - Schachner, Maja K.

AU - Suryani, Angela

AU - Zeinoun, Pia

AU - Zhang, Rui

A2 - Mõttus, René

PY - 2017/11/1

Y1 - 2017/11/1

N2 - This study aims to evaluate a number of procedures that have been proposed to enhance cross-cultural comparability of personality and value data. A priori procedures (anchoring vignettes and direct measures of response styles (i.e. acquiescence, extremity, midpoint responding, and social desirability), a posteriori procedures focusing on data transformations prior to analysis (ipsatization and item parcelling), and two data modelling procedures (treating data as continuous vs as ordered categories) were compared using data collected from university students in 16 countries. We found that (i) anchoring vignettes showed lack of invariance, so they were not bias-free; (ii) anchoring vignettes showed higher internal consistencies than raw scores where all other correction procedures, notably ipsatization, showed lower internal consistencies; (iii) in measurement invariance testing, no procedure yielded scalar invariance; anchoring vignettes and item parcelling slightly improved comparability, response style correction did not affect it, and ipsatization resulted in lower comparability; (iv) treating Likert-scale data as categorical resulted in higher levels of comparability; (v) factor scores of scales extracted from different procedures showed similar correlational patterning; and (vi) response style correction was the only procedure that suggested improvement in external validity of country-level conscientiousness. We conclude that, although no procedure resolves all comparability issues, anchoring vignettes, parcelling, and treating data as ordered categories seem promising to alleviate incomparability. We advise caution in uncritically applying any of these procedures.

AB - This study aims to evaluate a number of procedures that have been proposed to enhance cross-cultural comparability of personality and value data. A priori procedures (anchoring vignettes and direct measures of response styles (i.e. acquiescence, extremity, midpoint responding, and social desirability), a posteriori procedures focusing on data transformations prior to analysis (ipsatization and item parcelling), and two data modelling procedures (treating data as continuous vs as ordered categories) were compared using data collected from university students in 16 countries. We found that (i) anchoring vignettes showed lack of invariance, so they were not bias-free; (ii) anchoring vignettes showed higher internal consistencies than raw scores where all other correction procedures, notably ipsatization, showed lower internal consistencies; (iii) in measurement invariance testing, no procedure yielded scalar invariance; anchoring vignettes and item parcelling slightly improved comparability, response style correction did not affect it, and ipsatization resulted in lower comparability; (iv) treating Likert-scale data as categorical resulted in higher levels of comparability; (v) factor scores of scales extracted from different procedures showed similar correlational patterning; and (vi) response style correction was the only procedure that suggested improvement in external validity of country-level conscientiousness. We conclude that, although no procedure resolves all comparability issues, anchoring vignettes, parcelling, and treating data as ordered categories seem promising to alleviate incomparability. We advise caution in uncritically applying any of these procedures.

U2 - 10.1002/per.2132

DO - 10.1002/per.2132

M3 - Article

VL - 31

SP - 642

EP - 657

JO - European Journal of Personality

JF - European Journal of Personality

SN - 0890-2070

IS - 6

ER -