Rating versus ranking: What is the best way to reduce response and language bias in cross-national research?

Anne-Wil Harzing*, Joyce Baldueza, Wilhelm Barner-Rasmussen, Cordula Barzantny, Anne Canabal, Anabella Davila, Alvaro Espejo, Rita Ferreira, Axele Giroud, Kathrin Koester, Yung-Kuei Liang, Audra Mockaitis, Michael J. Morley, Barbara Myloni, Joseph O. T. Odusanya, Sharon Leiba O'Sullivan, Ananda Kumar Palaniappan, Paulo Prochno, Srabani Roy Choudhury, Ayse Saka-HelmhoutSununta Siengthai, Linda Viswat, Ayda Uzuncarsili Soydas, Lena Zander

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

Abstract

We propose solutions to two recurring problems in cross-national research: response style differences and language bias. In order to do so, we conduct a methodological comparison of two different response formats-rating and ranking. For rating, we assess the effect of changing the commonly used 5-point Likert scales to 7-point Likert scales. For ranking, we evaluate the validity of presenting respondents with short scenarios for which they need to rank their top 3 solutions. Our results - based on two studies of 1965 undergraduate and 1714 MBA students in 16 different countries - confirm our hypotheses that both solutions reduce response and language bias, but show that ranking generally is a superior solution. These findings allow researchers to have greater confidence in the validity of cross-national differences if these response formats are used, instead of the more traditional 5-point Likert scales. In addition, our findings have several practical implications for multinational corporations, relating to issues such as selection interviews, performance appraisals, and cross-cultural training. (C) 2009 Elsevier Ltd. All rights reserved.

Original languageEnglish
Pages (from-to)417-432
Number of pages16
JournalInternational Business Review
Volume18
Issue number4
DOIs
Publication statusPublished - Aug 2009
Externally publishedYes

Keywords

  • Cross-national research
  • Research methods
  • Response style differences
  • Language bias
  • Survey research
  • CULTURAL ACCOMMODATION
  • HONG-KONG
  • STYLE
  • VALUES
  • PERFORMANCE
  • DIMENSIONS
  • ISSUES
  • IMPACT

Cite this

Harzing, Anne-Wil ; Baldueza, Joyce ; Barner-Rasmussen, Wilhelm ; Barzantny, Cordula ; Canabal, Anne ; Davila, Anabella ; Espejo, Alvaro ; Ferreira, Rita ; Giroud, Axele ; Koester, Kathrin ; Liang, Yung-Kuei ; Mockaitis, Audra ; Morley, Michael J. ; Myloni, Barbara ; Odusanya, Joseph O. T. ; O'Sullivan, Sharon Leiba ; Palaniappan, Ananda Kumar ; Prochno, Paulo ; Choudhury, Srabani Roy ; Saka-Helmhout, Ayse ; Siengthai, Sununta ; Viswat, Linda ; Soydas, Ayda Uzuncarsili ; Zander, Lena. / Rating versus ranking : What is the best way to reduce response and language bias in cross-national research?. In: International Business Review. 2009 ; Vol. 18, No. 4. pp. 417-432.
@article{19859c9948d844b190f8e76a4860bc2c,
title = "Rating versus ranking: What is the best way to reduce response and language bias in cross-national research?",
abstract = "We propose solutions to two recurring problems in cross-national research: response style differences and language bias. In order to do so, we conduct a methodological comparison of two different response formats-rating and ranking. For rating, we assess the effect of changing the commonly used 5-point Likert scales to 7-point Likert scales. For ranking, we evaluate the validity of presenting respondents with short scenarios for which they need to rank their top 3 solutions. Our results - based on two studies of 1965 undergraduate and 1714 MBA students in 16 different countries - confirm our hypotheses that both solutions reduce response and language bias, but show that ranking generally is a superior solution. These findings allow researchers to have greater confidence in the validity of cross-national differences if these response formats are used, instead of the more traditional 5-point Likert scales. In addition, our findings have several practical implications for multinational corporations, relating to issues such as selection interviews, performance appraisals, and cross-cultural training. (C) 2009 Elsevier Ltd. All rights reserved.",
keywords = "Cross-national research, Research methods, Response style differences, Language bias, Survey research, CULTURAL ACCOMMODATION, HONG-KONG, STYLE, VALUES, PERFORMANCE, DIMENSIONS, ISSUES, IMPACT",
author = "Anne-Wil Harzing and Joyce Baldueza and Wilhelm Barner-Rasmussen and Cordula Barzantny and Anne Canabal and Anabella Davila and Alvaro Espejo and Rita Ferreira and Axele Giroud and Kathrin Koester and Yung-Kuei Liang and Audra Mockaitis and Morley, {Michael J.} and Barbara Myloni and Odusanya, {Joseph O. T.} and O'Sullivan, {Sharon Leiba} and Palaniappan, {Ananda Kumar} and Paulo Prochno and Choudhury, {Srabani Roy} and Ayse Saka-Helmhout and Sununta Siengthai and Linda Viswat and Soydas, {Ayda Uzuncarsili} and Lena Zander",
year = "2009",
month = "8",
doi = "10.1016/j.ibusrev.2009.03.001",
language = "English",
volume = "18",
pages = "417--432",
journal = "International Business Review",
issn = "0969-5931",
publisher = "Elsevier Science BV",
number = "4",

}

Harzing, A-W, Baldueza, J, Barner-Rasmussen, W, Barzantny, C, Canabal, A, Davila, A, Espejo, A, Ferreira, R, Giroud, A, Koester, K, Liang, Y-K, Mockaitis, A, Morley, MJ, Myloni, B, Odusanya, JOT, O'Sullivan, SL, Palaniappan, AK, Prochno, P, Choudhury, SR, Saka-Helmhout, A, Siengthai, S, Viswat, L, Soydas, AU & Zander, L 2009, 'Rating versus ranking: What is the best way to reduce response and language bias in cross-national research?', International Business Review, vol. 18, no. 4, pp. 417-432. https://doi.org/10.1016/j.ibusrev.2009.03.001

Rating versus ranking : What is the best way to reduce response and language bias in cross-national research? / Harzing, Anne-Wil; Baldueza, Joyce; Barner-Rasmussen, Wilhelm; Barzantny, Cordula; Canabal, Anne; Davila, Anabella; Espejo, Alvaro; Ferreira, Rita; Giroud, Axele; Koester, Kathrin; Liang, Yung-Kuei; Mockaitis, Audra; Morley, Michael J.; Myloni, Barbara; Odusanya, Joseph O. T.; O'Sullivan, Sharon Leiba; Palaniappan, Ananda Kumar; Prochno, Paulo; Choudhury, Srabani Roy; Saka-Helmhout, Ayse; Siengthai, Sununta; Viswat, Linda; Soydas, Ayda Uzuncarsili; Zander, Lena.

In: International Business Review, Vol. 18, No. 4, 08.2009, p. 417-432.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Rating versus ranking

T2 - What is the best way to reduce response and language bias in cross-national research?

AU - Harzing, Anne-Wil

AU - Baldueza, Joyce

AU - Barner-Rasmussen, Wilhelm

AU - Barzantny, Cordula

AU - Canabal, Anne

AU - Davila, Anabella

AU - Espejo, Alvaro

AU - Ferreira, Rita

AU - Giroud, Axele

AU - Koester, Kathrin

AU - Liang, Yung-Kuei

AU - Mockaitis, Audra

AU - Morley, Michael J.

AU - Myloni, Barbara

AU - Odusanya, Joseph O. T.

AU - O'Sullivan, Sharon Leiba

AU - Palaniappan, Ananda Kumar

AU - Prochno, Paulo

AU - Choudhury, Srabani Roy

AU - Saka-Helmhout, Ayse

AU - Siengthai, Sununta

AU - Viswat, Linda

AU - Soydas, Ayda Uzuncarsili

AU - Zander, Lena

PY - 2009/8

Y1 - 2009/8

N2 - We propose solutions to two recurring problems in cross-national research: response style differences and language bias. In order to do so, we conduct a methodological comparison of two different response formats-rating and ranking. For rating, we assess the effect of changing the commonly used 5-point Likert scales to 7-point Likert scales. For ranking, we evaluate the validity of presenting respondents with short scenarios for which they need to rank their top 3 solutions. Our results - based on two studies of 1965 undergraduate and 1714 MBA students in 16 different countries - confirm our hypotheses that both solutions reduce response and language bias, but show that ranking generally is a superior solution. These findings allow researchers to have greater confidence in the validity of cross-national differences if these response formats are used, instead of the more traditional 5-point Likert scales. In addition, our findings have several practical implications for multinational corporations, relating to issues such as selection interviews, performance appraisals, and cross-cultural training. (C) 2009 Elsevier Ltd. All rights reserved.

AB - We propose solutions to two recurring problems in cross-national research: response style differences and language bias. In order to do so, we conduct a methodological comparison of two different response formats-rating and ranking. For rating, we assess the effect of changing the commonly used 5-point Likert scales to 7-point Likert scales. For ranking, we evaluate the validity of presenting respondents with short scenarios for which they need to rank their top 3 solutions. Our results - based on two studies of 1965 undergraduate and 1714 MBA students in 16 different countries - confirm our hypotheses that both solutions reduce response and language bias, but show that ranking generally is a superior solution. These findings allow researchers to have greater confidence in the validity of cross-national differences if these response formats are used, instead of the more traditional 5-point Likert scales. In addition, our findings have several practical implications for multinational corporations, relating to issues such as selection interviews, performance appraisals, and cross-cultural training. (C) 2009 Elsevier Ltd. All rights reserved.

KW - Cross-national research

KW - Research methods

KW - Response style differences

KW - Language bias

KW - Survey research

KW - CULTURAL ACCOMMODATION

KW - HONG-KONG

KW - STYLE

KW - VALUES

KW - PERFORMANCE

KW - DIMENSIONS

KW - ISSUES

KW - IMPACT

U2 - 10.1016/j.ibusrev.2009.03.001

DO - 10.1016/j.ibusrev.2009.03.001

M3 - Article

VL - 18

SP - 417

EP - 432

JO - International Business Review

JF - International Business Review

SN - 0969-5931

IS - 4

ER -