Addressing methodological challenges in culture-comparative research

Ronald Fischer*, Ype H. Poortinga

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

Abstract

We address methodological challenges in cross-cultural and cultural psychology. First, we describe weaknesses in (quasi-)experimental designs, noting that cross-cultural designs typically do not allow any conclusive evidence of causality. Second, we argue that loose adherence to methodological principles of psychology and a focus on differences, while neglecting similarities, is distorting the literature. We highlight the importance of effect sizes and discuss the role of Bayesian statistics and meta-analysis for cross-cultural research. Third, we highlight issues of measurement bias and lack of equivalence, but note that recent large-scale projects involving researchers across many countries from the beginning of a study have much potential for overcoming biases and improving standards of equivalence. Fourth, we address some implications of multilevel models. Cultural processes are multilevel by definition and recent statistical advances can be used to explore these issues further. We believe this is an area where much theoretical work needs to be done and more rigorous methods applied. Fifth, we argue that the definition of culture and the psychological organization of cross-cultural differences as well as the definition of cultural populations to which research findings are generalized requires more attention. Sixth, we address the scope for anchoring cross-cultural research in biological variables and by asking multiple questions simultaneously, as advocated by Tinbergen for classical ethology. Bringing these discussions together, we provide recommendations for enhancing the methodological strength of culture-comparative studies to advance cross-cultural psychology as a scientific discipline.

Original languageEnglish
Pages (from-to)691-712
JournalJournal of Cross-Cultural Psychology
Volume49
Issue number5
DOIs
Publication statusPublished - 2018

Keywords

  • methodology
  • cultural psychology
  • psychodiagnostics
  • measurement
  • statistics
  • SIMPSONS PARADOX
  • PERSONALITY
  • PSYCHOLOGY
  • EQUIVALENCE
  • REFLECTIONS
  • UNIVERSALS
  • CONSTRUCTS
  • LANGUAGES
  • COGNITION
  • BEHAVIOR

Cite this

@article{8261dcd3ef96483d88db249aad287c6d,
title = "Addressing methodological challenges in culture-comparative research",
abstract = "We address methodological challenges in cross-cultural and cultural psychology. First, we describe weaknesses in (quasi-)experimental designs, noting that cross-cultural designs typically do not allow any conclusive evidence of causality. Second, we argue that loose adherence to methodological principles of psychology and a focus on differences, while neglecting similarities, is distorting the literature. We highlight the importance of effect sizes and discuss the role of Bayesian statistics and meta-analysis for cross-cultural research. Third, we highlight issues of measurement bias and lack of equivalence, but note that recent large-scale projects involving researchers across many countries from the beginning of a study have much potential for overcoming biases and improving standards of equivalence. Fourth, we address some implications of multilevel models. Cultural processes are multilevel by definition and recent statistical advances can be used to explore these issues further. We believe this is an area where much theoretical work needs to be done and more rigorous methods applied. Fifth, we argue that the definition of culture and the psychological organization of cross-cultural differences as well as the definition of cultural populations to which research findings are generalized requires more attention. Sixth, we address the scope for anchoring cross-cultural research in biological variables and by asking multiple questions simultaneously, as advocated by Tinbergen for classical ethology. Bringing these discussions together, we provide recommendations for enhancing the methodological strength of culture-comparative studies to advance cross-cultural psychology as a scientific discipline.",
keywords = "methodology, cultural psychology, psychodiagnostics, measurement, statistics, SIMPSONS PARADOX, PERSONALITY, PSYCHOLOGY, EQUIVALENCE, REFLECTIONS, UNIVERSALS, CONSTRUCTS, LANGUAGES, COGNITION, BEHAVIOR",
author = "Ronald Fischer and Poortinga, {Ype H.}",
year = "2018",
doi = "10.1177/0022022117738086",
language = "English",
volume = "49",
pages = "691--712",
journal = "Journal of Cross-Cultural Psychology",
issn = "0022-0221",
publisher = "Sage Publications, Inc.",
number = "5",

}

Addressing methodological challenges in culture-comparative research. / Fischer, Ronald; Poortinga, Ype H.

In: Journal of Cross-Cultural Psychology, Vol. 49, No. 5, 2018, p. 691-712.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Addressing methodological challenges in culture-comparative research

AU - Fischer, Ronald

AU - Poortinga, Ype H.

PY - 2018

Y1 - 2018

N2 - We address methodological challenges in cross-cultural and cultural psychology. First, we describe weaknesses in (quasi-)experimental designs, noting that cross-cultural designs typically do not allow any conclusive evidence of causality. Second, we argue that loose adherence to methodological principles of psychology and a focus on differences, while neglecting similarities, is distorting the literature. We highlight the importance of effect sizes and discuss the role of Bayesian statistics and meta-analysis for cross-cultural research. Third, we highlight issues of measurement bias and lack of equivalence, but note that recent large-scale projects involving researchers across many countries from the beginning of a study have much potential for overcoming biases and improving standards of equivalence. Fourth, we address some implications of multilevel models. Cultural processes are multilevel by definition and recent statistical advances can be used to explore these issues further. We believe this is an area where much theoretical work needs to be done and more rigorous methods applied. Fifth, we argue that the definition of culture and the psychological organization of cross-cultural differences as well as the definition of cultural populations to which research findings are generalized requires more attention. Sixth, we address the scope for anchoring cross-cultural research in biological variables and by asking multiple questions simultaneously, as advocated by Tinbergen for classical ethology. Bringing these discussions together, we provide recommendations for enhancing the methodological strength of culture-comparative studies to advance cross-cultural psychology as a scientific discipline.

AB - We address methodological challenges in cross-cultural and cultural psychology. First, we describe weaknesses in (quasi-)experimental designs, noting that cross-cultural designs typically do not allow any conclusive evidence of causality. Second, we argue that loose adherence to methodological principles of psychology and a focus on differences, while neglecting similarities, is distorting the literature. We highlight the importance of effect sizes and discuss the role of Bayesian statistics and meta-analysis for cross-cultural research. Third, we highlight issues of measurement bias and lack of equivalence, but note that recent large-scale projects involving researchers across many countries from the beginning of a study have much potential for overcoming biases and improving standards of equivalence. Fourth, we address some implications of multilevel models. Cultural processes are multilevel by definition and recent statistical advances can be used to explore these issues further. We believe this is an area where much theoretical work needs to be done and more rigorous methods applied. Fifth, we argue that the definition of culture and the psychological organization of cross-cultural differences as well as the definition of cultural populations to which research findings are generalized requires more attention. Sixth, we address the scope for anchoring cross-cultural research in biological variables and by asking multiple questions simultaneously, as advocated by Tinbergen for classical ethology. Bringing these discussions together, we provide recommendations for enhancing the methodological strength of culture-comparative studies to advance cross-cultural psychology as a scientific discipline.

KW - methodology

KW - cultural psychology

KW - psychodiagnostics

KW - measurement

KW - statistics

KW - SIMPSONS PARADOX

KW - PERSONALITY

KW - PSYCHOLOGY

KW - EQUIVALENCE

KW - REFLECTIONS

KW - UNIVERSALS

KW - CONSTRUCTS

KW - LANGUAGES

KW - COGNITION

KW - BEHAVIOR

U2 - 10.1177/0022022117738086

DO - 10.1177/0022022117738086

M3 - Article

VL - 49

SP - 691

EP - 712

JO - Journal of Cross-Cultural Psychology

JF - Journal of Cross-Cultural Psychology

SN - 0022-0221

IS - 5

ER -