Broken windows, mediocre methods, and substandard statistics

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Broken windows theory states that cues of inappropriate behavior like litter or graffiti amplify norm-violating behavior. In a series of quasi-experiments, Keizer, Lindenberg, and Steg altered cues of inappropriate behavior in public places and observed how many passersby subsequently violated norms. They concluded that particular norm violations transgress to other misdemeanors (e.g., graffiti leads to littering or even theft) and that the presence of prohibition signs heightens the saliency of norm violations, thereby aggravating the negative effects of cues such as litter and graffiti. We raise several methodological and statistical issues that cast doubt on Keizer et al.’s results. Problems include confounding factors, observer bias, lacking scoring protocols, a failure to establish interobserver reliabilities, inflated Type I error rates due to dependencies, sequential testing, and multiple testing. We highlight results of a highly similar study that does not support the notion that prohibition signs aggravate the effects of observed norm violations. We discuss potential improvements of the paradigm.
Keywords: descriptive norms, littering behavior, methodological critique, statistics, theft
Original languageEnglish
Pages (from-to)388-403
JournalGroup Processes & Intergroup Relations: GPIR
Volume17
DOIs
Publication statusPublished - 2014

Fingerprint

norm violation
graffiti
statistics
Statistics
Cues
larceny
Testing
misdemeanor
Observer Variation
Experiments
paradigm
experiment
trend
Graffiti
Violations

Cite this

@article{8bc1151d54e7466a954858536e8002d7,
title = "Broken windows, mediocre methods, and substandard statistics",
abstract = "Broken windows theory states that cues of inappropriate behavior like litter or graffiti amplify norm-violating behavior. In a series of quasi-experiments, Keizer, Lindenberg, and Steg altered cues of inappropriate behavior in public places and observed how many passersby subsequently violated norms. They concluded that particular norm violations transgress to other misdemeanors (e.g., graffiti leads to littering or even theft) and that the presence of prohibition signs heightens the saliency of norm violations, thereby aggravating the negative effects of cues such as litter and graffiti. We raise several methodological and statistical issues that cast doubt on Keizer et al.’s results. Problems include confounding factors, observer bias, lacking scoring protocols, a failure to establish interobserver reliabilities, inflated Type I error rates due to dependencies, sequential testing, and multiple testing. We highlight results of a highly similar study that does not support the notion that prohibition signs aggravate the effects of observed norm violations. We discuss potential improvements of the paradigm.Keywords: descriptive norms, littering behavior, methodological critique, statistics, theft",
author = "J.M. Wicherts and M. Bakker",
year = "2014",
doi = "10.1177/1368430213502557",
language = "English",
volume = "17",
pages = "388--403",
journal = "Group Processes & Intergroup Relations: GPIR",
issn = "1368-4302",
publisher = "Sage Publications Ltd",

}

Broken windows, mediocre methods, and substandard statistics. / Wicherts, J.M.; Bakker, M.

In: Group Processes & Intergroup Relations: GPIR, Vol. 17, 2014, p. 388-403.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Broken windows, mediocre methods, and substandard statistics

AU - Wicherts, J.M.

AU - Bakker, M.

PY - 2014

Y1 - 2014

N2 - Broken windows theory states that cues of inappropriate behavior like litter or graffiti amplify norm-violating behavior. In a series of quasi-experiments, Keizer, Lindenberg, and Steg altered cues of inappropriate behavior in public places and observed how many passersby subsequently violated norms. They concluded that particular norm violations transgress to other misdemeanors (e.g., graffiti leads to littering or even theft) and that the presence of prohibition signs heightens the saliency of norm violations, thereby aggravating the negative effects of cues such as litter and graffiti. We raise several methodological and statistical issues that cast doubt on Keizer et al.’s results. Problems include confounding factors, observer bias, lacking scoring protocols, a failure to establish interobserver reliabilities, inflated Type I error rates due to dependencies, sequential testing, and multiple testing. We highlight results of a highly similar study that does not support the notion that prohibition signs aggravate the effects of observed norm violations. We discuss potential improvements of the paradigm.Keywords: descriptive norms, littering behavior, methodological critique, statistics, theft

AB - Broken windows theory states that cues of inappropriate behavior like litter or graffiti amplify norm-violating behavior. In a series of quasi-experiments, Keizer, Lindenberg, and Steg altered cues of inappropriate behavior in public places and observed how many passersby subsequently violated norms. They concluded that particular norm violations transgress to other misdemeanors (e.g., graffiti leads to littering or even theft) and that the presence of prohibition signs heightens the saliency of norm violations, thereby aggravating the negative effects of cues such as litter and graffiti. We raise several methodological and statistical issues that cast doubt on Keizer et al.’s results. Problems include confounding factors, observer bias, lacking scoring protocols, a failure to establish interobserver reliabilities, inflated Type I error rates due to dependencies, sequential testing, and multiple testing. We highlight results of a highly similar study that does not support the notion that prohibition signs aggravate the effects of observed norm violations. We discuss potential improvements of the paradigm.Keywords: descriptive norms, littering behavior, methodological critique, statistics, theft

U2 - 10.1177/1368430213502557

DO - 10.1177/1368430213502557

M3 - Article

VL - 17

SP - 388

EP - 403

JO - Group Processes & Intergroup Relations: GPIR

JF - Group Processes & Intergroup Relations: GPIR

SN - 1368-4302

ER -