Insights into the accuracy of social scientists' forecasts of societal change

Igor Grossmann*, Amanda A. Rotella, Cendri Hutcherson, Konstantyn Sharpinskyi, Michael E. W. Varnum, Sebastian K. Achter, Mandeep Dhami, Xinqi Evie Guo, Mane R. Kara-Yakoubian, David Mandel, Louis Raes, Louis Tay, Aymeric Vie, Lisa Wagner, Matus Adamkovic, Arash Arami, Patricia Arriaga, Kasun Bandara, Gabriel Banik, Frantisek BartosErnest Baskin, Christoph Bergmeir, Michal K. Bialek, Caroline T. Borsting, Dillon M. Browne, Eugene Caruso, Rong Chen, Bin-Tzong J. Chie, William N. Chopik, Robert Collins, Chin Wen G. Cong, Lucian Conway, Matthew V. Davis, Martin A. Day, Nathan D. Dhaliwal, Justin Durham, Martyna T. Dziekan, Christian Elbaek, Eric Shuman, Marharyta Fabrykant, Mustafa T. Firat, Geoffrey A. Fong, Jeremy M. Frimer, Jonathan B. Gallegos, Simon Goldberg, Anton Gollwitzer, Julia Goyal, Lorenz D. Graf-Vlachy, Scott Gronlund, Sebastian Hafenbraedl, Andree J. Hartanto, Matthew J. Hirshberg, Matthew Hornsey, Piers D. L. Howe, Anoosha Izadi, Bastian Jaeger, Pavol Kacmar, Yeun Joon Kim, Ruslan G. Krenzler, Daniel Lannin, Hung-Wen Lin, Nigel Mantou Lou, Verity Y. Q. W. Lua, Aaron L. Lukaszewski, Albert R. Ly, Christopher Madan, Maximilian M. Maier, Nadyanna S. Majeed, David A. March, Abigail Marsh, Michal Misiak, Kristian Ove R. M. Myrseth, Jaime Napan, Jonathan Nicholas, Konstantinos Nikolopoulos, Tobias Otterbring, Mariola Paruzel-Czachura, Shiva Pauer, John Protzko, Quentin Raffaelli, Ivan Ropovik, Robert M. Ross, Yefim Roth, Espen Roysamb, Landon Schnabel, Astrid Schuetz, Matthias Seifert, A. T. Sevincer, Garrick T. Sherman, Otto Simonsson, Ming-Chien Sung, Chung-Ching Tai, Thomas Talhelm, Bethany A. Teachman, Philip E. Tetlock, Dimitrios Thomakos, Dwight C. K. Tse, Oliver J. Twardus, Joshua M. Tybur, Lyle Ungar, Daan Vandermeulen, Leighton Vaughan Williams, Hrag A. Vosgerichian, Qi Wang, Ke Wang, Mark E. Whiting, Conny E. Wollbrant, Tao Yang, Kumar Yogeeswaran, Sangsuk Yoon, Ventura R. Alves, Jessica R. Andrews-Hanna, Paul A. Bloom, Anthony Boyles, Loo Charis, Mingyeong Choi, Sean Darling-Hammond, Z. E. Ferguson, Cheryl R. Kaiser, Simon T. Karg, Alberto Lopez Ortega, Lori Mahoney, Melvin S. Marsh, Marcellin F. R. C. Martinie, Eli K. Michaels, Philip Millroth, Jeanean B. Naqvi, Weiting Ng, Robb B. Rutledge, Peter Slattery, Adam H. Smiley, Oliver Strijbis, Daniel Sznycer, Eli Tsukayama, Austin van Loon, Jan G. Voelkel, Margaux N. A. Wienk, Tom Wilkening

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

24 Citations (Scopus)
66 Downloads (Pure)

Abstract

How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender-career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists' forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data.

Original languageEnglish
Pages (from-to)484-501
JournalNature Human Behaviour
Volume7
Issue number4
DOIs
Publication statusPublished - 2023

Keywords

  • IMPLICIT
  • ATTITUDES

Fingerprint

Dive into the research topics of 'Insights into the accuracy of social scientists' forecasts of societal change'. Together they form a unique fingerprint.

Cite this