“When expert predictions fail”

Abstract: How well do social scientists capture the dynamic nature of behavioral and societal processes? How accurate are they at estimating social change when relying on their expert causal models? To address these questions, I will examine whether and how social scientists made predictions about societal change during the COVID pandemic, and introduce a crowdsourcing framework - the Behavioral and Social Science forecasting Collaborate – through which my colleagues and I have systematically evaluated accuracy of a host of domains at the heart of social sciences. The results showed that expert predictions of societal change were often no more accurate than those of laypeople or simple statistical models, and that relying on specifical causal models did not improve their forecasting accuracy. At the same time, the public expected social scientists to perform better in forecasting pandemic-related societal change than other experts, highlighting a perception-accuracy gap. Why? I argue that many causal models in social sciences are oversimplified, misaligned with the phenomena they aim to explain, and too narrowly focused on specific theories. To address these issues, like theorists before me, I advocate for more precise and bold predictions, greater intellectual humility, and the integration of simulation-based computational models to improve forecasting accuracy in social sciences. Together, I hope we can discuss value of prediction (vs. post-hoc explanations) and ways to increase intuitions and reasoning about social change.