The Poverty of Epidemiology or Why Model Uncertainty Matters

Compartmental models have been widely used during the pandemic to make projections and estimate the impact of government interventions, which in turn has informed policy by presenting decision makers with various scenarios depending on what options they choose. I argue that such models have largely failed to deliver useful inputs for decision makers, but that epidemiologists have on the whole failed to acknowledge this fact, because they ascribed this failure to scenario uncertainty even though model uncertainty was much more important. In order to illustrate this point, after briefly presenting compartment models, I examine how they were used to make projections about the impact of new variants believed to be more transmissible. I argue that, in several instances, a new variant's transmissibility advantage was overestimated because epidemiologists did not take seriously enough the distinction between a transmission advantage and a transmissibility advantage. These estimates of the variant's transmissibility advantage were used to calibrate the parameters of compartmental models that in turn were used to make projections that vastly overestimated the size of the wave associated with the variant. However, instead of blaming the failure of their projections on the estimates of the variant's transmissibility advantage, epidemiologists claimed that it was because people voluntarily changed their behavior more than they had anticipated or that government interventions were more effective than expected, something they "proved" by using in a circular way the very models that, according to me, were badly misspecified not only because their parameters were incorrectly calibrated but also for more fundamental reasons I briefly discuss. I conclude that, while the failure of projections (when it was even acknowledged) was blamed on scenario uncertainty, the real culprit was model misspecification.