The Cardigan Factor: why more evidence has made COVID-19 policy harder and what to do about it.

In recent posts, I have argued that, if we want to improve the prospects for evidence-based policy making, we should pay more attention to the institutions of scientific advice than to the quantity or even the quality of the scientific research being undertaken on a subject – whether that be measured by numbers of scientists or scientific papers, patents and other evidence of IP, or the size of research budgets. The policy responses to COVID-19 have illustrated both the strengths and the limits of this argument, which, as I can hardly deny, comes from my background in political science, where the importance of institutions is a rather basic assumption of the discipline.

More specifically, however, the argument draws heavily on the work of Daniel Sarewitz, from whom I have pinched part of my title¹. In his classic article on environmental disputes, Sarewitz begins by questioning what he regards as the standard account of the proper relationship between science and policy:  we begin with uncertainty; we fund science to reduce that uncertainty; when uncertainty has been reduced far enough, we can make evidence-based recommendations to policy makers; and then we can act.  Instead, Sarewitz argues, policy attention starts as a result of a conflict of values about how to address an urgent problem; the initial effect of more science will be to increase rather than diminish the uncertainty around the problem and its proposed solutions; and the uncertainty will be exploited by the different parties to the original dispute in order to support positions that are ultimately based on values rather than facts. More science makes an evidence-based solution less rather than more likely.

The initial response to COVID-19 provides a dramatic illustration of Sarewitz’s argument. Large sums of money were immediately thrown at the science of the virus and the number of studies duly exploded.  The effect? General confusion, the retraction of a number of hastily designed and shoddily conducted studies (but not before they had achieved wide currency on the web and social media, promoted by those who wanted to believe their conclusions), and a dearth of “certainty” on which policy decisions – in this case, literally matters of life and death – could be made².  Some critical assumptions on which these decisions were initially made turned out to be false or at best only partially true – for example, the original consensus that asymptomatic cases did not exist or, if they existed, could not infect others. At the time of writing, the equally pressing question of whether the disease is primarily contracted from contaminated surfaces or directly from airborne droplets is still not clearly decided, with scientists adding to the problems of policy makers by writing earnest public letters on the subject³.

COVID-19 has also exposed the importance of beginning with value conflict to understand why science will struggle to drive policy and of having strong institutions in order to see a way forward.  Writing about the US pandemic in March⁴, Sarewitz himself was optimistic about the speed at which “political agendas fall away” in the face of the urgent need to save lives (value convergence) and  to tackle such eminently practical problems as finding enough hospital beds or keeping businesses afloat. He pointed to convergence (at that time) between some Democratic and Republican governors on what to do.  Subsequent events showed that Sarewitz was altogether too sanguine. Where value conflict and associated political agendas around, for example, the legitimate use of state power or trust in elites are prominent in the politics of the day (the US and the UK being prominent examples of each), the politicization of the pandemic continues to fester, fuelled in part by the normal uneven progress of science in reducing uncertainty through trial and, inevitably, error.

Nonetheless, some jurisdictions have undoubtedly done better than others at handling the pandemic and, since they all struggled with the same dearth of evidence about the behaviour of the virus, the explanation for success must lie elsewhere. Sarewitz, as we have already seen, believes this is because there is convergence on values such as protecting lives, making the problem what he calls a hard one (in the absence of scientific certainty) but not a complex one (where values collide). I disagree, in part, and point in addition to the importance of some key institutions of scientific advice which have served both to modulate the noise coming from the research-dollar feeding frenzy and to moderate the extremes of political conflict.  Some of the most important of these institutions turned out to be embedded in public administration in the form of Scientific and Medical Officers of Health.  Emerging from the shadowlands of public health, a generally disregarded medical speciality in contemporary North America, Chief Medical Officers, picturesquely if inaccurately labelled “top doctors” by the media, have become the public face of COVID-19 governance.

There have been well-argued opposition to these kinds of appointments in the past. The idea that “the best science comes from the best person” is certainly problematic and uncomfortably close to the discredited “great man” theory of scientific progress⁵. But COVID-19 has exposed the flaw in the associated idea that the best policy waits on the best science.  The pursuit of “certainty, predictability and linear causality” as the evidential gold standard has turned out to ignore the pressing need to answer questions about the relative merits of a variety of possible policy interventions, e.g mandatory masks on public transit or in school classrooms, for which there is no time, and possibly no adequate research design, to answer decisively⁶. We have had to fall back on practice-based solutions, such as improved personal hygiene and testing and tracing, solutions that have been the cornerstone of the control of contagious diseases in the developing world for many decades. In these circumstances, trust in the messenger providing this advice has proved absolutely critical to the effectiveness of the message.

This is not, of course, the end of the story.  As Sarewitz reminds us, where value conflict is intense, the effect will simply be a switch from evidence shopping to character assassination, as the long-suffering federal public health and disease control appointments in both Canada and the US can sadly testify.  But where effective political leadership has been able to mitigate or marginalize value conflict, these appointments have proved indispensable to getting the message across. In the western provinces, in particular (no strangers to the polarization of competing values), the Chief Medical Officers have become public personalities, their trustworthiness supported by media attention to the person behind the role. As academics and professionals, we often dismiss this sort of thing too easily.  The focus on taste in shoes (British Columbia), dresses (Alberta) and knitwear (Saskatchewan, inevitably) seems slightly risible if not actually offensive. In fact, helping the messenger display evidence of character as an essential aid to connect with and persuade their audience, is an essential component of classical theories of rhetoric and clearly on display here. To object that this effect has nothing to do with evidence-based policy making is both to miss the essential point about practical necessity and to demand an inappropriately narrow standard of “evidence”.  And, writing as someone with a closet full of them, if one consequence is the restoration of the cardigan – an eminently practical garment – to some semblance of respectability in the world of fashion, I, for one, will be duly grateful.

Download the Making Wave blogpost.

¹ Sarewitz, D., 2004. How science makes environmental controversies worse. Environ. Sci. Policy 7, 385–403

² https://retractionwatch.com/retracted-coronavirus-covid-19-papers/

³ https://www.the-scientist.com/news-opinion/scientists-urge-consideration-of-airborne-sars-cov-2-transmission-67702 

⁴ https://slate.com/technology/2020/03/coronavirus-pandemic-science-politics-values-convergence.html 

⁵ McPherson https://www.cost-ofliving.net/mistakes-of-the-masters-the-problem-with-chief-scientific-officers/ 

⁶ https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1003266