Part 1: Lessons from the COVID-19 Pandemic: Where We’ve Succeeded and Where We’ve Failed

The COVID-19 pandemic has raised several interesting questions about the management of risks and the role of science in decision making. While risk management is a part of life, some risks are well articulated and effectively managed, while others are not. In the context of the pandemic, risk management strategies are meant to identify and rectify systematic and procedural failures.

The part of the risk response that was well articulated and worked the way it was designed was the drug discovery system. Some people may have been surprised with the quick turnaround in the production of a vaccine when the normal gestational period for vaccines is years, if not decades. Prior investments in new technologies and a large amount of money from governments around the world motivated companies to quickly and aggressively produce vaccines that have materially changed the trajectory of the disease. The response in terms of identifying risks and adjusting use-protocols for the AstraZeneca vaccine due to vaccine-induced immune thrombotic thrombocytopenia (VITT), showed the efficacy of the system in detecting relatively rare side effects of vaccines. Similarly, there was the adaptation of the delivery of the vaccine, where we went from the approved protocols for two doses of the same vaccine with a set delivery schedule to a more flexible protocol, allowing for mixing vaccines and longer lags between doses. The responsiveness of our regulatory system allowed for an efficient and effective evaluation of the efficacy and safety of the vaccines in real time.

In this entire process, we have learned by doing. Our system allowed us to identify, trace and redirect the use of vaccines to areas of lower risks and higher impact. What we learned is that extending the period between vaccines and even mixing and matching vaccines has low or no incremental risks. In some cases, our new approach may produce a better immune response than the original protocols. This part of the system response worked very well.

However, the system showed some weakness in the early detection of disease, with many issues still to be addressed. For instance, there are still debates about the role of the Chinese government and how others may have contributed to the spread of COVID-19. With all the politics embedded in global public health, science loses its ability to be as responsive as it should be. These are issues we need to work on.

Similarly, science has not been as influential as it could have been in that it has not driven most of our social response to the disease. This is partly the fault of scientists and governments. Although science has driven our vaccine program, the various lock downs, shutdowns, masking and quarantine rules are disturbingly not driven by science. One can understand that we did not have the scientific evidence to drive those choices, but as a scholar of risk management, it is particularly concerning that we have had a natural experiment that we have failed to document or learn from. Each of the more than 190 countries in the world adopted a different strategy to address COVID-19. Moreover, all the provinces and territories in Canada adopted different strategies and plans of action which were then refined further at the municipal and regional level. The frustration is that we do not know which ones worked or why. We do not really know whether 14 days quarantine coming into the country, masking in outdoor settings, or social distancing, for example, were efficient and effective in preventing people from being infected. This is troubling as it means we have not learned anything we can use the next time a pandemic erupts and we did capture much in the way of real-time data to help us do ex poste evaluations. Moreover, nothing was held constant in any particular jurisdiction to try a specific measure and see what happens. While scientists may be able to recreate some of the evidence to undertake ex poste studies, we may have lost the potential to teasing out the relative effects of each of these strategies. That is where the science failed us. The scientists either lacked the imagination and the intention to construct experimental spaces to capture the evidence in real time or, they might have had all that, but lacked the capacity to convince the regulators and the decision-makers that this is a critical part of public policy. This would not have been such a big issue if this was a small, one-time event, but the opportunity cost of COVID-19 (e.g., the shutdowns, disruptions, deaths and other public health investments) is arguably the largest public investment in public health in history. The fact that we have invested so much with so little return beyond the immediate response to the disease suggests that there is a lot more we need to do in terms of the risk management.

We have also learned that scientific communications can be confusing and create unintended problems. The Canadian National Advisory Committee on Immunization (NACI), a requirement of the World Health Organization, has in many ways confused rather than clarified the risks and opportunities of our vaccine program. NACI is tasked with advising the government on vaccines. It has variously told us vaccines are risky, then that they are safe if used exactly as evaluated, then that it is ok to mix and match and change the timing, that unused vaccines have critical best-before dates, which were then extended with limited explanation, and finally in a nod to Orwell that vaccines are all good, but that some are better than others. All of the deliberations have been behind closed doors, with the chair or another appointed spokesperson reporting on their summative findings. Many of those who are hesitant to take the vaccine point to the often confused and changing messaging.  There are a number of ways we could better position this kind of scientific advice in decision-making. One illustrative example is the US Advisory Committee on Immunization Practices (ACIP). The US committee does not come up with definitive statements but rather has a wide-ranging conversation that does not produce a definitive conclusion about the context for vaccines. These discussions are recorded and uploaded to the internet for anyone to watch, which allows decision makers and viewers to hear the full breadth of opinions about a topic.^[1] In times of uncertainty and rapidly evolving understanding, the more open and transparent ACIP process seems more suited to informing us of key things we need to know.  A lot of the communications about the vaccines were objectively correct, but subjectively problematic.

Science is vital to decision making in many policy areas, but we have a long way to go to properly situate it in our system to ensure effective policy decisions and outcomes.  Science unambiguously brought us vaccines that will go a long way to managing the disease, but it has failed to effectively drive many of the socioeconomic choices we have made or to effectively inform us about the risks and opportunities of different management tools.  In spite of these failings, we have muddled through, but I fear that our lack of design and foresight has undercut our ability to learn from this once in a generation event.

[1] https://www.cdc.gov/vaccines/acip/meetings/index.html

Dr. Peter W.B. Phillips is the director of the Centre for the Study of Science and Innovation, and a distinguished professor in the Johnson Shoyama Graduate School of Public Policy's University of Saskatchewan campus.

He earned his Ph.D. at the LSE and practiced for 13 years as a professional economist in industry and government. At the University of Saskatchewan, he was the Van Vliet Research Professor, created and held an NSERC SSHRC Chair in Managing Technological Change in Agriculture, and was director of the virtual College of Biotechnology.

He has had appointments at the LSE, OECD, European University Institute in Florence, University of Edinburgh and University of Western Australia. He was a founding member of the Canadian Biotechnology Advisory Committee and was on the boards of Canadian Agri-food Policy Institute, Pharmalytics and Ag-West Bio Inc. He has also held over 15 peer-reviewed grants worth more then $250 million and is author/editor of 15 books, and over 60 journal articles and 55 book chapters.