Would you know sound policy if you saw it?

Someone asserted during a teleconference with climate experts that we were supposed to generate ‘sound policies’ to advance the climate agenda. We read a brief description of soundness that stated such policies would be effective, efficient, equitable, focused and durable. We were directed to look for policies that would actually work at a minimal expense, distribute costs and benefits in some balanced way, avoid unintended impacts and last as long as needed to achieve the goals.

We quickly moved on, assuming that we all would know a sound policy if we saw one. As policy ‘wonks’ (know spelled backwards), we exuded confidence and conviction. That got me thinking—would our group (or any group) really be able to converge on a set of policies exhibiting those factors as ‘sound’? And if we did, would that list be legitimate?

This debate about sound policy is not new. Sometimes it is cast as a discussion about the role evidence can play in influencing or determining policy choices. Scholars, practitioners and citizens all have suggestions of policies they like—or more often policies they hate.

In many ways we are asking about validity. Technocratic validation of the cause-and-effect relationships of a policy is vitally important—it is often called internal validity. It should be necessary, but if progress on a whole host of policy files, especially the environment and climate change, is anything to go by, it is far from sufficient. In many ways most of us refuse to defer to experts. We no longer want simply to be told what is best—we want to be convinced of the merit of their advice. Therefore, external validity is also needed—we need to know that the advice underlying the candidate policies is both trustworthy and meaningful. That requires opening up the conversation about what is proposed, why and on what basis, disclosing all the assumptions, implicit and explicit value judgements and uncertainties.

To begin, we need to see the full cause-and-effect story and the evidence in support of any measure, especially the diversity and range of responses to each application of the policy measures. Human and natural systems have significant variabilities. We need to know the most likely outcome, the range of possibilities, and the relative likelihood of them emerging.

We also need to see the assumptions that drive the analysis. What do the experts assume about what the economy looks like and how it operates? What do they assume about how people will respond to different incentives? Each assumption can have a profound impact on whether a policy will actually work in the real-world context.

Then we need to see the efficiency analysis. What costs and benefits are included? Which are excluded? What explicit or implicit value is put on different impacts at different times? The devil is always in the details.

Next, we need to see the political economy analysis—we want to know the full array of expected winners and losers. Regulatory and policy systems are frequently captured and manipulated by those seeking to enrich themselves by writing the rules in their favour. As US Supreme Court Justice Louis Brandeis wrote, “Sunlight is said to be the best of disinfectants.”

Finally, we need some sense that those targeted by the policy will respond in the way intended. It is unreasonable to assume that people will do what policy makers want. Policies falter when they are so poorly designed that people can exploit their weaknesses or when the underlying goals and values are incommensurate or antagonistic to the targeted population.

Perhaps an example might put this into perspective. Right now, most governments that have grabbed hold of the climate agenda have adopted some form of pricing for carbon. While almost universally supported by experts as efficient, effective and equitable, the responses have ranged from nagging doubts to outright opposition in many places. The challenge is that while the case about carbon may have been made successfully, the pathway for impact between the taxes proposed and the emissions of carbon is vague—the average consumer cannot conclusively see the relationship between the relative price changes they face and their emission of carbon. In some instances, especially where there are limited alternatives to existing production and consumption choices, the rising prices do nothing more than raise costs and marginally lower activity, making this a less efficient strategy than one that offers ready alternatives. Nor is it necessarily equitable. Wealthier people may have the means to adapt to higher prices by retrofitting houses and buying electrical vehicles (often aided with further subsides); less well-off people simply become poorer.

We need to see information to validate the real-world applicability of any measure, especially those assumptions and value judgements about the trade-offs inherent in any policy. Only then can we advance towards sound policies that are enduring and able.

Download this issue of Making Waves.