Evidence-Based Health Policy: Beyond Magical Thinking

Writing in the New England Journal of Medicine, a pair of Harvard health economists emphasize the need for healthcare policy based upon evidence, rather than slogans.

“Voters, physicians, and policymakers are left to wade through a jumble of anecdotes, aspirations, associations, and well-designed studies as they try to evaluate policy alternatives,” Katherine Baicker, PhD, and Amitabh Chandra, PhD, write. There are 3 essential characteristics of evidence-based health policy (EBHP) that must be in place for the conversation to evolve.

Specificity is the first. “Expand Medicaid,” isn’t a policy, and adding “to cover all adults below the poverty line,” still isn’t detailed enough to be implementable. Without specificity, health policy decisions don’t have a clear framework, and political posturing “sidestep[s] the hard work” of healthcare reform.

The second key is to distinguish between policies and goals. One policy may be made with multiple goals that are prioritized differently by various stakeholders. Baicker and Chandra use the example of financially incentivizing physicians to provide coordinated care. The evidence that such policy reduces healthcare costs is weak, they write, but the evidence that it improves outcomes is strong. Politicizing the idea that policy “doesn’t” work because it failed to achieve 1 of many goals is detrimental to EBHP.

“Being clear about goals is the only way to evaluate a policy’s effectiveness and the implied trade-offs between competing goals,” the commentary says. A policy will be judged on whether it achieves its stated goals: Positive outcomes that were not stated from the outset will be undermined by failure to meet others that were.

The third crucial element of EBHP is the evidence itself. “Introspection and theory are terrible ways to evaluate policy,” the economists argue, although finding evidence strong enough to correctly inform policy can be difficult. All medical studies are imperfect and very rarely are they able to definitively answer whether a policy will work.

The quality of the data being discussed is as important as the quality of the debate surrounding it. “Often the ‘policies’ being discussed are so ill specified that it’s impossible to bring any evidence to bear,” they write. Independent assessments like those made by the Congressional Budget Office can help inform one aspect of a debate, but that only takes into account federal budget with no consideration for public health.

The pair call the process of making health policy “fraught and uncertain” by nature, but they lay out a chart of examples on how their 3 elements can be used. Moving from a policy slogan of “coordinate care,” a more specific policy would be to pay primary care physicians and nurse managers to coordinate care. The 2 goals would be to improve outcomes and reduce spending.

A stylized assessment of evidence for outcome improvement would look at both theory and evidence: theory suggests a positive impact on outcomes, and evidence suggest a small improvement. For reducing spending, theory would suggest ambiguity (savings in some places, inefficiency in others). There is little empirical evidence, they note, of savings. But by establishing defined goals and identifying the evidence that exists or is lacking, a better debate can emerge.

Evidence is almost always lacking somewhere, they write, so it is often necessary to make policy based on the best available. It will fall on policymakers to analyze priority: Does the known outweigh the unknown? Waiting for perfect empirical evidence would paralyze the debate, Baicker and Chambra write, because it is unlikely to ever come. But that doesn’t mean that slogans and aspirations are the only basis for policy.

“Magical thinking won’t improve our health care system,” they conclude. “EBHP helps separate facts from aspiration…Better policy requires being both clear about our goals and clear-eyed about the evidence.”