Why is “Evidence-Based” Such a Controversial Term?
According to the Washington Post, the Trump Administration is prohibiting the CDC from using several words or phrases in future budget documents. Banning the words “fetus,” “transgender,” “diversity,” and “vulnerable.” is being met with widespread outrage (as it, of course, should be). Likewise, public health researchers, like their associates at the EPA are enraged that “science-based,” a term they have based entire careers around, is forbidden.
But the most crucial term the Trump administration is telling the CDC not to use is “evidence-based.”
As a professional who works to create that evidence through ProofPilot, a tech company I founded, my initial reaction was an immediate angry surprise. But, the reality is I’ve seen the writing on the wall for a long time. If liberals and the scientific community explain this away as “bowing to the religious right,” we risk repeating the same mistakes we have made over the past two years. If fiscal conservatives let this slide, we have lost one of our most crucial budget controls.
At the CDC, “evidence-based” means there is some verifiable proof that a health program, treatment or campaign has the desired effect and works. Evidence-based goes well beyond the CDC. There is an evidence base that certain pharmaceutical drugs work to reduce cholesterol; that some math textbooks successfully teach 9th graders basic algebra; and that the food stamp program provides adequate nutrition for low-income children.
Governments make huge investments in public health, education and social service programming. They want to know this money is being spent on things that work. In fact, Between 2004 and 2014, one hundred laws were passed in 42 states to encourage public funding be used to prove “evidence-based practices and treatments.”
Think of it a little like running a business. When a corporation makes a big investment, executives want to ensure investments pay off. Because of differing markets, cultures, and operations, what works for one business may not work for another. So, no business takes another’s word for it. Each business has their own accounting system measuring costs and profit to determine a return on investment. They continue things with a positive ROI. They modify or end things that don’t.
But unlike businesses, the process of determining what works and what doesn’t in health and social service programming goes beyond simple financial considerations. This is where the trouble begins.
The tools to measure what works and what doesn’t to create this “evidence-base,” in public services are similar to those used in the pharmaceutical industry. They are made far more complicated by the realities of health and wellness outside a clinical biomedical setting. The randomized controlled trial, programmatic evaluation, and longitudinal outcome studies are so difficult, expensive and wonky few organizations can run them. So most public health, social service and educational organizations servicing our communities are left without that accounting system. They take the word of a small number of well funded, well-resourced organizations in very different environments and situations than their own.
The result is a relatively limited set of evidence-based initiatives that can feel prescriptive and irrelevant. In our knee jerk political environment, it’s no wonder we see a backlash to the term.
At ProofPilot, as we talk with small healthcare and service organizations across the country, we continually hear a version of the same complaint: “I’m required to implement this evidence-based treatment/program/campaign. My environment, situation, and population is entirely different from where this was tested. Just because it’ll work there doesn’t mean it’ll work here.”
Those complaining continue to tell us that there are solutions right there in front of them. Those local innovations are relevant, appropriate and cost effective. They just need a way to prove their solution works.
And that’s where ProofPilot sees an opportunity. We encourage other scientists and policy-makers to go beyond the surface bias on terminology. No one wants to spend valuable resources on programs that don’t work. It’s just we don’t agree on what works and what doesn’t, because we don’t actually know. The reality is we know so little about what works and what doesn’t to improve health and society. Social and behavioral science is just beginning to scratch the surface. We need to focus on improving on the ground capacity of local organizations to measure the impacts of healthcare, social service, and educational programs. We need to give them that accounting system.
George Washington famously called our democracy “the Great Experiment.” Right now, across our country, we are not running enough experiments to determine what works and what doesn’t. Instead of requiring implementation of a limited set of prescriptive evidence-based initiatives, we should be reducing the barriers to innovation and continuously monitoring for a positive return on investment and impact on our society. Only then will we find unique and worthwhile solutions to our country’s pressing and varied challenges.
You'll Also Be Interested In
Other Stories in the ProofPilot Blog