In the Big Data age, everyone wants to measure things — and see if they can be made to work better. It’s a good impulse in most cases, but is it being applied appropriately to government?
In a new NYT post, David Leonhardt examines trends in testing government programs for quantifiable effectiveness. He notes initially that despite a widespread public suspicion of central government (or any) in this country, the Federal government actually does have a pretty impressive track record in a lot of areas. But it also has come under increasing fire in the past twenty years for being slow to adopt popular private-sector tools for measuring effectiveness of dollars for outcomes.
Of the 11 large programs for low- and moderate-income people that have been subject to rigorous, randomized evaluation, only one or two show strong evidence of improving most beneficiaries’ lives. “Less than 1 percent of government spending is backed by even the most basic evidence of cost-effectiveness,” writes Peter Schuck, a Yale law professor, in his new book, “Why Government Fails So Often,” a sweeping history of policy disappointments.
As Mr. Schuck puts it, “the government has largely ignored the ‘moneyball’ revolution in which private-sector decisions are increasingly based on hard data.”
And yet there is some good news in this area, too. The explosion of available data has made evaluating success – in the government and the private sector – easier and less expensive than it used to be. At the same time, a generation of data-savvy policy makers and researchers has entered government and begun pushing it to do better. They have built on earlier efforts by the Bush and Clinton administrations.
The result is a flowering of experiments to figure out what works and what doesn’t.
Now, I support measuring government programs to try to make them better. But there are immediate red flags for me surrounding the “how” part of measuring and the “what happens next” after the measuring.
I have four major areas of concern about this trend:
1) Who gets to determine the definitions of “cost-effective” or efficient? Who sets the cutoff points for when a program is simply too ineffective or not getting enough bang-for-the-buck to continue? Do these people consider realities on the ground and the lives affected or just look at spreadsheets?
Are the people creating measurement systems representatives of the people at large and the communities being served by the programs? Are they comprehensively trained in the relevant area backgrounds? Are they just more Wall Street-turned-public-servant-turned-future-lobbyist folks? Are they trying to measure things just to prove government “doesn’t work”?
2) Are we currently under-funding many of these programs so severely and chronically that we can’t effectively demonstrate success they might otherwise have if consistently funded at appropriate levels? Are we going to cut off money to these “under-performing” programs that we’ve already starved of money?
In education, in particular, we’ve seen the paired trend of measuring performance standards (which I agree is very important) and then tying Federal funding to districts and local funding to teachers to these results without first making the changes (including funding increases!) necessary to improve the results. Are we also going to start taking away money from programs that aren’t “improving” enough each year because they’re already doing well? (This was the famous backfiring of No Child Left Behind in high-performing education states like Massachusetts and New Jersey.)
To return to the Leonhardt article for a moment (my bolding added):
New York City, Salt Lake City, New York State and Massachusetts have all begun programs to link funding for programs to their success: The more effective they are, the more money they and their backers receive. The programs span child care, job training and juvenile recidivism.
The approach is known as “pay for success,” and it’s likely to spread to Cleveland, Denver and California soon. David Cameron’s conservative government in Britain is also using it. The Obama administration likes the idea, and two House members – Todd Young, an Indiana Republican, and John Delaney, a Maryland Democrat – have introduced a modest bill to pay for a version known as “social impact bonds.”
Republicans have moved the goalposts so far since the start of the Reagan Administration with their view that “government is the problem, not the solution” that everything seems to be catered toward “proving” this claim by decades of intentionally “starving the beast” — under-funding/de-funding programs across the board by slashing revenues to pay for them — and then measuring outcomes afterward.
Back to my areas of concern…
3) While it’s important to get as much out of each dollar invested as possible (so you can use as much of the money as possible for as many people as possible), many public functions are public because they are not effective money-makers and need to be funded regardless of balance sheet results. Sometimes things just aren’t all that “cost-effective,” yet are necessary for the promotion or execution of certain social and economic goals.
In fact, the push for “cost-effectiveness” as a measurement skips over the fact that the goal of some programs is to provide an emergency economic floor, below which citizens should not be able to fall, rather than being designed to lift them up. A floor is not an elevator, and you wouldn’t measure a floor’s elevation over time to find out if it’s getting you closer to the top of the building. Many of the War on Poverty programs, in particular, don’t get nearly enough credit for being a major force in keeping total destitution in check even during long recessions and stagnant recoveries, because critics are too busy asking why they haven’t outright ended poverty.
4) Is this just another way to insert private sector profiteering in the middle of public functions that don’t need them? For example, we’ve already seen Goldman Sachs forcing its way in on the revenue stream of the Massachusetts prison system to do something the state could do and, in doing so, taking away money the state could be re-investing to help more ex-convicts stay out of prison.
Again, these aren’t arguments against measuring. They’re more arguments for making sure we think carefully and deliberately about how to measure public programs (because the private sector practices can’t really translate directly) and about what we do with the information once it’s collected. (Not to mention my usual goal of reminding people to think about why government programs exist in the first place.)
Corporations — or their private equity hostile owners in some cases — have generally been using the data-driven approach not to build up under-performing assets (the private sector equivalent of different public programs) but rather to dump them for cash or drop them altogether. That has had real social harms all over the country. And maybe it’s not their job, as private companies, to worry so much about the social harms of their balance sheet decisions (though the Europeans have made a compelling case for a more integrated consideration), but in a country that doesn’t have a strong social safety net or a strong tradition of government stepping in to catch those left behind by corporate downsizing, outsourcing, and fire sales of assets, that’s a huge problem. Let’s not then replicate the problem in the already restricted government sphere and prevent effective balancing action.
