First, the control group fell apart. Wardens at Rikers Island, the New York City jail, could not separate teenagers who were to participate in a course of cognitive behavioral therapy from those who were not supposed to attend.
Then the city’s Education Department, which had offered to put teachers on Rikers to assist the intervention, pulled out. And the budget of the Osborne Association, which had been enlisted to carry out the therapeutic program, was cut when Rikers’s teenage population unexpectedly fell below the level written into its contract.
“We needed two facilitators in each class instead of one, and we were serving twice as many kids because we couldn’t separate the groups,” said Elizabeth Gaynes, chief executive of Osborne.
“We hoped to have people in the community to continue the M.R.T. outside after kids left, but we had to take outside staff and put it back in the jail,” she said, referring to the therapy by its formal name, Moral Reconation Therapy.
Can something that fails prove to be a success? That’s the question surrounding a bold experiment in putting Wall Street techniques to work in shaping government policy.
Aimed at reducing teenage recidivism by at least 10 percent, the Adolescent Behavioral Learning Experience was found, in a careful evaluation by the Vera Institute of Justice, not to keep teenagers from being sent back to Rikers at all.
This might all seem par for the course for delivering social services in the messy context of New York City’s famously troubled jail. But what happened next set this experiment apart from pretty much anything tried before in the nation’s correctional system: The city pulled the plug and walked away without losing a dime of taxpayer money.
The experiment, financed by the nation’s first social impact bond, offers a glimpse of a potential future for delivering government services. It is a future that promises more rigor in identifying failure and success in settings from prisons and homeless shelters to public hospitals and schools.
“So often in government you don’t have that level of clarity that comes from the rigorous evaluation that you had here,” said James Anderson, who runs government innovation programs at Bloomberg Philanthropies, which was closely involved in the experiment. “That leaves the government thinking a program was successful when young men’s lives were not actually improved.”
Governments around the world hope it may offer more money to address their policy priorities, too.
The beauty of the therapeutic experience, from New York City’s perspective, was that it was financed not by taxpayers but by a $7.2 million investment by Goldman Sachs — backed by a $6 million guarantee by Bloomberg Philanthropies. The city agreed to pay Goldman back only if the program could trim recidivism by a tenth, enough to save some money by closing a section of the jail. While Goldman’s profits would rise as recidivism fell below that threshold, the city would keep a share of the savings.
“These vehicles are structured in such a unique way that they carry very little risk for the government,” said Kristin Misner-Gutierrez, director of social services in the Office of the Deputy Mayor for Health and Human Services, who was closely involved in the program as it was being developed under the previous administration of Michael R. Bloomberg.
Despite the failure of the program, the experiment succeeded in blazing a new trail. “One of the things that was nice about this was that people were being brave, doing something bold,” said Susan Gottesfeld, who ran Osborne’s operation at Rikers. “They were doing an intervention that had never been done before in a place like Rikers on a scale never done before.”
For some young men, it was hardly a waste of time. “It is a steppingstone to realize what you did wrong, who you have hurt, what you can change,” says Louis Rivera, a 20-year-old who graduated from Rikers’s moral therapy program.
Mr. Rivera is now out on parole, holding a job at a paint and hardware store, regularly seeing his 3-year-old son and living with his brother in Queens. He acknowledges that sticking to the new persona can be tough, especially on the outside, “where you meet old friends; old enemies.” Still, he concludes “it worked for me.”
Yet for all the enthusiasm among governments eager to draft private capital into providing social services, the new instruments carry potential drawbacks, too.
“Nonprofits are starved for funding and governments have little money to finance innovation,” noted Gordon Berlin, head of the social policy research organization MDRC, which led the program at Rikers for the city, contracting with Osborne to execute the plan. “Maybe people are loading a little bit more on this than they should.”
For one, designing social policy around precisely measurable results and narrow incentive mechanisms to draw private sector money might circumscribe the types of services the government will provide, in effect orphaning the messier, hard-to-measure challenges.
“If the government objective is to save money, it limits the use of programs like these to things that are very expensive, like prison beds, hospital beds, foster care beds,” Mr. Berlin said. “The choice of projects might be affected.”
At the extreme, worries Ms. Gaynes, the Osborne chief executive, investors could take over policy design.
“Neither Goldman Sachs nor Bloomberg interfered at all with the model, but one fear is that investors will drive the decision of what the government is doing in the same way as with philanthropy,” she said. “Government shouldn’t say we don’t want to do this unless Goldman tests it first.”
The question is twofold. First, to what extent could, or should, private capital assist in achieving the government’s goals? Second, to what extent could the notion of paying for specific outcomes — like measurable declines in adolescent recidivism — be brought into the policy mix even without the contribution of private capital?
Social investment bonds are a hot topic in the investment and philanthropy communities. The British started the first one in 2010, also tied to reducing prison recidivism. Since then, 45 have been carried out around the world. In 2012, they made Harvard Business Review’s list of audacious ideas for solving the world’s problems.
The British government has posted menus of prices for particular objectives: £1,200 ($1,872) a child for programs that curb persistent truancy, meaning missing school more than 10 percent of the time, or £2,600 for moving somebody into a first job. Consortia of investors and social service providers then bid for the contracts
To date, seven have been set up in the United States. Starting at Rikers, Goldman has been involved in four, including a $17 million pre-Kprogram for disadvantaged children in Chicago, which will return $9,100 a child for each year he or she avoids special education later on, $2,900 for each child deemed prepared for kindergarten and $750 for each student who scores above the national average on the third-grade reading test.
“We continue to be enthusiastic about using capital this way,” said Andrea Phillips, vice president in the Goldman Sachs Urban Investment Group, the arm that handles its socially minded investments.
Ms. Phillips argues that institutional investors and wealthy individuals are increasingly trying to align their investments with social values, which could draw more capital into the model.
It’s too early, though, to say whether the idea will take off. Of the 38 that had been begun by March 31, 25 served 1,000 individuals or fewer, according to a study led by Emily Gustafsson-Wright at the BrookingsInstitution. Many investors — like Goldman at Rikers — have relied on hefty guarantees by philanthropic donors to cover potential losses.
There are other obstacles, including the classic “wrong pocket” problem: The government agency paying for the intervention is not the same one that might chalk up the savings. Then there is the more critical hurdle: finding a well-defined measurable outcome that is both meaningful in terms of policy and that investors could turn into cash, and agreeing on how to evaluate it.
Social investment bonds probably took off in the criminal justice area because, the Brookings report noted, “the criminal justice system has clearly defined and monetizable outcomes, and there is high political commitment due to the large number of negative outcomes and resulting community pressure.”
Not every desirable policy intervention will share these virtues.
With the experience of several deals under her belt, Ms. Phillips at Goldman detects the uses and limitations of the vehicle. “They work best when you are working with evidence-based programs that have a track record that you can scale,” she told me. “They work better for scaling than for experimenting with new programs.”
Ultimately, the biggest promise of these social impact bonds — also known as “pay for success” programs — might lie more in their ability to impose discipline on government programs than in their promise to draw private money.
We want to channel this into a broader conversation about paying for performance and focusing on outcomes instead of activities,” Mr. Anderson at Bloomberg said, arguing that it could help “bring evidence into policy deliberations.”
The Obama administration is exploring this path. Last October the Corporation for National and Community Service put up $12 million in grants to advance and evaluate emerging models that align payment for social services with verified outcomes.
This carries enormous promise. To the extent that governments could buy results, instead of covering the expense of services rendered, policy could be made more efficient regardless of where the money came from. If voters could be provided evidence of a government success, they might be more willing to support it.
And yet the push to evaluate all must be tempered as well. The Moral Reconation Therapy at Rikers may not have reduced recidivism as hoped. But the many services offered around the therapeutic interventions achieved other useful things, not least keeping potentially violent youths busy on productive activities for much of the day.
“All they were testing is whether M.R.T. by itself would make a difference, not whether something you could do in a jail would make a difference,” Ms. Gaynes said. “Even if we could have raised money to do other stuff, we were not allowed to because we were testing M.R.T. alone.”
To the hard-nosed evaluators of the new landscape of social services, this sort of argument may sound like a cop-out. But it carries a truth: Social policy probably cannot be limited exclusively to the things one can readily measure.