Assumptions: one of the challenges of a SIB feasibility study
“(…) Whether the assumptions are about the cohort size or the daily cost per offender, they have to be set carefully in order to represent the most probable case. As said before, this process is just as challenging as setting the right outcome metrics – and this is what made the research and modelling such an interesting project.”
Assumptions are part of every research to model the complexity of reality in the most accurate way possible. This article analyses the work process for building a strong Social Impact Bond (SIB) feasibility study on reducing recidivism (see article “How much time is enough, but not too much?” for more details).
In the previous article on this subject, the challenge of setting the right outcome metrics was described, which represents a generic challenge on the essence of a SIB: how to measure the true impact of the implementation project on the social outcome? However, a feasibility study is, at first, a work of research: data has to be aggregated and questions have to be asked to relevantly define the common theme. In fact, this groundwork on building the right scheme for the research was as much of a challenge as arranging the right outcome metrics. The present article describes what were the main issues involved and how the analysis overcame them.
The SIB under discussion has as purpose the reduction of recidivism, while helping offenders reinsert in society after release. In order to offer credibility to the importance of tackling this social issue, understanding the problem of recidivism is key. However, no real data was available to do so efficiently, as the recidivism rate in Portugal being unknown. Of course, there are no reasons as to why the rate should be extensively lower than in other comparable countries where it is on average 70% (UNODC Handbook, 2012). However, having an exact figure helps to convince stakeholders of the seriousness of this problem. The recidivism rate benchmark was therefore to be set within each cohort of the underlined project, assuming that the recidivism rate in Portugal was surely more than 50%. Therefore, a target rate of 80% non-recidivism was set within each cohort.
Furthermore, the eligible population for the project could not be estimated. The project’s target is within the male prison population, aged 25 to 35 years old, has approximately one year of sentence left and does not have any drug or mental health issues. PORDATA made it possible to narrow down the data slightly, however it was not possible to find an exact number on which to base pertinent calculations. This figure could influence the number of participants per cohort, which again would influence the project’s profitability. Finally, it was decided to expect 25 participants per cohort in the base case, similar to the 22 participants in the Brazilian APAC project. If the number of participants is lower, the cost-effectiveness will be affected, a factor taken into account in the sensitivity analysis of the project.
The payment driver for the recidivism outcome metric (80% of each cohort does not reoffend in the year upon release) also was part of the many uncertainties that had to be defined. Indeed, in case the outcome metric is achieved, the commissioner pays for the success. However, what is the fair amount to pay for 80% of a cohort not returning to prison? The commissioner could pay what was spent to achieve the outcome. On the other hand, the payment could also amount what the state saves through the project’s implementation. Both of these options were considered fair and negotiations following the feasibility study will show which one will be used.
If cost-efficiency drives the payment and the outcome metric is met, the commissioner pays what was spent to achieve it. If the outcome is not met, the payment, depending on the initial negotiations, may be adjusted to the outcome achieved. As said before, 80% of the participants within each cohort are expected not to return to prison during the first year after release, which is seen as the efficiency of the project – 20% of the means are spent in order to achieve the outcome. The payment takes this into account, which means that, in the base case, 100% of the costs are covered. However, if the achieved outcome is lower than expected, the payment will not cover more than 20% inefficiency. The service provider is therefore punished for not achieving the outcome but is still compensated for the success achieved.
In the case of a savings-driven payment, the calculations are based on more ambiguous values. A successful programme ensures that 80% of each cohort will not go back to prison for one year. The annual cost per offender times these 80% gives a first idea of the direct state’s savings. However, only part of them returns to prison under the traditional system. For this reason, only 50% of this amount was considered to be effective savings under the implementation project. By taking this percentage of the savings, the improvement of the social outcome achieved through the implementation of the programme represents an increased efficiency of 40%.
This leads to an important uncertainty: the daily cost per offender in Portugal is, depending on the sources, between 40€ to 53€ per day. When calculating the savings-driven payment, the difference in taking the lowest or the highest daily cost is about 50,000€. This variance can be the difference between a barely and a highly profitable project. The more conservative daily cost of 40€ per offender represents the base case scenario, however the matter is explained in the sensitivity analysis. With this brought to light in the feasibility study, its position will be important in the negotiations between the parties. Indeed, the first decision will be whether to make the payment efficiency or savings-driven and if so, which daily cost to take when calculating the savings-driven payment.
These are all details that will be decided by either time and implementation or negotiations. Without assumptions, modelling would not possible. However the sensitivity analysis shows their influence and reach over the project’s profitability. Whether the assumptions are about the cohort size or the daily cost per offender, they have to be set carefully in order to represent the most probable case. As said before, this process is just as challenging as setting the right outcome metrics – and this is what made the research and modelling such an interesting project.
Milena integrated the second edition of the SIB Research Programme and developed a feasibility study for a Social Impact Bond for the methodology of APAC.
 In the case of the daily costs being 40€, the payment amounts 146’000€ versus 193’450€ if the costs are assumed to be 53€. The IRR also changes substantially: it is 1.94% in the first case and 9.56% otherwise.