Impact Evaluation of the prisonbased core Sex Offender
Impact Evaluation of the prison-based core Sex Offender Treatment Programme (SOTP): Aidan Mews, Laura Di Bella, Mark Purver
Background The prison-based core Sex Offender Treatment Programme (SOTP): • SOTP was a cognitive-behavioural psychological intervention intended to reduce sexual re-offending amongst sex offenders • Delivered by NOMS (HMPPS) to imprisoned sex offenders • Accredited by CSAAP since 1992 • Available in approximately one-quarter of male prisons in England Wales • Available for those sentenced to over 12 months - with a current (index) sex offence or a history of sexual offending; min 18 years old The study aimed to evaluate the ‘core’ prison-based SOTP only, and not other course variants
Previous research Evaluation of the pre-2000 SOTP (Friendship et al. 2003): • No impact on sexual reoffending • Reduced reconviction rates for treated offenders for combined sexual & violent reoffending The international research on the effectiveness of treatment for sex offenders provides a mixed picture. • Some quantitative reviews of available research have found that treatment reduces reoffending in sex offenders (Hanson et al. , 2002; Hanson et al. , 2009; Lösel and Schmucker, 2005; Schmucker and Lösel, 2008; Schmucker and Lösel, 2015) • A review of randomised control trials only found no beneficial effect of psychological interventions (Dennis et al. , 2012).
Common challenges in the field Widespread agreement that there is a need for high quality evaluations of treatment programmes for sex offenders ‘Reviewers are forced to consider which of the less than ideal studies are “good enough. ” Often they disagree’ (Hanson et al. , 2009, p. 866). Main challenges in the field: • • Low reoffending rates for sex offenders Heterogeneity of sex offenders Small samples/short follow-up periods Poorly matched counterfactuals (only a handful of RCTs)
Study design: Selection of candidates 2000 2012 2002 2015 Treatment Comparison Treatment Period Release Period Re-offending Period Index offence (sex offence excluding breach/soliciting Index Sentence Release Sexual Offence (SO) • Index sex offence • Released during the release period • Over 18 yr old at release • Sentence 12 months + Prison Sentence Treatment (SOTP) Non-Sexual Offence (NSO) First release that fit these criteria Comparison Group: Didn’t received core SOTP or other sex offender treatment during index prison sentence Treatment Group: As above + treated on core SOTP within the prison spell 5
Index Sentence Release Date Non-Sexual Offence (NSO) Sexual Offence (SO) Illustrative example candidates Comparison 2000 2002 Treatment Period Release Period Re-offending Period >18 <18 Treatment incomplete treatment Excluded 6 Any treatment commencing before 2000 >18 2012 Prison Sentence Treatment (SOTP) 2015
Creating the database Reoffending Cohort Database Master Treatment Database 11, 582 identified prisoners receiving any treatment (1996 -2013) Derived from several NOMS datasets Police National Computer (PNC) Comprehensive records on individual’s criminal/court history Yearly prison release data (2002 -2012) ANALYSIS DATASET 2, 5621 Treatment 13, 219 Comparison Offence types Re-categorisation of 320 sex offences into 30 categories of victim characteristics & seriousness 1 OASys Offender Assessment System database (20042012) Criminal/ personal history, attitudes & motivation Treated people who met inclusion criteria
Propensity Score Matching • Established technique to evaluate the effect of treatments in the absence of a randomised controlled trial • Compare cohort of treated individuals with a counterfactual/comparison group – matched in terms of their propensity score (likelihood) to undergo the treatment • Stata 10, psmatch (Leuven & Sianesi, 2003) • Chosen PSM model: kernel matching on the logit of the propensity score with a normal distribution, a bandwidth of 0. 03 and a requirement of common support. 8
Kernel Matching Retains all in untreated pool within the bandwidth Weights scores closer to the ‘treated’ score more (so will have a bigger weight in effect size calculation) weighted 0. 07 weighted 0. 6 9 weighted 0. 2 weighted 0. 3
Distribution of the logit of the propensity scores across the treatment and comparison groups, before and after matching
Matching factors A wide range of matching factors were included: • Either deemed theoretically important, • And/or were empirically related to both selection onto SOTP and one of the main five outcome measures (overall, sexual, adult serious, child contact, and child image binary reoffending) at the 20% significance level. Excellent matching: No statistically significant differences between the matched groups across any of the 87 matching variables (all standardised mean differences were < 5%). 11 offenders in the treatment group could not be matched. • They had high propensity scores. • • They received index life sentences (69 matched treatment group offenders also received life sentences). Two of them reoffended.
Time from offence to reconviction Around 80% of all non-sex offences are convicted within 6 months of taking place Around 20% of sexual re-offences among the SOTP treatment and control groups are convicted within 6 months of taking place (excluding breach and soliciting) Main SOTP analysis does not limit time from offence to conviction
RESULTS: Reoffending rate over follow-up period (average 8. 2 years) Treated rate Compariso n rate Difference T-value Total 39. 4% 38. 9% 0. 5%pts 0. 35 Sexual 10. 0% 8. 0% 2. 0%pts 2. 45* Non-sexual 24. 0% 23. 7% 0. 3%pts 0. 27 Adult serious 2. 7% 2. 4% 0. 3%pts 0. 59 Adult other 2. 1% 1. 4% 0. 7%pts 1. 86 Child contact 2. 2% 2. 1% 0. 1%pts 0. 18 Child image 4. 4% 2. 9% 1. 6%pts 2. 96* Child other 0. 5% 1. 0% -0. 4%pts -1. 86 Breach 21. 9% 22. 4% -0. 5%pts -0. 41 Soliciting 0. 3% 0. 1% 0. 2%pts 1. 36
Sexual reoffending survival curves
Child image reoffending survival curves
Sensitivity analyses • A number of sensitivity analyses were performed in order to ascertain whether the results (sexual reoffending unless otherwise stated) were stable across groups of offenders with different characteristics. • In each case, the matching process was re-run with the specific offenders under analysis, so that the matches between individuals in the treatment and comparison groups were able to change.
RESULTS: Reoffending rates for sensitivity analyses Analysis Treated rate Comparison rate Difference T-value Overall 10. 0% 8. 0% 2. 0%pts 2. 45* With OASys (OASys variables included) 5. 1% 6. 2% -1. 1%pts -0. 93 With OASys (OASys variables excluded) 5. 3% 6. 2% -0. 9%pts -0. 71 Extended SOTP 17. 5% 10. 0% 7. 5%pts 2. 71*
Sexual reoffending survival curves, with comparison observations censored at the point of Core SOTP treatment in follow-up period
Quantifying bias: sensitivity of the results to unobserved factors • Sensitivity analysis to quantify the magnitude of bias from unmeasured factors that would need to be present to change the key results of the study. • This indicates the amount of bias required for the sexual reoffending impact to become not statistically significant (i. e. the p-value to rise above 0. 05) when applying a two-sided hypothesis test.
Quantifying bias: sensitivity of the results to unobserved factors The level of bias needed, after controlling for factors included in the model, for the sexual reoffending impact to become statistically insignificant is equivalent to: • 48% increase in the odds of treatment and a 48% rise in the odds of reoffending. It may be for example that we are missing one important binary factor from the PSM model, e. g. overall measure of sexual deviancy. If one third of the treatment group are highly deviant, then these odds would require their probability of commencing SOTP to be 20. 3% (compared to 14. 7% for those not highly deviant) and the sexual reoffending rate to be 9. 6% (compared to 6. 7% for those not highly deviant). • A one quarter increase in the odds of treatment and a doubling of the odds of reoffending or vice versa
Survival analysis – going one step further In addition to the survival curves we also did some survival analysis: • Provides a combined view of binary reoffending and time to reoffence. • Makes better use of the data than a binary reoffending measure. • Enables more powerful hypothesis testing. • Addition to sensitivity analysis Methodology • Kaplan-Meier (nonparametric) vrs Cox proportional hazards model (parametric but allows controlling for confounders) • Log-rank test (equally sensitive to events over time) vrs Peto test (more sensitive to earlier events). Overall the findings reaffirmed the conclusions of the binary reoffending outcomes, with a statistically significant increase in sexual reoffending of around 3%pts after 13 years and 11 months.
Limitations • PSM is not as robust as a prospectively-matched evaluation or a RCT; can only match on observed factors. • Core SOTP is intended for individuals who are not in denial of their offending and are willing to engage in treatment; comparison group included individuals regardless of whether they were willing to engage in treatment or in denial of their offending. • It is possible that a large number of comparison individuals did other cognitive behavioural programmes (e. g. Enhanced Thinking Skills). • Dataset was derived from administrative IT systems which, as with any large scale recording system, are subject to possible errors with data entry and processing. • Only proven reoffending was measured.
Conclusions • Results suggest that while Core SOTP is generally associated with little or no changes in reoffending, there were some statistically significant differences. • In particular, there were small increases in the sexual and child image reoffending rates. • The effect was stable across most of the sensitivity analyses performed. • The small changes in the sexual reoffending rate suggest that either Core SOTP does not reduce sexual reoffending as it intends to do, or that the true impact of the Programme was not detected. • This study does not reveal the extent to which Core SOTP reoffending outcomes are due to treatment design or poor implementation.
Media responses
Further research • Further research into the effectiveness of SOTP using matching techniques should focus on measuring potentially relevant confounding factors. These include exposure to other treatment delivered in prison and on release of offenders into the community, and the level of supervision received on release. • General data issues would also be addressed, including ensuring that sex offender treatment names and dates are accurately recorded in the HMPPS treatment database. More research could also be undertaken to explore in further detail the characteristics of certain subsamples of the treatment group, such as those offenders who are selected for Extended SOTP treatment, who were found to have the highest sexual reoffending rates. • In addition, a process study could be undertaken to understand any systematic bias between being selected onto SOTP and not being selected. • To undertake as robust as possible an evaluation of SOTP, a randomised control trial should be considered.
- Slides: 25