Predictive Risk Modeling in Child Welfare Some Ethical

Predictive Risk Modeling in Child Welfare: Some Ethical Considerations Brett Drake, Ph. D. George Warren Brown School of Social Work and Public Health, Washington University in St. Louis. September 15, 2017

There an almost infinite number of ethical questions that can be raised about any human endeavor. In this presentation, we will cover some of the more commonly discussed ethical concerns around PRM. Among these are • Effectiveness (better tools are ethically preferable) • Data Access (what data are ethically permissible to use? ) • Data Use (what uses of data are ethically desirable? ) • Ethics of Referral-Level (hotline) vs. Universal-level (preventative) PRM • Specific ethical issues and suggested remedies.

CPS cases progress through several stages : A Hotline Call is made by professional (60%) or nonprofessional (40%) source Case is screened in (investigated/assessed: 60%) or out (dropped: 40%) Case receives initial investigation or assessment Case may be closed with no action taken (about 60%) Case may be served and then closed (about 35%) Placement may occur (about 5%)

Stage in CPS Process Ethical Cost of False Negative Ethical Cost of False Positive Hotline Screening High Low Initial Assessment High Med Court Involvement Decision High Termination of Parental Rights High

The better predictive tools you have, the more false negatives and false positives you can avoid.

Decades ago, we made predictions by asking the workers involved what they thought would happen. This is called “Clinical Judgment”. Turns out this is not very accurate. Then we started using “consensus-based instruments” to assess risk. They didn’t work well either. For the past decade or two, we’ve been using “actuarial instruments” which track known predictors of bad outcomes. These work better, and are generally used in conjunction with clinical judgment. Now we have the opportunity to use PRM.

Ethical Concerns About Data Access Internal CPS Data Prior reports, prior placements…. Non-CPS State-Held Data IM, Juvenile Delinquency, Medicaid, Criminal Justice, Employment, Vital Statistics Non-State Data Hospital, Private Service Agency, Education (state or local) Public Domain Neighborhood (Census), Arrest, Sex Offender Registries, other data

Different ways people think about the ethics of accessing data: Is this more like an academic research project, where informed consent is valued? Is this more like an internal agency evaluation, which does not require consent or internal or external review? Is this a typical state agency function, governed by preexisting state rules and guidelines about data sharing? Is this an “emergency” situation, such as a developing law enforcement situation, where the necessity of intervention can waive or reduce nonemergency concerns (e. g. a policeman responding to an active assault).

Questions about the ethics of data access ultimately depend on the situation. In the simplest case, when no new data are accessed, the question is a non-issue. In other cases, changes in agency policy or state law may be necessary

Ethics of Data Use Two potential uses of PRM stand out in child welfare. 1) Referral-Level Modeling Using PRM to assess risk after a hotline call is received. May or may not involve accessing new data. 2) Population-Level Modeling CPS uses universal data (e. g. birth records) or agency data (e. g. hospital data or other sources to predict high risk situations before a hotline report is made for the purposes of offering preventative services.

Referral-Level Modeling with Current Data This involves CPS using a better tool to process data it already has to make case decisions it would have to make anyway on people it is already seeing. Here, we find relatively few ethical concerns. Note that if new data are used, we need to assess the ethics of using those data.

Let’s flip this over a bit – is there an ethical cost of electing not to use PRM? This depends on an empirically answerable question – Does PRM work better than current practice?

So does PRM work? We are very fortunate here – PRM can be piloted in advance, and predictions from the PRM model can be compared with predictions actually made in the field using “service as usual”. Unlike virtually all other tools you can virtually “field test” PRM in advance, without service disruption or worker or client inconvenience. * In my mind, an empirical comparison of PRM to existing practice is an ethical imperative. Why guess what works the best when you can know? *although ideally, “on the ground” field testing will come next to see what implementation issues may arise, and if the virtual test results obtain.

Some work has already been done assessing the effectiveness and ethics of referral-level PRM Ethical Analysis: Predictive Risk Models at Call Screening for Allegheny County. Dare, T. & Gambrill E. (2017) http: //www. alleghenycou ntyanalytics. us/wpcontent/uploads/2017/04/ Developing-Predictive-Risk -Models-package-withcover-1 -to-post-1. pdf

Summary Conclusion, Allegheny County In our assessment, subject to the recommendations in this report, the implementation of the AFST is ethically appropriate. Indeed, we believe that there are significant ethical issues in not using the most accurate risk prediction measure. Instruments that are more accurate will result in fewer false positives and false negatives, thus reducing stigmatization (false positives) and more lost opportunities to protect children. It is hard to conceive of an ethical argument against use of the most accurate predictive instrument.

New PRM Application: Population-Level Modeling (more complicated ethically) There is a preexisting ethical review which has been done exploring the potential ethical concerns attendant to population-level modeling.

Predictive Risk Modeling and Child Maltreatment: An Ethical Review, Dare, T. (2013) https: //www. msd. govt. nz/ documents/about-msd-and -our-work/publicationsresources/research/predicti ve-modelling/00 -predicitverisk-modelling-and-childmaltreatment-an-ethicalreview. pdf

Predictive Risk Modeling and Child Maltreatment: An Ethical Review, Dare, T. (2013) https: //www. msd. govt. nz/ documents/about-msd-and -our-work/publicationsresources/research/predicti ve-modelling/00 -predicitverisk-modelling-and-childmaltreatment-an-ethicalreview. pdf

Summary Evaluation: The application of predictive risk modelling to child maltreatment does raise significant ethical concerns. Many of these concerns can be significantly mitigated or ameliorated. Remaining concerns may plausibly be regarded as outweighed by the very considerable potential benefits of the Vulnerable Children PRM. In sum, the application of predictive risk modelling to child maltreatment is ethically justified provided the recommendations below are addressed. (New Zealand case)

Or in simple language…. There are ethical concerns with Population-Level Modeling. These concerns can often be remedied in advance or mitigated through ongoing efforts (e. g. training) A final ethical judgment requires balancing the potential ethical costs of the new program against the potential ethical advantages (e. g. prevention of maltreatment) of the new program

Potential Ethical Problems in PRM (and what to do about them) Drawn largely from Dare 2013 and Dare & Gambrill 2017. Some of these issues apply only to population-level modeling.

Over and Under- Identification: When PRM is used to identify cases which would not otherwise be reported, it is possible that errors will be made. Recommendations: • Include as many databases as possible, including universal databases to expand coverage and avoid including people just because they accessed a given service • PRM should supplement, not replace existing early identification systems • Experienced professionals should exercise judgment regarding PRM scores • All users must understand that PRM can miscategorize families • Workers must be trained to avoid “confirmation bias” – reflexively agreeing with the PRM and seeking to find only information consistent with the PRM prediction.

Stigmatization: Individuals can suffer from the stigmatization of being identified by a PRM and involved with a service system. Recommendations: • Interventions are at the minimum level necessary • Information from the PRM should only be distributed to those needing access • Some detail which could lead to stigmatization could be omitted if not necessary to effective practice • Workers must be trained that risk models are not “guilty” verdicts and that they are doing preventative work, not serving “wrongdoers”. • Workers must be trained to avoid “confirmation bias” – reflexively agreeing with the PRM and seeking to find only information consistent with the PRM prediction. • High PRM scores should not cause workers to treat clients more punitively or in a more stigmatizing fashion

Resource Allocation PRM is a preventative intervention, and should not degrade or reduce currently available services, particularly at the expense of families with lower PRM scores. PRM-based efforts must be responsibly managed with regard to resources based efforts should be seen as an avenue of additional and workload.

Privacy Careful monitoring of potential invasions of priority must be ongoing, particularly with regard to potentially discriminatory effects. Staff must be thoroughly trained with regard to their duties of confidentiality.

Professionalism and PRM Perhaps most importantly, staff must be carefully trained to understand that PRM is a tool best used in conjunction with clinical judgement, and that PRM does not replace clinical judgment.

Other Issues Information about “others” – family members, paramours, etc… Should data about these people be treated differently than about children? Increased risk following “bogus” reports: How to deal with data elements which are known to represent errors – will people “just be screened in because they were screened in last time”? .

Bias It is possible that various forms of bias (over- or underidentification) exist in the CPS reporting, screening and response systems. Among these are: • Racial Bias • Class Bias • Visibility Bias due to service use These concerns are not in any way unique to PRM.

Among these, racial bias has received by far the most attention in recent years.

The possibility that any tool may over-identify or under-identify (e. g. screen-in or screen-out) due to the race* of the subject is a serious ethical concern. This is not a concern particular to PRM, but a general concern for any tool or procedure – including services as currently delivered. * or class, or service history or other factors

This problem, fortunately, can be addressed through advance testing of the PRM Any PRM model should be rigorously pretested using known case histories. The procedure is simple: 1) Take data from prior calls to the hotline 2) Use the PRM to predict chosen later outcome (e. g. re-referral, later placement…) 3) Compare accuracy of PRM to existing system, including determining if the PRM systematically over-identifies or under-identifies particular racial (or other) populations. Note – this example is referral-level, but the same approach can be taken with universal –level systems.

The propensity for a PRM to over- or underidentify particular populations can be known in advance. The degree of bias in a PRM can therefore be compared to the degree of bias in the system as it currently stands. PRM systems can then be adjusted to reduce any potential biases attendant to race or other factors.

This is a unique strength of PRM, the ability to “test drive” the models without burdening the system or causing potential harm. This advantage is not limited to determining bias, but applies to the utility of the PRM in general. It remains, of course, necessary to do good implementation science – the use of the PRM model “on the ground” will have to be monitored after deployment and checked against prior projections.

Conclusions:

Referral-level PRM using no new data poses few new ethical problems. Referral Level PRM using new data requires at least some review of the ethical appropriateness of accessing those data.

Population level PRM does include ethical challenges. A “balance” must be calculated, weighing ethical costs against ethical gains. Prior work (Dare) suggests that it may represent a more, not less, ethical form of practice on balance.

Safeguards, particularly worker training and pretesting of PRM models, particularly with regard to subpopulations must be employed.

Thank you