Risk Adjustment Big Data and Machine Learning Challenge
Risk Adjustment, Big Data and Machine Learning; Challenge and Opportunity Dov Chernichovsky, Ben Gurion University of the Negev, Israel Alvaro Riascos, Los Andes University, Colombia Ran Bergman, Deloitte, Israel Risk Adjustment Network Meeting. The Hague. October 11 -14, 2017
Goal of presentation Prompt further discussion about the role of big data and machine learning in RA
Rationale • The technology is there, and evolving fast • Insurers and plans, at least in Israel, are using it for assessing risk and potentially for risk selection • Can induce government (e. g. , Israel, Colombia) to use more rigorous risk adjustment mechanism than those used today
Presentation I. III. IV. Introduction to Big Data and Machine Learning The case of Israel The case of Colombia Conclusion
Big Data – Its 4 V’s
Major Types of Data • Traditional: • Electronic medical records (e. g. , billing, physician visits, measurements, lab tests, prescriptions and purchases) • Claims data • Patient and MDs’ surveys • Registry data • “Omics” – data at the cellular level • Genomic and genetic data • Patient generated data • Social media • Sensors
Machine Learning and Conventional Methods Conventional Machine • Handle different types of data, including numbers, text, and/or visual images, that are of different dimensions and structures • Detect and learn fast, through computer algorithms, highly complex and intricate relationships in high-dimensional data • Do not pre-impose constraints on the relationship between inputs and outcomes • Fewer assumptions in a non-parametric statistical model • • Need Structured data Slow Resource consuming Require pre-specified modeling and parameters
Goals of Machine Learning • Predict an outcome • Measure success by ‘out of sample performance’ • Take advantage of rich and complex data that entail outcomes that are determined by many potential predictors with complex interrelationships
Issues • (Still) imprecision and ambiguity of medical data • Validation of model, for policy making • predicts well an outcome (e. g. , 90 percent of the time) but has a high false positive rate • The incentives that the calculations produce
Some Evidence • Ross S. (2016) argues that a simplified risk adjustment formula selected via this nonparametric framework maintains much of the efficiency of a traditional larger formula. The ensemble approach also outperformed classical regression and all other algorithms studied. • Buchner, F. , Wasem, J. & Schillo, S. (2015) show that including interactions from a machine learning algorithm improves the adjusted R 2 from 25. 43% to 25. 81% on the evaluation data set. Predictive ratios are calculated for subgroups affected by the interactions. The R 2 improvement detected is only marginal. • Li et al. (2013) argue that the non linear relations among risk factors is usually very difficult to capture with linear models. The random forest model reaches a R 2 of 38% with an standard deviation on 0. 008 while the linear regression model reaches a R 2 of 31% with a standard deviation on 0. 01
Israel Based on confidential estimates of a sickness fund
Colombia (as in Israel) • Adds variables to current formula and explores potential interactions • Estimates with conventional and machine learning methods
Colombia – Step I: New Specification • Add to current state formula, based on linear regression of gender, age groups, location, and their two-way interactions (UPC): • enrollees' morbidity characterized by 29 long-term disease groups (Dx) • the severity of health condition using indicators of hospitalizations (H) and consultation with specialists (E), and admission to an intensive care unit (U) UPC + Dx + H + E + U UPC * H * E * U + Dx
Data Panel of claims -- 2010 and 2011
Estimation Models • The linear model estimated through weighted least squares • Three machine learning models: • Artificial neural networks (ANN) • Random forests (RF) • Boosted trees (GBM) To control for a selection bias, some specification of the machine learning models include an additional regressor: the probability of claiming a service since 20% of enrollees do not claim any service during the year Negative predictions of machine learning models were truncated at zero.
Criteria for Evaluating Models
Data
Entire Distribution Estimate
Lowest Quintile Estimate
Upper Quintile Esimate
Conclusions (for Colombia and Israel) • Risk adjustment policy can redistribute resources more efficiently by adjusting for the enrollees' health conditions and by using non parametric specifications that capture better than the linear models the non linear relation between risk factors • The non-parametric machine learning approach appears (in Colombia) to perform well for the entire cost distribution, poorly at its lower end, and well at the its higher end • The Israeli political economy associated with Big Data and Machine Learning suggests that sickness funds “are tempted” to use the new technological options for risk selection
References • Buchner, F. , Wasem, J. , & Schillo, S. 2015. “Regression Trees Identify Relevant Interactions: Can This Improve the Predictive Performance of Risk Adjustment? ” Health economics 26 (1): 74 -85 • Li, L. , Bagheri, S. , Goote, H. , Hasan, A. , & Hazard, G. 2013. Risk adjustment of patient expenditures: A Big Data Analytics Approach. Ieee international conference on Big data. 2013 (pp. 12{14) • Rose S. 2016. “A Machine Learning Framework for Plan Payment Risk Adjustment. ” Health Services Research 51: 6, Part I
Some more food for thought……. . Thanks !
- Slides: 27
