Clinical Informatics YearinReview 2017 Blackford Middleton MD MPH

Clinical Informatics: Year-in-Review 2017 Blackford Middleton, MD, MPH, MSc Apervita / Harvard TH Chan School of Public Health Twitter: @bfm, #AMIA 2017

Disclosures Blackford Middleton, MD, MPH, MSc - Chief Informatics & Innovation Officer at Apervita (employee, stockholder) - Co-Chair, Steering Committee Patient-centered Clinical Decision Support – Learning Network - Consultant / Grant Support: MITRE, RTI, Gordon & Betty Moore Foundation AMIA 2017 | amia. org 2

Agenda • Describe the Approach taken to arrive at this year’s papers and events for the Year-in-Review • Quick Review: Top 10 Downloaded articles (more or less) for each of: • Journal of the American Medical Informatics Association (JAMIA) • Journal of Biomedical Informatics (JBI) • J Applied Clinical Informatics (ACI) • International Journal of Medical Informatics (IJMI) • Methods of Information in Medicine (MIM) • Top papers discussion: • Title, Authors, Institution, • Highlights, Summary, Memorable take aways • Notable Events and Quotes from 2017 AMIA 2017 | amia. org 3

Approach • Timeframe: November 16, 2016 – October 15, 2017 • Sources • Google Form Input • Relevant Working Groups • Colleagues, students • Top Ten Downloaded or Tweeted Articles From Key Informatics Journals or Conferences • JAMIA, JBI, ACI, MIM, JMIR, CHI, CSCW • Curator Review & Initial Summarization • From Student Working Group • YIR Leader Selection and Final Summarization • - Reviews, - same institution with >1 entry, • Key Events and Quotes: Crowd sourced AMIA Power. Point Template 4

Top 10 Downloads: JAMIA Rank 1* Title Crossing the Health IT Chasm: Considerations and Policy Recommendations to Overcome Current Challenges and Enable Valuebased Care 2* Problems with health information technology and their effects on care delivery and patient outcomes: A systematic review 3 Safety Huddles to Proactively Identify and Address Electronic Health Record Safety 4 5 6 7 Key: * = also from Student WG review Authors Institution Julia Adler-Milstein, Peter J Embi, Blackford Middleton, Indra Neil Sarkar, and Jeff Smith U Michigan Mi Ok Kim Enrico Coiera Farah Magrabi Macquarie University Shailaja Menon Hardeep Singh Traber D Giardina William L Rayburn Brenda P Davis. Elise M Russo Dean F Sittig Houston VA / Baylor Yen S Low Aaron C Daugherty Elizabeth A Schroeder William Chen Tina Seto Susan Weber Michael Lim Trevor Hastie Maya Mathur Manisha Desai Carl Farrington Andrew A Radin Marina Sirota Pragati Kenkare Caroline A Thompson Synergistic Drug Combinations from Electronic Health Records & Gene Peter P Yu Scarlett L Gomez George W Sledge, Jr Allison W Expression Kurian Nigam H Shah Stanford Efficiency and safety of speech recognition for documentation in the electronic health record Implementation of a Scalable, Web-based, Automated Clinical Decision Support Risk Prediction Tool for Chronic Kidney Disease Using Consolidated Clinical Document Architecture and Application Programming Interfaces Tobias Hodgson Farah Magrabi Enrico Coiera Lipika Samal John D D’Amore David W Bates Adam Wright Jason Walonoski Mark Kramer Joseph Nichols Andre Synthea: An Approach, Method and Software Mechanism for Generating Quina Chris Moesel Dylan Hall. Carlton Duffett Kudakwashe Synthetic Patients and the Synthetic Electronic Healthcare Record Dube Thomas Gallagher Scott Mc. Lachlan Macquarie University Brigham & Women's / Harvard MITRE Using Health Information Technology for Clinical Decision Support and Predictive Analytics Lucila Ohno-Machado UCSD Suehyun Lee Jiyeob Choi Hun-Sung Kim Grace Juyun Kim Kye Hwa Lee Chan Hee Park. Jongsoo Han Dukyong Standard-based comprehensive detection of adverse drug reaction signals from nursing statements and laboratory results in electronic Yoon Man Young Park Rae Woong Park Hye-Ryun Kang Ju 9 health records Han Kim Seoul National Univ Brian J Zikmund-Fisher Aaron M Scherer Holly O Witteman Jacob B Solomon Nicole L Exe. Beth A Tarini Angela Graphics Help Patients Distinguish Between Urgent and Non-Urgent AMIA 2017 | amia. org 10 *CHI Deviations in Laboratory Test Results Fagerlin U Michigan 8 5

Top 10 Downloads: JBI Rank 1* 2* 3* Key: * = also from Student WG review Title Authors Institution (lead author) Brett K. Beaulieu-Jones, Casey S. Greene, the Pooled Resource Open-Access ALS Clinical Trials Semi-supervised learning of the electronic health record for Consortium phenotype stratification U Penn Jing Zhao, Panagiotis Papapetrou, Lars Asker, Henrik Learning from heterogeneous temporal data in electronic health Boström records Stockholm U Vasa Curcin, Elliot Fairweather, Roxana Danger, Templates as a method for implementing data provenance in Derek Corrigan decision support systems King's College, London Lamyae Sardi, Ali Idri, José Luis Fernández-Alemán 4* A systematic review of gamification in e-Health Longitudinal analysis of discussion topics in an online breast 5*CHI cancer community using convolutional neural networks Development and validation of a continuously age-adjusted measure of patient condition for hospitalized children using the electronic medical record 6* Clinical code set engineering for reusing EHR data for research: A 7* review 8* Predicting healthcare trajectories from medical records: A deep learning approach 9* Navigation in the electronic health record: A review of the safety and usability literature AMIA 2017 | amia. org An unsupervised machine learning model for discovering latent 10*PHI infectious diseases using social media data Shaodian Zhang, Edouard Grave, Elizabeth Sklar, Noémie Elhadad Michael J. Rothman, Joseph J. Tepas III, Andrew J. Nowalk, James E. Levin, Joan M. Rimar, Albert Marchetti, Allen L. Hsiao Richard Williams, Evangelos Kontopantelis, Iain Buchan, Niels Peek Trang Pham, Truyen Tran, Dinh Phung, Svetha Venkatesh Lisette C. Roman, Jessica S. Ancker, Stephen B. Johnson, Yalini Senathirajah Sunghoon Lim, Conrad S. Tucker, Soundar Kumara ENSIAS, University Mohammed V, Rabat, Morocco Columbia Pera. Health, Inc. U Manchester Deakin University Geelong, Australia Weill Cornell Medicine The Pennsylvania 6 State University

Top 10 Downloads: App Clin Inf Key: * = also from Student WG review Rank Title Authors 1 Evaluating the Impact of the Electronic Health Record on Patient Flow in a Pediatric Emergency Department D. J. Mathison (1), J. M. Chamberlain (1) 2 Information needs for the OR and PACU electronic medical record 3 Decay of References to Web sites in Articles Published in General Medical Journals: Mainstream vs Small Journals 4*CHI “Is There An App For That? ” Orthopaedic Patient Preferences For A Smartphone Application 5 Awareness of the Care Team in Electronic Health Records 6* Electronic Health Record Alert-Related Workload as a Predictor of Burnout in Primary Care Providers 7 Enhancing Patient Safety Event Reporting: A Systematic Review of System Design Features 8 Convergent evolution of health information management and health informatics 9* The Emerging Role of the Chief Research Informatics Officer in Academic Health Centers 10* Comparison of EHR-based diagnosis documentation locations to a gold standard for risk stratification in patients with multiple chronic conditions AMIA 2017 | amia. org Institution (lead author) Children's National Med Ctr, Washington, D. C. V. Herasevich (1, 2), M. A. Ellsworth (3), J. R. Hebl (1), M. J. Brown (1), B. W. Pickering Mayo Clinic (1, 2) Shiraz University of Medical P. Habibzadeh Sciences, Shiraz, Iran J. R. Dattilo (1), D. J. Gittings (1), M. Sloan (1), W. M. Hardaker (1), M. J. Deasey (2), N. U Penn P. Sheth (1) D. K. Vawdrey (1), L. G. Wilcox (2), S. Collins (1), S. Feiner (2), O. Mamykina (1), D. Columbia M. Stein (1), S. Bakken (1, 3), M. R. Fred (4), P. D. Stetson (1, 5) M. E. Gregory (1, 2), E. Russo (1, 2), H. Houston VA / Singh (1, 2) Baylor Y. Gong (1), H. Kang (1), X. Wu (1), L. Hua U Texas / (1) Houston C. J. Gibson (1), B. E. Dixon (2, 3, 4), K. Western U, Abrams (5) Ontario L. Sanchez-Pinto (1), A. S. M. Mosa (2), K. Fultz-Hollis (3), U. Tachinardi (4), W. K. U Chicago Barnett (5), P. J. Embi (5) S. Martin (1), J. Wagner (1), N. Lupulescu. Mann (2), K. Ramsey (3), A. Cohen (1), P. OHSU Graven (2), N. G. Weiskopf (1), D. A. Dorr (1) 7

Top 10 Downloads: Int J Med Informatics 1 Factors influencing acceptance of technology for aging in place: A systematic review Peek, S. ; Wouters, E. ; van Hoof, J. ; Luijkx, K. ; Boeije, H. ; Vrijhoef, H. 2 Definition, structure, content, use and impacts of electronic health records: A review of the research literature Häyrinen, K. ; Saranto, K. ; Nykänen, P. 3 Electronic health records implementation: An evaluation of information system impact and contingency factors Nguyen, L. ; Bellucci, E. ; Nguyen, L. 4 Organizational issues in the implementation and adoption of health information technology innovations: An interpretative review Cresswell, K. ; Sheikh, A. 5 6 7 8 9 U Kuopio, Finland Hefei U, China Schooley, B. ; Walczak, S. ; Hikmet, N. ; Patel, N. U South Carolina Smart homes and home health monitoring technologies for older adults: A Liu, L. ; Stroulia, E. ; Nikolaidis, I. ; Miguel-Cruz, A. ; Rios Rincon, systematic review A. Adopting electronic medical records in primary care: Lessons learned from Ludwick, D. ; Doucette, J. health information systems implementation experience in seven countries A literature review for large-scale health information system project planning, implementation and evaluation Deakin University, Melbourne U Edinburgh Visualizing the knowledge structure and evolution of big data research in Gu, D. ; Li, J. ; Li, X. ; Liang, C. healthcare informatics Impacts of mobile tablet computing on provider productivity, communications, and the process of care Fontys U, Netherlands U Alberta Sligo, J. ; Gauld, R. ; Roberts, V. ; Villa, L. U Otago, New Zealand AMIA 2017 | amia. org Swansea U, United 8 10 The other side of the coin: Harm due to the non-use of health-related data Jones, K. ; Laurie, G. ; Stevens, L. ; Dobbs, C. ; Ford, D. ; Lea, N. Kingdom

Top 10 Downloads: Methods Info Med Rank Title Authors 1 Approaches to Regularized Regression - A Comparison between Gradient Boosting and the Lasso Hepp, Tobias; Schmid, Matthias; Gefeller, Olaf; Waldmann, Elisabeth; Mayr, Andreas 2 The New Role of Biomedical Informatics in the Age of Digital Medicine Martin-Sanchez, Fernando J. ; Lopez-Campos, Guillermo H. 3 Technology in Rehabilitation: Evaluating the Single Leg Squat Exercise with Wearable Inertial Measurement Units Whelan, Darragh F. ; O'Reilly, Martin A. ; Ward, Tomas E. ; Delahunt, Eamonn; Caulfield, Brian 4 An Immersive Virtual Reality Platform to Enhance Walking Ability of Children with Acquired Brain Injuries Biffi, Emilia; Beretta, Elena; Cesareo, Ambra; Maghini, Cristina; Turconi, Anna C. ; Reni, Gianluigi; Strazzer, Sandra 5 Representation of People's Decisions in Health Information Systems A Complementary Approach for Understanding Health Care Systems and Population Health Bernaldo de Quiros, Fernan Gonzalez; Dawidowski, Adriana R. ; Figar, Silvana 6 Combined Vision and Wearable Sensors-based System for Movement Analysis in Rehabilitation Spasojevic, Sofija; Ilic, Tihomir V. ; Milanovic, Sladan; Potkonjak, Veljko; Rodic, Aleksandar; Santos-Victor, Jose 7 The Effect of Balance Training on Postural Control in Patients with Parkinson's Disease Using a Virtual Rehabilitation System Albiol-Perez, Sergio; Gil-Gomez, Jose-Antonio; Munoz-Tomas, Maria -Teresa; Gil-Gomez, Hermenegildo; Vial-Escolano, Raquel; Lozano. Quilis, Jose-Antonio 8 Exploring Possibilities for Transforming Established Subscription-based Scientific Journals into Open Access Journals Present Situation, Transformation Criteria, and Exemplary Implementation within Trans-O-MIM Haux, Reinhold; Kuballa, Stefanie; Schulze, Mareike; Boehm, Claudia; Gefeller, Olaf; Haaf, Jan; Henning, Peter; Mielke, Corinna; Niggemann, Florian; Schuerg, Andrea; Bergemann, Dieter 9 Challenges and Opportunities for Harmonizing Research Methodology: Raw Accelerometry van Hees, Vincent T. ; Thaler-Kall, Kathrin; Wolf, Klaus-Hendrik; Brond, Jan C. ; Bonomi, Alberto; Schulze, Mareike; Vigl, Matthaeus; Morseth, Bente; Hopstock, Laila Arnesdatter; Gorzelniak, Lukas; Schulz, Holger; Brage, Soren; Horsch, Alexander 10 Discussion of The New Role of Biomedical informatics in the Age of Digital Medicine Al-Shorbaji, Najeeb; Bellazzi, Riccardo; Bernaldo de Quiros, Fernan Gonzalez; Kochs, Sabine; Kulikowski, Casimir A. ; Lovell, Nigel H. ; Maojo, Victor; Park, Hyeoun-Ae; Sanz, Ferran; Sarkar, Indra N. ; Tanaka, Hiroshi AMIA 2017 | amia. org 9

Title Author(s) Institution (1 st) Summary: the Notable Papers in CI 2017 Crossing the Health IT Chasm: Considerations and Policy Recommendations to Overcome Julia Adler-Milstein, Peter J Embi, Blackford Middleton, Indra U Michigan Current Challenges and Enable Value-based Care Neil Sarkar, and Jeff Smith Semi-supervised learning of the electronic health record for phenotype stratification Brett K. Beaulieu-Jones, Casey S. Greene, the Pooled Resource Open-Access ALS Clinical Trials Consortium U Penn Learning from heterogeneous temporal data in electronic health records Jing Zhao, Panagiotis Papapetrou, Lars Asker, Henrik Boström Stockholm U Templates as a method for implementing data provenance in decision support systems Vasa Curcin, Elliot Fairweather, Roxana Danger, Derek Corrigan King's College, London Electronic Health Record Alert-Related Workload as a Predictor of Burnout in Primary Care Providers M. E. Gregory (1, 2), E. Russo (1, 2), H. Singh (1, 2) The Emerging Role of the Chief Research Informatics Officer in Academic Health Centers L. Sanchez-Pinto (1), A. S. M. Mosa (2), K. Fultz-Hollis (3), U. U Chicago Tachinardi (4), W. K. Barnett (5), P. J. Embi (5) Houston VA / Baylor S. Martin (1), J. Wagner (1), N. Lupulescu-Mann (2), K. Comparison of EHR-based diagnosis documentation locations to a gold standard for risk Ramsey (3), A. Cohen (1), P. Graven (2), N. G. Weiskopf (1), OHSU stratification in patients with multiple chronic conditions D. A. Dorr (1) Deep patient: An unsupervised representation to predict the future of patients from the electronic health records Miotto R, Li L, Kidd BA, Dudley JT. Mt Sinai Enhancing Patient Safety Event Reporting. A Systematic Review of System Design Features. Gong Y, Kang H, Wu X, Hua L. U Tx / Houston Privacy-preserving generative deep neural networks support clinical data sharing Beaulieu-Jones BK, Wu ZS, Williams C, Greene CS. U Penn Prospective Evaluation of a Multifaceted Intervention to Improve Outcomes in Intensive Care: The Promoting Respect and Ongoing Safety Through Patient Engagement Communication and Technology Study* AMIA 2017 | amia. org Patricia C. Dykes, Ph. D, RN 1, 2; Ronen Rozenblum, Ph. D 1, 2; Anuj Dalal, MD 1, 2; Anthony Massaro, MD 1, 2; Frank Chang, MSE 1; Marsha Clements, MSN, RN 1; Sarah Collins, Ph. D, RN 1, 2; Jacques Donze, MD 1; Maureen Fagan, DNP, RN 1; Priscilla Gazarian Ph. D, RN 1; John Hanna, BS 1; Lisa Brigham & Women’s / Harvard Lehmann, MD 1, 2; Kathleen Leone, MBA, RN 1; Stuart Lipsitz, Sc. D 1, 2; Kelly Mc. Nally, BS 1; Conny Morrison, A 1; Lipika Samal, MD, MSc 1, 2; Eli Mlaver, BA 1; Kumiko Schnock, Ph. D 1, 2; Diana Stade BA 1; Deborah Williams, BA 1; 10 Catherine Yoon, MPH 1; David W. Bates, MD, MSc 1, 2

Notable Papers The Finalist Year In Review Papers are presented in no particular order Review format for each: • Title: • Authors: • Institution: • Highlight Points: • Summary: • Memorable take away: AMIA 2017 | amia. org 11

Paper Title: Crossing the Health IT Chasm: Considerations and Policy Recommendations to Overcome Current Challenges and Enable Value-based Care Authors: Julia Adler-Milstein, Peter J Embi, Blackford Middleton, Indra Neil Sarkar, and Jeff Smith Institution: U Michigan Journal: JAMIA Highlight Points: A chasm exists between current health IT ecosystem and that needed to ensure delivery of highvalue care particularly in the face of transitioning system of care delivery to a value-based model. Short-term policy recommendations (from improving patient access to clinical data, to enabling interoperability with an API context, to creating a policy framework for research and innovation) from the perspectives of patients, providers and researchers/innovators are offered to guide policymakers in bridging these gaps. A patient vignette is provided to illuminate the discussion and highlight the differences from current state to the future state. Summary: This article summarizes Health IT challenges faced in providing, receiving, and improving healthcare from the patients’, providers’, and researcher/innovators’ perspectives. It follows this up by providing possible short-term policy recommendations for solutions to overcoming these challenges focusing on current healthcare payment reform of transitioning to a value-based care model and how present health IT infrastructure can be improved to support this transition. Memorable take away: (when you do the YIR you may get to talk about your own paper) A blueprint of technology and policy recommendations to achieve a Health IT infrastructure to support high value care. AMIA 2017 | amia. org 12

Paper Title: Semi-supervised learning of the electronic health record for phenotype stratification. Authors: Beaulieu-Jones BK, Greene CS; Pooled Resource Open-Access ALS Clinical Trials Consortium. Institution: Upenn Journal: JBI Highlight Points: Implementing Denoising Autoencoders can improve visualization and clustering for the discovery of new subtypes of disease by performing dimensionality reduction. They can also improve ALS patient survival. The proposed approach can be applied in datasets where small number of patients have high quality and complete data, a common scenario in EHR research. Summary: The authors developed Denoising Autoencoders for Phenotype Stratification (DAPS) that constructs phenotypes representation using an unsupervised learning algorithm. Denoising Autoencoders (DA) constructs the input vector using a corrupted or noisy input which mimics the incomplete or missing data, a common phenomenon in healthcare data. The authors implemented DA to learn a dense representation of patient record in unsupervised way. The learned vector could be used to extract or represent the phenotype of patient, which could be used in turn in phenotype stratification and classification. They evaluated their approach by implementing DA with Random forest on real ALS dataset. The results demonstrate that using DA with random forests improved classification across multiple simulated models. Hence, DAPS could be used to perform phenotype analysis even for a small cohort with missing data, and could enable processing missing data without performing imputation. Memorable take away: It’s a race between improving the quality of clincial data and learning from messy/missing data. AMIA 2017 | amia. org 13

Paper Title: Templates as a method for implementing data provenance in decision support systems Authors: Vasa Curcin, Elliot Fairweather, Roxana Danger, Derek Corrigan Institution: King’s College, London Journal: JBI Highlight Points: Increasing recognition of the importance of the provenance (of data, knowledge, system, actor, setting, patient) to the acceptance and use of health. IT for quality improvement, CDSSs, etc. Need to understand quality of input data to CDSSs (trust and reproducibility). Data provenance describes what happened to the data to produce it in its current form. The W 3 C PROV standard describes methods to capture all the metadata relevant to a provenance model and produce a semantically annotated graph DB. Summary: CDSS widely available but seldom used for diagnosis … perhaps due to concerns around lack of transparency (‘black box’) and dx performance. Data provenance is a method to improve trust and transparency. “Data provenance templates” are introduced – abstract provenance fragments representing meaningful domain actions. These may be used to generate a model-driven services interface for domain software tools to routinely capture the provenance of their data and tasks. This paper introduces theoretical concept for provenance templates and demonstrates a resulting architecture. These methods were validated in the CDSS developed in the EU FP 7 TRANSFo. Rm Project. Memorable take away: (worthy of further study) You can only trust what you can fully understand – data provenance templates may provide a method to better understand how CDSSs are used. AMIA 2017 | amia. org 14

Paper Title: Electronic Health Record Alert-Related Workload as a Predictor of Burnout in Primary Care Providers Authors: Gregory ME, Russo E, Singh H. Institution: VA Medical Center Houston / Baylor Journal: ACI Highlight Points: Burnout related to alert fatigue is more adequately assessed via subjective measurement. Solving alert fatigue won't be done by objective measurement alone. Additional protected time to manage alerts may alleviate subjective burden perception. 16 Providers answered a questionnaire and participated in focus groups. The key finding was that subjective, but not objective, alert workload was related to two of the three dimensions of burnout, including physical fatigue (p = 0. 02) and cognitive weariness (p = 0. 04), when controlling for organizational tenure. Summary: This paper seeks to measure the objective and subjective burden of asynchronous alerts (a. k. a. “inbox alerts”) as predictors of burnout among providers. Providers studied were from one institution and included physicians, nurse practitioners and physician assistants. Subjective alert workload was strongly correlated with physical fatigue and cognitive weariness. Subjective workload was not correlated with objective workload. To reduce alert workload and subsequent burnout, participants indicated a desire to have protected time for alert management, fewer unnecessary alerts, and improvements to the EHR system. Memorable take away: Physicians: Alert Thyself – possibly with appropriate time, training, and EHR functionality inbox alerts don’t have to be painful. AMIA 2017 | amia. org 15

Paper Title: The Emerging Role of the Chief Research Informatics Officer in Academic Health Centers Authors: L. Nelson Sanchez-Pinto; Abu S. M. Mosa; Kate Fultz-Hollis; Umberto Tachinardi; William K. Barnett; Peter J. Embi Institution: U Chicago Journal: ACI Highlight Points: CRIOs are of multidisciplinary backgrounds with advance training and extensive biomedical informatics experience. Though young in such roles, they are needed to shape the future of clinical research, innovation, and data analytics but face challenges related to funding (63%), data governance (56%), and building data analytics capabilities (50%), leveraging electronic health records for research (44%), dealing with data privacy and legal issues (44%), and securing senior leadership support (30%). Goal of this paper: To increase our understanding of the CRIO role, the leaders who serve it, and the factors associated with their success in their organizations. 16/25 targeted CRIOs responded to questionnaire (64%). Summary: The article examines the functions, significance of and challenges faced by individuals in the emerging role of Chief Research Informatics Officer (CRIO) within academic health centers. 88% first in their institutions in such roles (< 3 years). Memorable take away: Willy Sutton’s Law – seek alignment between institutional strategy, IT, and research = $. AMIA 2017 | amia. org 16

Paper Title: Comparison of EHR-based diagnosis documentation locations to a gold standard for risk stratification in patients with multiple chronic conditions Authors: Shelby Martin; Jesse Wagner; Nicoleta Lupulescu-Mann; Katrina Ramsey; Aaron A. Cohen; Peter Graven; Nicole G. Weiskopf; David A. Dorr Institution: OHSU Journal: ACI Highlight Points: Understanding how variability in storing patient diagnosis in different documentation locations affects risk scores’ ability to accurately predict future patient outcomes and risks such as future utilization costs, hospitalizations, and ED visits. Combining diagnosis from different EMR documentation locations can have similar predictive performance as manual human-mediated gold standard, a labor and time intense process. Summary: the study measured the variations in documenting diagnoses in four EHR-based documentation locations: 1) Problem Lists, 2) Encounter diagnoses, 3) Medical History, and 4) Phenotypic Rule on EHR data. The authors compared the predictive validity of the human generated dx gold standard, and each of the four EHR documentation locations on risk scores predicting three outcomes: ED visits, hospitalizations and healthcare costs. The results demonstrated that around 70% of the diagnoses from the EMR were verified by gold standard. The analysis demonstrated that problem list is the most specific and encounter the most sensitive in the accuracy of the diagnosis documentation. Memorable take away: Prioritize and teach what should be documented where in the EHR to get maximal predictive value out of the EHR. AMIA 2017 | amia. org 17

Paper Title: Deep patient: An unsupervised representation to predict the future of patients from the electronic health records Authors: Riccardo Miotto, Li Li, Brian A. Kidd, Joel T. Dudley Institution: Icahn School of Medicine, Mt Sinai Journal: Nature/Scientific Reports Highlight Points: Challenges in summarizing and representing patient data prevent effective predictive modeling using EHRs. While Deep Learning (hierarchical neural nets with denoising autoencoders) methods have been around for a long time, this is the first scaled application (>700 K patients) of these methods to EHR data. Prediction performance for severe diabetes, schizophrenia, and various cancers were among the top performing. Summary: The “deep patient” leads to more compact and lower dimensional representations than the original EHRs, allowing clinical analytics engines to scale better with the continuous growth of hospital data warehouses. a deep patient representation inferred from EHRs could benefit other tasks as well, such as personalized prescriptions, treatment recommendations, and clinical trial recruitment. Memorable take away: More data = more learning: use the ‘deep patient’ representation for a variety of inference tasks. AMIA 2017 | amia. org 18

Paper Title: Enhancing Patient Safety Event Reporting. A Systematic Review of System Design Features. Authors: Gong Y, Kang H, Wu X, Hua L. Institution: U Texas Houston Journal: ACI Highlight Points: Using a systematic review the authors developed a hierarchical model: 11 system design features and frequency of occurrence of these features were identified and summarized in 48 identified and reviewed e-reporting systems described in existing literature. The top 5: widgets (41), anonymity or confidentiality (29), hierarchy (20), validator (17), review notification (15). The current immature stage of e-reporting systems requires further development of more efficient and effective systems for improved patient safety reporting. Summary: The article highlights the current state of electronic patient safety event reporting (e-reporting) systems. It reveals that current e-reporting systems are at an immature stage of development and would require making them more efficient and effective to meet the needs of improving patient safety. Memorable take away: P. F. Drucker: “You cannot manage what you cannot measure” (after Lord Kelvin) – get patient safety and error reporting right. AMIA 2017 | amia. org 19

Paper Title: Privacy-preserving generative deep neural networks support clinical data sharing Authors: Beaulieu-Jones BK, Wu ZS, Williams C, Greene CS. Institution: Upenn Journal: bio. Rxiv Highlight Points: The authors trained deep neural networks to generate synthetic subjects closely resembling clinical trial participants. The methods incorporate differential privacy, which offers strong guarantees on the likelihood that a subject could (or could not) be identified as a member of the trial. To provide a formal privacy guarantee, they built GANs (generative adversarial networks) under the constraint of differential privacy. They suggest that investigators who have compiled a dataset can use their method to provide a freely accessible public version that enables other scientists to perform discovery-oriented analyses. Generated data can be released alongside analytical code to enable fully reproducible workflows, even when privacy is a concern. Summary: This paper discusses issues related to data sharing with an emphasis on how the need to preserve patient privacy often slows scientific progression. To alleviate this issue, the authors developed and evaluated a deep learning method that generates synthetic patient data that very closely resembles actual patient records. By addressing data sharing challenges, deep neural networks can facilitate the rigorous and reproducible investigation of clinical datasets. Memorable take away: (worthy of further study--consult Jason Moore or others at UPenn) Let’s increase clinical trial data set sharing via synthetic replicants – Blade Runner! AMIA 2017 | amia. org 20

Paper Title: Prospective Evaluation of a Multifaceted Intervention to Improve Outcomes in Intensive Care: The Promoting Respect and Ongoing Safety Through Patient Engagement Communication and Technology Study Authors: Patricia C. Dykes, Ph. D, RN; Ronen Rozenblum, Ph. D; Anuj Dalal, MD; et al. Institution: Brigham and Women’s Hospital / Harvard Journal: Critical Care Medicine Highlight Points: PROSPECT was designed to support integrated, multidisciplinary patient-centered communication through use of shared checklists, health information, and goals of care articulated by both clinicians and patients. The study’s aim was to reduce adverse (falls, pressure ulcers, catheter-associated urinary tract infections and ventilatorassociated events). Secondary aims included improved patient/care partners’ satisfaction. Adverse events were reduced by about a third (baseline: 59 per 1, 000 patient days; intervention: to 41. 9), and patient satisfaction scores improved significantly (71. 8% baseline and 93% intervention). Care partner satisfaction improved, from 84. 3% to 90%. The PROSPECT project is currently being spread to other BWH and Partners Health. Care ICUs. Summary: In this article, an experimental design was used to evaluate the impact of patient engagement, communication with/between care providers, and technology use on overall patient safety outcomes in two medical ICUs at a tertiary care center. In addition to showing decreased adverse patient events and improved patient-provider/provider-provider communication, the study underscores the need for effective and continuous program evaluation. Memorable take away: Ask the Patient “What matters to you? ” – Communicate, engage patients and partners, and use a shared checklist for the things that matter. AMIA 2017 | amia. org 21

Some Random Notable Events

Random Notable Quotes from 2017 Jonathan Bush, CEO, athenahealth • "No one wants to be the last CIO to spend $1 bn on an enterprise EMR" Dr. Ken Mandl, Professor of Medicine, and of Biomedical Informatics, Harvard, Director Computational Health Informatics Program, CHOB • ”We're still buying 20 year old technology & taking 3 years to install it”

Thank You, Student Curators (and Neil)! Raniah Aldekhyyel Gene Ren University of Minnesota OHSU Tiffany Callahan Lina Sulieman University of Colorado, Denver Carnegie Mellon University Karen Chen Jacob Van. Houten Carnegie Mellon University Vanderbilt University Yuqi He Rafeek Yusef University of Wisconsin, Madison UT Houston Todd Lingren Neil Sarkar Brown Cincinnati Children's Hospital AMIA Power. Point Template 24

Thank you! (MOC Q’s follow in deck) Blackford Middleton, MD, MPH, MSc, FACMI blackford. middleton@apervita. com

Questions 1. There is a perception that workload has increased as a result of alerts generated from electronic systems. Actual alert load, which may lead to “alert fatigue, ” is A. Objectively correlated with subjective load B. Not correlated with subjective load C. Difficult to quantify relative to subjective load D. Not important in light of clinical responsibility AMIA 2017 | amia. org 26

Answer: (B) Explanation: The subjective load has been shown to not be correlated with objective load. Subjective load can have a negative impact on clinicians (fatigue and cognitive weariness), and may be addressed through training and changes in perception. AMIA 2017 | amia. org 27

Questions 2. The widespread adoption of electronic health records has increased the role of informatics in clinical practice. Understanding the impact of EHRs, such as clinical workflow, and encompassed tools, such as CPOE, will require: A. Creation of senior management positions to support dialogue B. Federal oversight and incentive programs that better incentivize adoption C. Adoption of existing clinical standards, such as ICD-10 -CM D. The need formal taxonomies for categorizing potential positive and negative impacts AMIA 2017 | amia. org 28

Answer: (D) Explanation: Multiple studies in the last year have shown the need for taxonomies for better quantifying the impact of clinical load and tracking use of electronic health systems. AMIA 2017 | amia. org 29

Questions 3. Recent advances in technology have given rise to increased use of “deep learning” approaches for supporting classification in clinical contexts (e. g. , developing patient “phenotypes”). The success of these deep learning approaches make new use of: a. Electronic health data that are not used by other phenotyping algorithms b. Computational infrastructure that are in secure environments c. Health data standards, like ICD-10 -CM d. Multiple programming languages AMIA 2017 | amia. org 30

Answer: (A) Deep learning approaches are uniquely suited to leverage new types of data, including those that may not be represented using formal knowledge representation approaches. REF: Miotto R, Wang F, Wang S, Jiang X, Dudley JT. Deep learning for healthcare: review, opportunities and challenges. Briefings in Bioinformatics. 2017 May 6: bbx 044. AMIA 2017 | amia. org 31

Questions 4. Patients are increasingly being incorporated into the clinical decision making process. Within intensive care units, patient engagement has been shown to: a. Introduce contradicting data b. Reduce adverse events c. Increase costs of care d. Improve access to clinical data AMIA 2017 | amia. org 32

Answer: (B) It has been shown that involving patients using patient-centered care models can have a marked reduction in preventable adverse events. • REF: Prospective Evaluation of a Multifaceted Intervention to Improve Outcomes in Intensive Care: The Promoting Respect and Ongoing Safety Through Patient Engagement Communication and Technology Study* Patricia C. Dykes, Ph. D, RN 1, 2; Ronen Rozenblum, Ph. D 1, 2; Anuj Dalal, MD 1, 2; Anthony Massaro, MD 1, 2; Frank Chang, MSE 1; Marsha Clements, MSN, RN 1; Sarah Collins, Ph. D, RN 1, 2; Jacques Donze, MD 1; Maureen Fagan, DNP, RN 1; Priscilla Gazarian Ph. D, RN 1; John Hanna, BS 1; Lisa Lehmann, MD 1, 2; Kathleen Leone, MBA, RN 1; Stuart Lipsitz, Sc. D 1, 2; Kelly Mc. Nally, BS 1; Conny Morrison, A 1; Lipika Samal, MD, MSc 1, 2; Eli Mlaver, BA 1; Kumiko Schnock, Ph. D 1, 2; Diana Stade BA 1; Deborah Williams, BA 1; Catherine Yoon, MPH 1; David W. Bates, MD, MSc 1, 2 Crit Care Med. 2017 Aug; 45(8): e 806 -e 813 AMIA 2017 | amia. org 33

Questions 5. EHR systems present complex interfaces that can hinder clinical workflow. Safety is impacted by navigation according to usability studies that: a. Demonstrate poor use of colors in EHR implementation b. Introduce education for improving ability to manage more cognitive load c. Show that navigation involves significant cognitive load d. Highlight how EHRs are difficult to use compared to paper AMIA 2017 | amia. org 34

Answer: (C) The majority of usability studies include a component related to challenges in navigation. REF: Roman LC, Ancker JS, Johnson SB, Senathirajah Y. Navigation in the electronic health record: A review of the safety and usability literature. J Biomed Inform. 2017; 67: 69 -79. AMIA 2017 | amia. org 35

Questions 6. Increased need to harness electronic health data to support clinical research in academic health centers has given rise to the Chief Research Information Officer (CRIO). Effective CRIOs: a. Replace the clinical facing roles of the Chief Information Officer b. Provide guidance on how health data can be used within research contexts c. Come from consistent backgrounds d. Develop approaches to better support clinical practice with basic science research data AMIA 2017 | amia. org 36

Answer: (B) A recent survey of CRIOs reveals that they come from diverse backgrounds, which supports their ability to develop new approaches for harnessing clinical data for research purposes. REF: Sanchez-Pinto LN, Mosa ASM, Fultz-Hollis K, Tachinardi U, Barnett WK, Embi PJ. The Emerging Role of the Chief Research Informatics Officer in Academic Health Centers. Appl Clin Inform. 2017 Aug 16; 8(3): 845 -853. doi: 10. 4338/ACI-2017 -04 -RA-0062. Pub. Med PMID: 28832068. AMIA 2017 | amia. org 37