SNOMED historical associations A Quality Review JanuaryMay 2020

SNOMED historical associations: A Quality Review January-May 2020 Paul Amos, Jeremy Rogers, Jeff Pierson, Brian Carlsen, Anne Randorff Højen

Outline • How have concept inactivation and historical associations been used? • What problems are caused by incorrect usage? • How much is wrong? • Prevention : Sins of the Future • Rehabilitation : Sins of the Past

Concept Inactivations: Usage through time

2002 Reconciliation of CTV 3 -RT duplicates 2010 Mass inactivation of Limited Status 2009 Return of ~10 k “square bracket” ICD codes to UK 2018 Dr Case becomes CTO ; -) 2014 US Veterinary Extension 2019 Pharmacy Redesign

Effects of inactivation on cumulative EPR Old EPR data - as viewed in 2018 UK Primary Care (extrapolated) 2 000 000 4, 5 1 750 000 4 1 500 000 3, 5 3 1 250 000 2, 5 1 000 000 2 750 000 10 billion primary care EPR entries 2011 -2018 2% inactive by 2018 2. 5% <<275498002 Respiratory tract infection 28, 977, 526 active 755, 753 inactive 0. 44% <<363346000 Cancer 6. 3% <<74732009 Mental disorder 21% <<609328004 Allergy 1, 5 500 000 1 250 000 0, 5 0 0 2011 2012 Inactive 2013 2014 Active 2015 Inactive% 2016 2017 6. 8% of 888 k ED diagnoses 2009 -2017

Before October 2011… …changes happen… After October 2011… 363660007 Ambiguous concept 363662004 Duplicate concept 404684003 Clinical Finding 275498002 Respiratory Tract Infection Naïve query for RTI…. …never found these. 54150009 Upper respiratory tract infection IF you only consider the Now it won’t find this either IS-A relationships 195746005 Recurrent chest infection [V 2 H 06 z 2] 128486005 Upper respiratory bacterial infection (inactivated Jul 2002) MAY_BE_A SAME_AS 312118003 Bacterial upper respiratory infection [V 3 Xa. Dc. C] 50417007 Lower respiratory tract infection 308130008 448739000 Recurrent lower respiratory chest infection tract [V 3 infection Xa. BM 8]

Historical Associations: Does it matter if they’re wrong?

-- 183 151 CODES across 57672 47543 EPISODES met the report definition (6. 5% (5. 4% of all Episodes) 23078 episodes of 21522001 Abdominal pain (finding) 40. 02% 48. 54% 3110 episodes of 499111000000101 7723 7093002 Renal colic[D]Abdominal (finding) 55. 08% pain (situation) 53. 41%** 2423 episodes of 7093002 3110 249519007 Renal Diarrhea colic (finding) and vomiting 58. 8%(finding) 60. 18% 1745 episodes of 249519007 2423 62315008 Diarrhea(finding) and vomiting 63. 85% (finding) 63% 1689 episodes of 62315008 1745 12063002 Diarrhea Rectal hemorrhage (finding) 66. 03% (disorder) 67. 4% 1532 episodes of 12063002 1689 314212008 Rectal Abdominal hemorrhage pain - cause (disorder) unknown 68. 96% (finding) 70. 62% 1405 episodes of 314212008 1532 85189001 Acute Abdominal appendicitis pain - cause (disorder) unknown 73. 58% (finding) 71. 61% 1177 episodes of 85189001 1405 79922009 Acute Epigastric appendicitis pain (finding) (disorder) 76. 06% 74. 05% 1114 episodes of 79922009 1177 409966000 Epigastric Acute diarrhea pain (finding) (disorder) 76. 09% 78. 4% 991 episodes 1114 episodesofof 18165001 409966000 Jaundice Acute diarrhea (finding) (disorder) 80. 48% 78. 02% 970 episodes of 18165001 991 314041007 Jaundice Abdominal (finding) pain in 79. 74% early pregnancy (finding) 82. 52% 943 episodes of 314041007 970 197119006 Abdominal Acute constipation pain in early (disorder) pregnancy 84. 51% (finding) 81. 42% 712 episodes of 197119006 943 74400008 Appendicitis Acute constipation (disorder) 86% 83. 06% 693 episodes of 74400008 712 444673007 Appendicitis Hyperemesis (disorder) 84. 29% 87. 46% 503 episodes of 444673007 693 37389005 Biliary Hyperemesis colic (finding) (disorder) 88. 52% 85. 49% 472 episodes of 502841000000108 656 102570003 Inguinal[D]Nonspecific pain (finding) 89. 51% abdominal pain (situation) 86. 63%** 386 episodes of 498731000000103 574 266599000 Dysmenorrhea [D]Renal(disorder) colic (situation) 90. 32%87. 62%** 383 episodes of 37389005 503 236069009 Biliary Chronic colic constipation (finding) 88. 5% (disorder) 91. 13% 379 episodes of 102570003 472 54586004 Lower Inguinal abdominal pain (finding) 89. 32% 91. 93% 364 episodes of 266599000 386 309737007 Dysmenorrhea Abdominal pain(disorder) in pregnancy 89. 98% (finding) 92. 69% 261 episodes of 236069009 383 274279008 Chronic Renal pain constipation (finding) 93. 24% (disorder) 90. 65% 260 episodes of 54586004 379 9991008 Abdominal Lower abdominal colic (finding) pain (finding) 93. 79% 91. 31% 246 episodes of 309737007 364 304542004 Abdominal Nonspecificpain abdominal in pregnancy pain (finding)94. 31% 91. 94% 237 episodes of 428811000000108 294 271857006 Loin pain [D]Icterus (finding)NOS 94. 8% (situation) 92. 45%** 178 episodes of 274279008 261 34000006 Crohn's Renal pain disease (finding) (disorder) 92. 9%95. 18% 160 episodes of 9991008 260 17329003 Abdominal Ureteric colic(finding)95. 52% 93. 35% 150 episodes of 304542004 246 247355005 Nonspecific Flank pain (finding) abdominal 95. 83% pain (finding) 93. 78% 139 episodes of 271857006 237 111985007 Loin Chronic painabdominal (finding) 94. 19% pain (finding) 96. 12% 133 episodes of 499161000000104 228 77880009 Rectal pain [D]Epigastric (finding) 96. 4% pain (situation) 94. 58%** 108 episodes of 34000006 178 95545007 Crohn's Hemorrhagic disease diarrhea (disorder) 94. 89%96. 63% 106 episodes of 390251000000109 161 21005005 Burning epigastric [D]Abdominal pain (finding) NOS 96. 85% (situation) 95. 17%** 83 episodes 160 episodesofof 162051008 17329003 Ureteric Right iliaccolic fossa (finding) pain (finding) 95. 45%97. 03% 80 episodes 150 episodesofof 285387005 247355005 Left Flank sided painabdominal (finding) 95. 71% pain (finding) 97. 2% 69 episodes 139 episodesofof 272047006 111985007 Complaining Chronic abdominal of loin pain(finding)97. 34% 95. 95% 68 episodes 138 episodesofof 285388000 502591000000100 Right sided [D]Upper abdominal pain (finding) pain (situation) 97. 48% 96. 19%** 66 episodes 133 episodesofof 116290004 77880009 Rectal Acute abdominal pain (finding) pain 96. 42% (finding) 97. 62% 63 episodes 114 episodesofof 236071009 468471000000105 Chronic diarrhea [X]Diarrhea (disorder) and gastroenteritis 97. 76% of presumed infectious origin (disorder) 96. 62%** ? ? FALSE +VE ? ? 61 episodes 108 episodesofof 35363006 95545007 Infantile Hemorrhagic colic (finding) diarrhea 97. 88% (disorder) 96. 8% 53 episodes 106 episodesofof 274288004 21005005 Burning On examination epigastric - epigastric pain (finding) pain 96. 99% (finding) 98% 42 episodes of 162051008 83 43548008 Mittelschmerz Right iliac fossa (finding) pain (finding) 98. 08%97. 13% 40 episodes of 285387005 80 162049009 Left sided flank pain abdominal (finding) pain 98. 17% (finding) 97. 27% 38 episodes of 499221000000103 71 162052001 Left iliac[D]Groin fossa pain (finding) (situation) 98. 25% 97. 39%** 38 episodes of 272047006 69 162042000 Complaining Abdominal wall of loin pain (finding) 98. 33% 97. 51% 32 episodes of 285388000 68 301717006 Right sided upperabdominal quadrant pain(finding)98. 4% 97. 63% 31 episodes of 116290004 66 162053006 Acute Suprapubic abdominal pain (finding) 98. 46% 97. 75% 30 episodes of 236071009 63 162050009 Chronic Right flank diarrhea pain (finding) (disorder) 98. 52% 97. 86% 28 episodes of 35363006 61 271681002 Infantile Stomachcolic ache (finding) 97. 96% 98. 58% 27 episodes of 274288004 53 235841007 On Chronic examination nonspecific - epigastric abdominal pain (finding) 98. 05% 98. 64% 25 episodes of 43548008 42 74704000 Mittelschmerz Abdominal pain(finding) through 98. 13% to back (finding) 98. 69% 24 episodes of 162049009 40 83132003 Upper Left flank abdominal pain (finding) 98. 19%98. 74% 23 episodes of 499141000000100 40 162046002 Central [D]Infantile abdominal pain colic(finding) (situation) 98. 79% 98. 26%** 23 episodes of 162042000 38 426549001 Abdominal Crohn's disease wall pain in remission (finding)(disorder) 98. 33% 98. 84% 22 episodes of 162052001 38 75879005 Abdominal Left iliac fossa migraine pain (finding) (disorder)98. 4% 98. 89% 22 episodes of 236076004 35 371102005 Infective Generalized diarrhea colicky(disorder) abdominal 98. 46%** pain (finding) 98. 93% “Abdominal Presentations” One ED dept, 7 years, 888 k attendances 1000 900 800 700 600 500 300 200 100 0 Oct '08 Dec '08 Feb '09 Apr '09 Jun '09 Aug '09 Oct '09 Dec '09 Mar '10 May '10 Jul '10 Sep '10 Nov '10 Jan '11 Mar '11 May '11 Jul '11 Sep '11 Nov '11 Jan '12 Mar '12 May '12 Jul '12 Sep '12 Nov '12 Jan '13 Mar '13 May '13 Jul '13 Sep '13 Nov '13 Jan '14 Mar '14 May '14 Jul '14 Sep '14 Nov '14 Jan '15 400 INCLUDE: << 74400008 Appendicitis (code and all its descendants) << 197119006 Acute constipation (code and all its descendants) << 236069009 Chronic constipation (code and all its descendants) << 62315008 Diarrhoea (code and all its descendants) = 444673007 Hyperemesis (code only and NOT also its descendants) << 34000006 Crohn's disease (code and all its descendants) << 21522001 Abdominal pain (code and all its descendants) << 12063002 Rectal haemorrhage (code and all its descendants) << 18165001 Jaundice (code and all its descendants)

Historical Associations: Intended role in Data Repair Technical Implementation Guide July 2009

Historical Associations: Intended role in Data Repair Technical Implementation Guide July 2009

Historical Associations: How many are wrong?

Quality Review of Historical Associations: Methodology Substrate: SNOMED Int Ed July 2018 + UK Clinical and Drug For all 91, 756 inactive concepts, identify: Date on which inactivated Reason for inactivation All already explicit or complex inferable historical associations Lexical similarity of original and target FSNs Partition resulting 302, 981 rows between 26 subcategories Randomly select (up to) 100 from each category for review Dual independent review all items selected, marking each as either Yes, No or OK

Quality Review of Historical Associations: Sample of review data

Quality Review of Historical Associations: Outcome and headline results • 1926 inactive concepts with at least one historical association 400 431 994 101 0 reviews 1 review 2 reviews 3 reviews • High reviewer disagreement 44% (4 -92%) • About 30 -40% of all historical associations are wrong

Quality Review of Historical Associations: Examples of errors

Quality Review of Historical Associations: Results Duplicate, erroneous, outdated, moved, limited status, non-conformant 12% incorrect Ambiguous 54% incorrect

Discussion: Prevention and Rehab

Discussion questions • How do we reduce errors going forward? • How much of the historical data do we need to repair? • All of it, or just some? Which bit first? • Targetting options • • • inactivation reason flavour of association pre-inactivation semantic type usage of inactivated concept or its designated substitute(s) can we automate any repairs? • Algorithms with high precision but poor recall could still exempt much of the “good” historical data from review