Detection of underspecifications in SNOMED CT concept definitions

  • Slides: 28
Download presentation
Detection of underspecifications in SNOMED CT concept definitions using language processing Edson Pacheco 1,

Detection of underspecifications in SNOMED CT concept definitions using language processing Edson Pacheco 1, 2, Holger Stenzhorn 3, Percy Nohama 1, Jan Paetzold 3, 4, Stefan Schulz 2, 3, 4 1 Federal Technical University of Paraná (UTFPR), Curitiba, Brazil; 2 Pontifical Catholic University of Paraná (PUCPR), Curitiba, Brazil; 3 Institute of Medical Biometry und Medical Informatics, University Medical Center Freiburg; 4 AVERBIS Gmb. H, Freiburg, Germany Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions SNOMED CT • “Standardized Nomenclature of Medicine - Clinical

Background Methods Evaluation Results Conclusions SNOMED CT • “Standardized Nomenclature of Medicine - Clinical Terms” • Comprehensive clinical terminology ( > 300, 000 representational units) • Concepts are arranged in extensive taxonomic (is-a) hierarchies Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Taxonomic Structure of SNOMED CT Relation: is-a Detection of

Background Methods Evaluation Results Conclusions Taxonomic Structure of SNOMED CT Relation: is-a Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions SNOMED CT • “Standardized Nomenclature of Medicine - Clinical

Background Methods Evaluation Results Conclusions SNOMED CT • “Standardized Nomenclature of Medicine - Clinical Terms” • Comprehensive clinical terminology ( > 300, 000 representational units) • Concepts are arranged in extensive taxonomic (is-a) hierarchies • Cross-reference between concepts from several branches via semantic relations obeying description logics semantics Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Cross-reference between SNOMED CT concepts Detection of underspecifications in

Background Methods Evaluation Results Conclusions Cross-reference between SNOMED CT concepts Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions SNOMED CT semantics in a nutshell A: C: B,

Background Methods Evaluation Results Conclusions SNOMED CT semantics in a nutshell A: C: B, D: C B r S A A – is. A – C A–r–B A–s–D concept taxonomic parent attributes D A sub. Class. Of C and r some B and s some D EL+ description logics Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions SNOMED CT • “Standardized Nomenclature of Medicine - Clinical

Background Methods Evaluation Results Conclusions SNOMED CT • “Standardized Nomenclature of Medicine - Clinical Terms” • Comprehensive clinical terminology ( > 300, 000 representational units) • Concepts are arranged in extensive taxonomic (is-a) hierarchies • Cross-reference between concepts from several branches via semantic relations obeying description logics semantics • Burden of terminology content maintenance and quality assurance Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions SNOMED CT • “Standardized Nomenclature of Medicine - Clinical

Background Methods Evaluation Results Conclusions SNOMED CT • “Standardized Nomenclature of Medicine - Clinical Terms” • Comprehensive clinical terminology ( > 300, 000 representational units) • Concepts are arranged in extensive taxonomic (is-a) hierarchies • Cross-reference between concepts from several branches via semantic relations obeying description logics semantics • Burden of terminology content maintenance and quality assurance • To be supported by automated approaches Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Looking for underspecifications of cross-linkage • Nearly half (45.

Background Methods Evaluation Results Conclusions Looking for underspecifications of cross-linkage • Nearly half (45. 2%) of the SNOMED CT concepts (132, 125) have no attributes. • Textual descriptions suggest composed meanings • Examples: – Cerebral function • only related to its parent Nervous system function • expected relation with Brain structure missing – Hepatitis notification • only related to its parent Disease notification • expected relation with Inflammatory disease of liver missing Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Automatic suggestion of attributes • Source: 01/2009 release of

Background Methods Evaluation Results Conclusions Automatic suggestion of attributes • Source: 01/2009 release of SNOMED CT Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Automatic suggestion of attributes • Source: 01/2009 release of

Background Methods Evaluation Results Conclusions Automatic suggestion of attributes • Source: 01/2009 release of SNOMED CT • Algorithm: 1. Identify non-attributed concepts Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Select non-attributed SNOMED CT concepts Detection of underspecifications in

Background Methods Evaluation Results Conclusions Select non-attributed SNOMED CT concepts Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Select non-attributed SNOMED CT concepts Detection of underspecifications in

Background Methods Evaluation Results Conclusions Select non-attributed SNOMED CT concepts Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Automatic suggestion of attributes • Source: 01/2009 release of

Background Methods Evaluation Results Conclusions Automatic suggestion of attributes • Source: 01/2009 release of SNOMED CT • Algorithm: 1. Identify non-attributed concepts 2. Extract concept names Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Extract names of non-attributed concepts “notification of disease” “notification

Background Methods Evaluation Results Conclusions Extract names of non-attributed concepts “notification of disease” “notification of AIDS” “hepatitis notification” Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Automatic suggestion of attributes • Source: 01/2009 release of

Background Methods Evaluation Results Conclusions Automatic suggestion of attributes • Source: 01/2009 release of SNOMED CT • Algorithm: 1. Identify non-attributed concepts 2. Extract concept names 3. Perform semantic abstraction from word to sets of morphosemantic identifiers (MIDs) Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Perform morphosemantic abstraction Using Morpho. Saurus* morphosemantic indexing “notification

Background Methods Evaluation Results Conclusions Perform morphosemantic abstraction Using Morpho. Saurus* morphosemantic indexing “notification of disease” #report #disease “notification of AIDS” #report #aids “hepatitis notification” #liver #inflamm #report *Markó K, Schulz S, Hahn U: Morpho. Saurus - Design and Evaluation of an Interlingua-based, Cross-language Document Retrieval Engine for the Medical Domain. Meth Inf Med 4/2005(44): 537 -545. Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Automatic suggestion of attributes • Source: 01/2009 release of

Background Methods Evaluation Results Conclusions Automatic suggestion of attributes • Source: 01/2009 release of SNOMED CT • Algorithm: 1. Identify non-attributed concepts 2. Extract concept names 3. Perform semantic abstraction from word to sets of morphosemantic identifiers (MIDs) 4. Compare MID sets between children and parents and reduce child sets Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Automatic suggestion of attributes • Source: 01/2009 release of

Background Methods Evaluation Results Conclusions Automatic suggestion of attributes • Source: 01/2009 release of SNOMED CT • Algorithm: 1. Identify non-attributed concepts 2. Extract concept names 3. Perform semantic abstraction from word to sets of morphosemantic identifiers (MIDs) 4. Compare MID sets between children and parents and reduce child sets 5. Match reduced child set against MID representations of all SNOMED descriptions Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Matching heuristics • For the FSN MID set of

Background Methods Evaluation Results Conclusions Matching heuristics • For the FSN MID set of every non-attributed concept: – remove MID that occurs in any of this concept’s parents – check whether the remainder set coincides with the MID representation of some other SNOMED CT concept, considering all descriptions (FSNs, PTs, synonyms) – consider this concept a refinement candidate SNOMED description FSN: “notification of disease” FSN: “hepatitis notification” “inflammatory FSN: disease of liver” = SYN: “hepatitis” MID set #report #disease #liver #inflamm #report #inflamm #disease #liver #inflamm Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Automatic suggestion of attributes • Source: 01/2009 release of

Background Methods Evaluation Results Conclusions Automatic suggestion of attributes • Source: 01/2009 release of SNOMED CT • Algorithm: 1. Identify non-attributed concepts 2. Extract concept names 3. Perform semantic abstraction from word to sets of morphosemantic identifiers (MIDs) 4. Compare MID sets between children and parents and reduce child sets 5. Match reduced child set against MID representations of all SNOMED descriptions 6. Suggest candidates for refining attributes Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Addition of refinement candidate “notification of disease” “inflammatory disease

Background Methods Evaluation Results Conclusions Addition of refinement candidate “notification of disease” “inflammatory disease of liver” = “hepatitis” #liver #inflamm #report #disease “hepatitis notification” #liver #inflamm #report *Markó K, Schulz S, Hahn U: Morpho. Saurus - Design and Evaluation of an Interlingua-based, Cross-language Document Retrieval Engine for the Medical Domain. Meth Inf Med 4/2005(44): 537 -545. Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Evaluation Methodology • For each of 14 SNOMED subhierarchies:

Background Methods Evaluation Results Conclusions Evaluation Methodology • For each of 14 SNOMED subhierarchies: random sample of 20 underspecified concepts, compared to attribute refinement candidate proposed by the system • For each of the sample concept verify 1. whether this concept should be refined 2. whether one of the suggested refinement candidates can be plausibly used for refinement. • Performed by two domain experts. Double rating for interrater agreement measurement: 25% Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Results of retrieval experiments Detection of underspecifications in SNOMED

Background Methods Evaluation Results Conclusions Results of retrieval experiments Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Results of retrieval experiments Active Concepts SNOMED hierarchies Organism

Background Methods Evaluation Results Conclusions Results of retrieval experiments Active Concepts SNOMED hierarchies Organism Substance body structure qualifier value observable entity Finding physical object morphologic abnormality Occupation Product Event Disorder Procedure Others TOTAL 31840 23554 25637 8823 7885 32780 4408 Non-attributed Concepts n % Refinement candidates n % 31840 23554 22386 8823 7885 5356 4408 100. 0 87. 3 100. 0 16. 3 100. 0 4973 8627 15076 3533 3647 2253 1339 15. 6 36. 6 58. 8 40. 0 46. 3 6. 9 30. 4 4297 4289 3843 19310 3541 3578 3529 63874 2812 47764 2256 14511 7603 292104 132125 99. 8 100. 0 18. 3 98. 6 4. 4 4. 7 52. 4 45. 2 2164 1330 686 447 1080 1001 2396 48552 50. 4 34. 6 3. 6 12. 5 1. 7 2. 1 16. 5 16. 6 Sample based Analysis of estimation samples(n=20) refinecorrect refinable with correct ment suggest concepts suggestjustified ions 0% 0% 0 0 55% 35% 4700 3000 5% 0% 800 0 0% 0% 0 0 70% 50% 2600 1800 90% 75% 2000 1700 85% 80% 1100 80% 75% 100% 85% 90% 85% 75% 60% 10% 60% 45% 60% 65% 60% 1700 1000 700 400 1000 900 18700 1300 100 400 200 600 700 1400 12300 Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions Results • Interrater agreement (Kohen’s kappa): – A concept

Background Methods Evaluation Results Conclusions Results • Interrater agreement (Kohen’s kappa): – A concept should be refined: 0. 55 (low !) – There is a proposed refinement candidate: 0. 74 • Estimation: approximately 18, 000 SNOMED CT concepts can be refined. • Problematic suggestions: – Macaroni for Macaroni maker – Canada for Salmonella canada – Acyl carnitine for Acylcarnitine hydrolase – First for Female first cousin (already fully defined by the intersection of First cousin and Female cousin) Detection of underspecifications in SNOMED CT concept definitions using language processing

Background Methods Evaluation Results Conclusions • Many SNOMED CT concepts are underdefined and can

Background Methods Evaluation Results Conclusions • Many SNOMED CT concepts are underdefined and can / should be refined • The proposed methodology was useful to detect underspecifications • Large difference between SNOMED hierarchies re harvesting and approval of refinement candidates • “Grey areas” – many proposed refinements are debatable – only part of refinement candidates not retrieved due to restrictions of the methodology • Should be considered for future SNOMED CT editing policies Detection of underspecifications in SNOMED CT concept definitions using language processing

Acknowledgements: Thank You! Contact: CNPq, Brazil: 550830/2005 -7 Stefan Schulz http: //purl. org/steschu BMBF-IB,

Acknowledgements: Thank You! Contact: CNPq, Brazil: 550830/2005 -7 Stefan Schulz http: //purl. org/steschu BMBF-IB, Germany: BRA 05/022 Detection of underspecifications in SNOMED CT concept definitions using language processing

Detection of underspecifications in SNOMED CT concept definitionsDetection of underspecifications in SNOMED CT concept definitions
SNOMED CT and SNOMED International Ian Green CustomerSNOMED CT and SNOMED International Ian Green Customer
SNOMED Training Workshop SNOMED CT in primary careSNOMED Training Workshop SNOMED CT in primary care
SNOMED Training Workshop SNOMED CT in primary careSNOMED Training Workshop SNOMED CT in primary care
Introduction to SNOMED CT Why SNOMED CT FeaturesIntroduction to SNOMED CT Why SNOMED CT Features
SNOMED CT Order Set Strategy SNOMED International NursingSNOMED CT Order Set Strategy SNOMED International Nursing
Mapping from antecedent SNOMED to SNOMED CT inMapping from antecedent SNOMED to SNOMED CT in
LOINC SNOMED CT Cooperation on Content SNOMED InternationalLOINC SNOMED CT Cooperation on Content SNOMED International
SNOMED Clinical Terms Snomed CT the language forSNOMED Clinical Terms Snomed CT the language for
Introduction to SNOMED CT Why SNOMED CT FeaturesIntroduction to SNOMED CT Why SNOMED CT Features
SNOMED CT and SNOMED International Nursing CRG IanSNOMED CT and SNOMED International Nursing CRG Ian
Concept Maps Definitions of concept maps A conceptConcept Maps Definitions of concept maps A concept
ORAL PATHOLOGY DEFINITIONS PATHOLOGY DEFINITIONS PATHOLOGY DIAGNOSIS DEFINITIONSORAL PATHOLOGY DEFINITIONS PATHOLOGY DEFINITIONS PATHOLOGY DIAGNOSIS DEFINITIONS
Genetic Definitions Karyotype Genetic Definitions Genetic Definitions GeneticGenetic Definitions Karyotype Genetic Definitions Genetic Definitions Genetic
Species definitions Species definitions Phenetic species concept ASpecies definitions Species definitions Phenetic species concept A
An extended SNOMED CT concept model for observationsAn extended SNOMED CT concept model for observations
Representing SNOMED CT Concept Evolutions using Process ProfilesRepresenting SNOMED CT Concept Evolutions using Process Profiles
Face detection Face detection Stateoftheart face detection demoFace detection Face detection Stateoftheart face detection demo
Collision Detection Collision Detection in Games Collision detectionCollision Detection Collision Detection in Games Collision detection
Intrusion Detection System 1 Intrusion Detection Intrusion detectionIntrusion Detection System 1 Intrusion Detection Intrusion detection
Edge Detection Edge detection n n Edge detectionEdge Detection Edge detection n n Edge detection
Assembly Method Concepts Concept 2 Concept 1 ConceptAssembly Method Concepts Concept 2 Concept 1 Concept
Chapter 6 Concept Modeling Concept Modeling Contents ConceptChapter 6 Concept Modeling Concept Modeling Contents Concept
Today Eerste Concept Tweede Concept Defenitief Concept OnderzoekToday Eerste Concept Tweede Concept Defenitief Concept Onderzoek