National Information Exchange Model NIEM Building a NIEM

National Information Exchange Model (NIEM) Building a NIEM IEPD: the NIEM IEPD Life Cycle and Tools 6 September 2006 John Wandelt and Mark Kindl Georgia Tech Research Institute

NIEM Builds on GJXDM NIEM = GJXDM + Expanded Scope + Lessons Learned • Incremental changes to GJXDM to address specific issues and expanded scope • A few examples: – Expanded Domains (Intel, International Trade, Emergency Mgt, Infrastructure Protection, etc. ) – Better Modularity – Improve Reusability - Leveraging work of others – Discovery of IEPDs and components – Capturing Business Context – Incorporating feedback into the model - Model Refinement through use – Support for components from external standards – IEPD consistency 2

NIEM Use Cases • To effectively exchange information, there must be a common semantic understanding of data among participants, and the data must be formatted in a consistent manner. • NIEM provides reusable components and IEPDs for exchanges • Between NIEM domains (inter-domain). • Within NIEM domains (intra-domain) (where multiple sub communities exist within a single NIEM domain, e. g. justice). 3

IEPD Development vs the Model IEPD Development The Model – NIEM reusable components • National priorities (e. g. NDEx, TWPDES) • Authoritative sources (e. g. Rap Sheet) • Local requirements NIEM NDR reuse / revise reuse add / replace IEPD Library (virtual) add / replace NIEM 4

NIEM Naming and Design Rules • Provides a normative technical reference for building NIEM data model components and IEPDs. • Draft NIEM NDR will be available for review and comment with the 1. 0 RC release. • Specifies some of the following: – – – – Use of types, elements, attributes, groups Enumerations (code lists) Annotations (documentation and app. Info) Namespaces ISO 11179 naming External standards use (e. g. Geospatial) Creating new entities (objects) • • Composition Specialization w/ inheritance Associations (relationships) Roles – Adding properties to existing entities • Type augmentation • Metadata 5

NIEM Architecture (Model View) NIEM Total Migration (special case) External domains or standards NIEM Core Universal Structures Justice (b) Extract / Use Common (GJXDM) Justice specific External Domain 1 XN te candida J Im Intel IT IP EM External domains or standards Sc ZN . . Code. Lists Create (boot strap) External Domain 3 Z NIEM Domains Adopt (a) Translate / Map External Domain 2 Y Y Wrap / Refer 6

What is a NIEM IEPD? Documentation and specification for describing NIEM information exchanges. Contains: • A set of NIEM conforming XML schemas that – (Re)use or correctly extend NIEM components. – Define a class of XML exchange instances. • Documentation detailing – How to implement the IEP with the schemas. – Exchange model, business rules, use cases, etc. – Example instances, stylesheets • Metadata for indexing, search, discovery, maintenance, registration, etc. 7

NIEM XML Reference Architecture XML Schema *Code list xsd NIEM xsd subset Conformance Validation *Constraint xsd *Extension xsd Constraint Validation *Exchange xsd Namespace * Optional Exchange xml IEPDs have their own target namespaces 8

IEPD Component Perspective NIEM IEPD schemas are NIEM conforming. IEPD Extension External 0 4 4 5 External 1 6 6 Subset External domains or standards 1 External 3 2 External 2 3 3 2 2 1 NIEM 3 1, 5 – create 2, 4 – wrap / refer 3 – translate / map 6 – adopt 9

The IEPD Life Cycle • General framework for building IEPDs in logical steps. • Primary process from the practitioner perspective. • Guide to understanding how IEPDs are ideally built and published. • Not prescriptive, IEPD builder … – Adjusts the scope of the framework to the IEPD level of effort. – Enters the life cycle at appropriate step. • The value of formal specifications for input/outputs of the IEPD Life Cycle. – Ensure consistency across IEPDs. – Capture business context to facilitate search and discovery of NIEM components and IEPDs. – Machine readable and process-able to automate support for the IEPD Life Cycle. – Encourage and facilitate commercial tool development and value added capabilities. 10

Executive orders, mission statements, business requirements, business context, policies/procedures 0 Scenarios, BRM Taxonomy IEPD for reuse library Specific information exchange, high-level business requirements, existing IEPDs NIEM 1 Info exchange requirements 5 Publish + implement Complete IEPD NIEM NDR NIEM IEPD Life Cycle Data requirements, business context, exchange model Submission of candidate NIEM components, component business context 2 Map + model Existing IEPD artifacts to share/reuse, metadata for discovery 4 Assemble to IEPD spec 3 Build + validate Valid IEPD schemas, example instances, documentation, metadata Gap analysis, specs for new components NIEM NDR

IEPD Life Cycle Feeds the Data Model Maturity Life Cycle Top down Bottom up IEPDs NBAC / NTAC: vet, harmonize, conformance 0 Scenarios, BRM taxonomies 1 Info exchg requirements Staging Area 5 Publish + implement NIEM IEPD Life Cycle 4 Assemble to IEPD spec 2 Map + model Early Awareness & coordination NIEM Governance NIEM 3 Build + validate New component candidates & business context Groups with common data interests 13

From the IEPD Life Cycle Perspective GJXDM Tools • Justice Information Exchange Model (JIEM) Tool • JIEM-GJXDM direct mapping capability (through SSGT) • Schema Subset Generation Tool (SSGT) • Moving forward • GJXDM Spreadsheet • IEPD Clearinghouse NIEM Tools • Component Mapping Template (CMT) • Graphical Browser • Schema Subset Generation Tool (SSGT) • Information Exchange Package Documentation (IEPD) Tool • Toward NIEM 1. 0 RC 1 (30 Sep 2006) 14

GJXDM Tools

Justice Information Exchange Model (JIEM) Modeling Tool • JIEM is a Web application that facilitates analysis of business context and content (payload) requirements for justice information exchanges. • The analysis is represented in a site database. 16

JIEM Exchanges and Dimensions A site database defines exchanges in 5 dimensions: process, event, agency, condition, document (information). 17

JIEM Reference Model • There is a special site database called the JIEM Reference Model, which defines exchanges common to most jurisdictions. • A site database can be created from scratch or copied from another site database, or the Site Database Builder can be used to create one from the JIEM Reference Model. Using the Site Database Builder typically reduces the effort of site database creation by about 75%. 18

JIEM Data Elements A site database includes data elements, which can be simple or may be composed of other data elements. 19

JIEM Data Element Usage Documents contain data elements and define their typical usage. 20

JIEM Reports • A variety of textual and graphic reports can be generated. • Reports generally relate to content of the database or usage of data (comprehension or conceptualization of the data, data admin, etc. ) 21

How JIEM and SSGT Interoperate SSGT Mapping Tool JIEM Schema Subset Generation Tool (SSGT) . XMI Argo. UML Data exchange requirements (local vocabulary) output in XMI (i. e. UML-XML) 22

SSGT Mapping Tool Enables user that has built an AS-IS or TO-BE model with JIEM to map local data components directly to GJXDM data components through the SSGT. Schema Subset Generation Tool (SSGT) 23

SSGT Mapping Tool – Search / Discover • User local data components are displayed (left) with convenient access to the SSGT (right). • User can search with SSGT for GJXDM data components that match local data components. Schema Subset Generation Tool (SSGT) 24

SSGT Mapping Tool – Select / Build User maps by – selecting matching GJXDM data components (check box). – confirming (clicking) the match to build the map. Schema Subset Generation Tool (SSGT) 25

SSGT Mapping Tool – Generate Subset and Report When all local data components have been reviewed and mapped or not, the Mapping Tool can generate: 1. Subset for mapped elements (also contains the wantlist). 2. Report detailing what mapped and what did not. Schema Subset Generation Tool (SSGT) 26

How JIEM and SSGT interoperate SSGT Mapping Tool JIEM Schema Subset Generation Tool (SSGT) Mapping Report Wantlist SSGT. XMI Argo. UML 27

Schema Subset Generation Tool (SSGT) • SSGT is a Web application that allows a user to browse the GJXDM and create GJXDM schema subsets and their wantlists. • Supports basic and advanced search for GJXDM components. 28

SSGT – Expand a Type or Property • A complex type can be expanded to see its properties. • A property can be expanded to see its type’s properties. 29

SSGT – Types User clicks a type to see its: – – – definition base types derived types properties that it is the type of 30

SSGT – Properties User clicks a property to see its: – definition – types in which it’s contained – its types. 31

SSGT – Select Properties and Types Properties and types can be selected for inclusion in the GJXDM schema subset. 32

SSGT – Generate Subset. zip Generating a GJXDM schema subset, creates a. zip file containing both the subset and its corresponding wantlist. 33

SSGT – Save and Load Wantlists can be saved and loaded so that work can be suspended and resumed. 34

How JIEM and SSGT interoperate SSGT Mapping Tool JIEM Schema Subset Generation Tool (SSGT) Mapping Report Wantlist SSGT. XMI Argo. UML Subset. zip 35

Moving Forward • Export other dimensions beyond information dimension in standard open specification (e. g. BPMN, etc. ). • Evaluate applicability to other domains and NIEM. • Capture additional metadata (security, privacy, etc. ) to support the implementation of an SOA. • Support generation of extension and constraint schemas. • Support sharing/reuse of mappings, wantlists, and schemas across JIEM exchanges and site DBs. • Support sharing/reuse with other non-JIEM users through standard import/export specifications (the NIEM IEPD specification is well suited for this). 36

GJXDM Spreadsheet 37

GJXDM Spreadsheet 38

GJXDM IEPD Clearinghouse 39

GJXDM IEPD Clearinghouse 40

GJXDM Tool Summary • Justice Information Exchange Model (JIEM) Tool: – Define/model exchange context and content requirements. – Generate XMI (UML-XML) representation of local data exchange requirements. • GJXDM Spreadsheet: – Off-line textual representation of GJXDM (the model and dictionary). – Search, navigate, browse, discover GJXDM data components. • Schema Subset Generation Tool (SSGT): – – – On-line textual representation of GJXDM (the model and dictionary). Search, navigate, browse, discover GJXDM data components. Select and map GJXDM data components to local components (Mapping Tool). Generate mapping report spreadsheet (Mapping Tool). Select data components and generate schema subset and associated Wantlist. • Manually produce other artifacts required for an IEPD. – Examples: exchange, extension, and constraint schemas, and example instances. • IEPD Clearinghouse (CH): – Register and share a GJXDM IEPD. 41

Available Tools: GJXDM 0 Scenarios, BRM taxonomies JIEM Spreadsheet 1 Info exchg requirements IEPD CH SSGT Map 5 Publish + implement IEPD Life Cycle 4 Assemble to IEPD spec SSGT 2 Map + model 3 Build + validate SSGT 42

NIEM Tools

Component Mapping Template (CMT) A spreadsheet for mapping local data elements to NIEM elements. • • Developed to expand GJXDM to NIEM by identifying data requirement gaps. Developed as submission format for NIEM pilots. Reflects experience gained from working with key GJXDM customers. Structures the process of using NIEM for exchange content requirements. Identifies new data components for addition to NIEM. Structured format allows for automated processing. Structured method to capture: – Local data requirements (vocabulary) and map them to existing NIEM data components • exact matches • partial mapping details • no mapping – Information that can provide a metric for modeling refinement and improvement – Information that can be used for keywords, search hints, mapping assistance, etc. 44

CMT – General Tab General information on exchange(s) being mapped (business context) 45

CMT – Template Tab Information on how local components are mapped to NIEM 46

CMT – Codes Tab Information on how codes are mapped to NIEM 47

NIEM Spreadsheet 48

NIEM Tools for Browse, Navigate, Search, Discover, Generate/Build Schema Subset Generation Tool Graphical Browser IEPD Tool 49

NIEM Graphical Browser Java-based GUI to browse, navigate, and generally familiarize with NIEM structure. 3 different views 8 different starting locations 50

NIEM Graphical Browser – Starting Points and Views • User may start from one of 8 different locations in NIEM (others can be added). • Select from 3 different views of the NIEM: (1) Data model, (2) Inheritance, (3) Associations • The 3 views always correlated to the same location in the model. Data Model (types + properties) Associations Inheritance 51

NIEM Graphical Browser – Entities and Relationships • Model is displayed as a set of nodes and edges corresponding to entities and fundamental data modeling relationships between them: – parent_of (IS-A) – contains (HAS-A) – type_of • Double-clicking an entity (node) moves focus to that entity (by centering and highlighting) and displays all entities and relationships to/from it. 52

NIEM Graphical Browser – Relationships (edges) are directional; to expose a relationship label, hover the cursor over its corresponding edge. 53

NIEM Graphical Browser – Screen Management When an entity has >10 relationships, only the first 10 entities are exposed; next 10 can be seen by clicking the “next 10” button (lower right). 54

NIEM Graphical Browser – Focus The entity of focus can be dragged around to expose entities that may be outside the window. 55

NIEM Schema Subset Generation Tool (SSGT) • A repository of all NIEM data components with their associated metadata linked as a consistent, cohesive data model • Functions: search, find, and select components for, build and generate a subset schema for an IEPD. • Linkage of data components is key to the tool’s ability to provide model navigation and efficient generation of schema subsets. 56

NIEM SSGT – Search Allows both simple and advanced search for data components using various means and criteria including: – Type of component (property, type, facet, external, or association) – With string, substrings, wildcard, exact/partial, etc. – Within component name or definition or both 57

NIEM SSGT – Select / Add • User finds and selects components required by checking boxes to build a NIEM subset schema. • A property component can be added to a subset – as a content bearing element or as a reference to content bearing element. – as an element within its Subject type or as an independent element. 58

NIEM SSGT – Dependencies • SSGT adds all other components that the user requirements depend on. • User may delete components from the subset only if he added them. • Components that user requirements depend on can only be deleted by SSGT (when corresponding user requirements have been deleted). 59

NIEM SSGT – Subset Schema When user is done adding components, SSGT can automatically generate a complete NIEM NDR conformant subset schema for a given version of NIEM. Subset. zip Wantlist 60

NIEM SSGT – Wantlist • A Wantlist is an open spec XML file representing user requirements (without dependencies) and the state of an SSGT session. • A Wantlist can be saved and reloaded later to resume work or to reconstitute or modify a subset schema. 61

NIEM IEPD Tool (IEPDT) • Enables user to easily assemble, document, generate, and share complete or partial IEPD per NIEM IEPD specification. • User registers for account and corresponding work space. 62

NIEM IEPDT – Entering an IEPD User creates an IEPD through a 5 step process: 1. Upload artifact files (optional) 2. Enter basic IEPD metadata. 3. Enter additional IEPD metadata. 4. Enter authoritative source metadata. 5. Verify all information and create the IEPD. 63

NIEM IEPDT – Validation and Review • Validates package completeness against NIEM IEPD content specification and reports results (does not have to pass validation to share). • Provides a review of metadata and artifact files. 64

NIEM IEPDT – Auto Format and Assembly • • Formats metadata file per NIEM IEPD spec (metadata. xml). Formats catalog file of artifacts per NIEM IEPD spec (catalog. xhtml). Assembles IEPD as compressed archive (filename. zip). Installs IEPD in user work space and provides option to share. 65

IEPD Catalog. html

xhtml IEPD Catalog. html (source view)

NIEM IEPDT – Download / Upload • Download IEPD as single, compressed, self-contained archive (<filename>. zip). • Upload IEPD archive file (created manually or by IEPDT per IEPD spec). • IEPDs created manually or by any other tool can be uploaded by IEPDT. 74

NIEM Tool Summary • Component Mapping Template (CMT): – Map off-line data requirements for content submissions to NIEM. – Capture information – metrics, business context, POC data, mapping statistics, etc. • Graphical Browser – Visually navigate and familiarize with NIEM. • Schema Subset Generation Tool (SSGT): – Repository of NIEM data components and metadata. – Textually navigate, search, discover NIEM data components. – Build and generate subset schemas for NIEM IEPDs. • Information Exchange Package Documentation (IEPD) Tool: – User work space – assemble, document, download, upload IEPDs per NIEM IEPD specification. – Share complete IEPDs or their artifacts. 75

Available Tools: NIEM 0 Scenarios, BRM taxonomies Spreadsheet 1 Info exchg requirements 5 Publish + implement IEPDT IEPD Life Cycle 2 Map + model SSGT CMT IEPDT 4 Assemble to IEPD spec Graph Brwsr 3 Build + validate SSGT 76

Available Tools: NIEM + GJXDM 0 Scenarios, BRM taxonomies Spreadsheet JIEM Spreadsheet 1 Info exchg requirements IEPD CH SSGT Map 5 Publish + implement IEPDT IEPD Life Cycle SSGT 2 Map + model SSGT CMT IEPDT NIEM + 4 Assemble to IEPD spec GJXDM Graph Brwsr 3 Build + validate SSGT 77

NIEM 1. 0 RC 1 (30 Sep 2006) Refinements and Improvements • URI for each NIEM data component. – A universally unique identifier; can be a URL that resolves to a documentation page defining semantics, structure, and relationships with other data components. • Selective Sharing of IEPD artifacts and metadata. – IEPD Tool will allow user to selectively share parts of an IEPD (rather than simply the entire IEPD or nothing). • Support the building of a new IEPD from an existing IEPD. – Capability to build IEPDs directly by selecting components to be used within the IEPD, generate, and optionally register/share. • Improved integration between IEPD Tool, Graphic Browser, and SSGT. – Intent is to remove extra manual steps between tools (e. g. , download SSGT subset and upload to IEPD Tool). • Version support for IEPDs. – For bug fixes and changes; will allow for multiple versions of the base IEPD to exist to avoid possible conflicts. Track version history and change log in IEPD metadata. • Basic reporting feature. – usage statistics and management metrics, for example: components used most often in IEPDs, common search terms, commonly selected results, etc. • Export NIEM as a MS Access database 78

Discussion & Questions