e Commerce Technology 20 751 Lecture 8 Data

e. Commerce Technology 20 -751 Lecture 8: Data Interchange

Outline • The need for data interchange – Transactions imply data exchange • The XML concept – Separation of • content • presentation • document structure • ASN. 1 for self-describing data formats – Solves a different problem than XML does – Not what the data means but how it is encoded 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Data Exchange in e. Commerce Choose Vendor B 2 C B 2 B Review Catalogs Identify Specifications Define Requirements Send RFP Ship Select Vendor Approve P. O. Fill Out P. O. Review Vendors Prepare Fulfillment Letter of Credit / Terms Process Credit Card Select & Order Set Up Credit Send P. O. to Vendor Check Availability Generate P. O. Receipt Send Invoice Send Order Arrange Shipping Inspections Tariffs / Customs 20 -751 ECOMMERCE TECHNOLOGY Receive Order Post Payment SUMMER 2003 Prepare Invoice Receive Invoice / Prepare Payment Send Check COPYRIGHT © 2003 MICHAEL I. SHAMOS SOURCE: METALSITE

Typical Purchase to Payment Cycle Distributor Manufacturer 1 Purchase Order Quotes Revise Data 7 ? Error Resolution Adjust Payables 8 2 X IF Corrected Invoice Receipt Payment Order Receipt 20 -751 ECOMMERCE TECHNOLOGY Incorrect Order Delay? 6 3 9 5 10 11 IF NOT Corrected w/Deduction Credit Taken Prices Notification of Error 13 SUMMER 2003 4 Adjust Invoice Send Invoice 12 Payment Recv’d Allow Deduct? 14 X Credit Memo Allow or Issue Credit COPYRIGHT © 2003 MICHAEL I. SHAMOS

Basis for Communication “Any meaningful exchange of utterances depends upon the prior existence of an agreed upon set of semantic and syntactic rules” ISO TR 9007: 1987 Information Processing Systems – Concepts and Technology for the Conceptual Schema and the Information Base 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Invoice Example <Unit. Price>6. 05</Unit. Price> SOURCE: PROF. JEROME YEN 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Components of Data Structure Presentation ABC Content Text Pictures SOURCE: SOFTWARE AG 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

How to Make Data Portable • Tell what the data is • Tell how the data is structured • Tell how it should look SO COMPUTERS CAN UNDERSTAND IT • BUT DO THESE SEPARATELY. MIXING IS BAD • • The meaning -- XML The structure -- XML Schema (document type definition) The formatting -- XSL (e. Xtensible style sheet language) Example: XML catalog structure – DTD, XSL

XML at a glance Well Formed Document (obeys XML syntax rules): <Book> <Author>George Soros</Author> <Title>The Crisis of Global Capitalism</Title> <Year>1998</Year> <Publ>Public Affairs</Publ> <Price>26. 00</Price> <ISBN>1 -891620 -27 -4</ISBN> </Book> SOURCE: PROF. JEROME YEN 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

XML Recipe Example <? xml version="1. 0"? > <Recipe> <Name>Apple Pie</Name> <Ingredients> <Ingredient> <Qty unit=pint>1</Qty> <Item>milk</Item> </Ingredient> <Qty unit=each>10</Qty> <Item>apples</Item> </Ingredients> <Instructions> <Step>Peel the apples</Step> <Step>Pour the milk into a 10 -inch saucepan</Step> <!-- And so on. . . --> </Instructions> </Recipe>

Document Is Now Tree-Structured <? xml version="1. 0"? > <Recipe> <Name>Apple Pie</Name> <Ingredients> <Ingredient> <Qty unit=pint>1</Qty> <Item>milk</Item> </Ingredient> <Qty unit=each>10</Qty> <Item>apples</Item> </Ingredients> <Instructions> <Step>Peel the apples</Step> <Step>Pour the milk into a 10 -inch saucepan</Step> <!-- And so on. . . --> </Instructions> </Recipe>

An XML Document is a Tree BOOKLIST BOOK TITLE BOOK YEAR AUTHOR PUBLISHER Each XML element and subelement is a node 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

XML Document Tree XMLDOMDocument XMLDOMNode ID Element Document Attribute CATALOG WIDGET Value WIDGET ID 1 2 NAME Widget 1 PRICE 10 20 -751 ECOMMERCE TECHNOLOGY SOURCE: MICROSOFT NAME PRICE CURRENCY Widget 2 USD SUMMER 2003 20 CURRENCY USD COPYRIGHT © 2003 MICHAEL I. SHAMOS

XML Namespaces • Two different elements with the same name will cause confusion <FLIGHT> <NAME> DELTA </NAME> <NUMBER> 101 </NUMBER> </FLIGHT> 20 -751 ECOMMERCE TECHNOLOGY <TICKET> <NAME> Leo Mark </NAME> <PRICE currency="US Dollar"> 244. 95 </PRICE> </TICKET> SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Sets of Tags Might Conflict • Example: This is a table of fruits This is a physical table They have different sets of subelements 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Namespaces • Example: Tags in the h namespace In the h namespace, table has subelements tr, td In the f namespace, table has subelements name, width, length Tags in the f namespace 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Namespaces External references to namespace definitions 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

XML Sample Document <? xml version=“ 1. 0” encoding=“UTF-8” ? > <catalog, schemalocation = “http: //euro. ecom. cmu. edu/catalog. xsd”> <widget id=“ 1”> <name>Widget 1</name> <price currency=“USD”>10</price> </widget> <widget id=“ 2”>. . . </widget>. . . </catalog> SOURCE: MICROSOFT 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

XML Data Schemas • XML for defining valid XML documents Contents of http: //euro. ecom. cmu. edu/catalog. xsd: <Element. Type name=“Name”> <String /> </Element. Type> <Element. Type name=“Price”> <String /> </Element. Type> <Element. Type name=“Widget”> <Element type=“#Name” min. Occurs=“ 1” /> <Element type=“#Price” min. Occurs=“ 1” /> </Element. Type> SOURCE: MICROSOFT 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

XML Data Transformation ASP / DOM (xmldom. asp) HTML XML Document (transform. xml) HTML XSL Style. Sheet (transform. xsl) ASP / DOM (xmldom. asp) XML Document (adors. xml) 20 -751 ECOMMERCE TECHNOLOGY XSL Stylesheet (render 2. xsl) XSL Stylesheet (render 1. xsl) SUMMER 2003 SOURCE: MICROSOFT COPYRIGHT © 2003 MICHAEL I. SHAMOS

XSD (XML Schema Definition) • XSD describes the structure of data • Defines the allowed tags and attributes • XSD is an XML language that : – – – defines elements that can appear in a document defines attributes that can appear in an element defines hierarchy and number of child elements defines data types for elements and attributes defines default and fixed values for elements and attributes 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Root tag = schema An XSD File <? xml version="1. 0"? > <!-- XML Schema Example for e. Commerce Technology --> Namespace for this file <xs: schema xmlns: xs="http: //www. cmu. edu/XMLschema" target. Namespace="http: //www. cmu. edu" Namespace being defined xmlns="http: //www. cmu. edu" element. Form. Default="qualified"> Local elements must specify a namespace Begin definition of element “textbook” <xs: element name="textbook"> <xs: complex. Type> subelements must occur in sequence <xs: sequence> “textbook” has string element <xs: element name="title" type="xs: string"/> subelements “author” <xs: element name="author" type="xs: string"/> <xs: element name="ISBN" type="xs: string"/> string element “title” date element <xs: element name="publishdate" type="xs: date"/> “publishdate” <xs: element name="price"> string element “ISBN” <xs: simple. Type> element “price” <xs: restriction base="xs: integer"> “price” must be an <xs: min. Exclusive value="0"/> integer > 0 but <xs: max. Inclusive value="100"/> not > 100 </xs: restriction> </xs: simple. Type> </xs: element> </xs: sequence> </xs: complex. Type> </xs: element> </xs: schema> 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

XSL (Extensible Stylesheet Language) • XSL describes presentation of data • Three parts of XSL: – XSLT (language for transforming XML documents) – XPath (language for extracting parts of an XML document) – XSL Formatting Objects (vocabulary formatting XML documents) 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

XSL Extensible Style Language REWRITING RULES FOR MAPPING XML TAG CONTENTS TO HTML LOOK FOR ALL “TITLE” TAGS <xsl> <rule> <target-element type="title"/> <H 1 color="red" font-family="Arial"> <children/> PUT THE CONTENTS OF “TITLE” TAGS INTO RED HEADER FONT ARIAL </H 1> </rule> DO THE SAME FOR THE CHILDREN OF </xsl> “TITLE” TAGS SOURCE: WDVL. COM 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

XML Style Sheet Processing XML Style Sheet Processor HTML Page XSL Style Sheet SOURCE: SOFTWARE AG

XSL as Part of an XML Application <? xml? > <weather-report> <date>September 10, 1999</date> <time>08: 00 </time> <area> <city>Darmstadt</ci ty> <temperature scale=“C”>25</tempera ture> XSL Style Sheet Schema Parser Style Sheet Processor HTML Page Application Document Tree 20 -751 ECOMMERCE TECHNOLOGY SOURCE: SOFTWARE AG SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

XSL Functions • Transform XML into any other language or format • Filter and sort XML data • Extract parts of an XML document • Format XML data based on the data value (like displaying negative numbers in red) • Output XML data to different devices, like screen, paper or voice (Speech. ML) SOURCE: CAMUS 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

XPath Expressions • //order[3]/item[2] – (2 nd “item” tag in 3 rd “order” tag) • //purchase/item[id=508] – (Inside a “purchase” tag, any “item” tag with “id” subelement having value 508) • //payment[payment_header/@date="1999 -0204"][currency="CAD"] – (Any payment in Canadian dollars made on April 4, 1999) SOURCE: CAMUS 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Multiple Data Uses with XML • Data created once can be delivered in different formats Data Capture Data Creation Editing & Update XML Data Management Product Staging Master Repository Product Repository Print Transform Print Production Web Transform Web Server Wireless Transform Wireless Delivery Interchange Transform Data Interchange SOURCE: OASIS 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

External Integration with XML Company A Company B Data Access / Update Internet Data Management Import / Export Transform Messaging Import / Export Transform Data Management System B System A SOURCE: OASIS 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Using XML – SAX and DOM Document DOM = Document Object Model DTD SAX = Simple API for XML Parser start. Document start. Element DOM end. Element Application implements Document. Handler SAX end. Document Tree SOURCE: SOFTWARE AG 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS 1

DOM Methods for Navigation parent. Node previous. Sibling next. Sibling first. Child last. Child child. Nodes (length, item()) get. Elements. By. Tag. Name SOURCE: SOFTWARE AG 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Financial Reporting Example Documents have many customers Creation Quality & Assurance Reporting Enterprise Banks XML Financial Auditor Server Tax Advisor 1 2 3 4 5 Stock Exchanges Government Media, Press & Financial Portals 6 20 -751 ECOMMERCE TECHNOLOGY Consumpti on Distribution Analysts SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS SOURCE: SOFTWARE AG

Ubiquitous XML Architecture SOURCE: PROF. JEROME YEN 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Software AG Entire. X Orchestrator Web Applications CRM ERP Legacy Database Middleware CICS, IMS, 3270, 5250 COBOL Natural VSAM File, Program DB 2, Oracle Sybase Informix SQL Server Adabas MQSeries Socket COM* CORBA* Adapter Web. Sphere Web. Logic Broad. Vision Siebel Vantive Clarify SAP Poeople. Soft JDEdwards Oracle Adapter Transformation Repository Routing Rules * via Entire. X Communicator 20 -751 ECOMMERCE TECHNOLOGY Integration Management SOURCE: SOFTWARE AG SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Industry XML Schemas • Many XML schemas have been defined for specific tasks or marketplaces Open Trading Protocol (OTP) Open Financial Exchange (OFX) VCard Webbroker Legal Working Group Workflow Classified Ads Automotive Industry Web IDL (WIDL) Telecomms Chemical Markup Language SOURCE: MICROSOFT 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Financial XML “Implementations” • (Financial Products Markup Language) • FIXML (FInancial e. Xchange ML) • (FINancial XML) • FSML (Financial Services ML) • • (Straight-Through Processing ML) (SWIFT ML) 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

e. Business XML • eb. XML (e. Business XML) • e. CX (e. Catalog XML) • Tracker XML (for import/export) • t. XML (Transportation XML, for logistics) • • (VISA XML Invoice Specification) (e. Xtensible Business Reporting Language) 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Scientific XML • CML (Chemical ML) • GEML (Gene Expression Markup Language) • Math. ML (Mathematics ML) • Robo. ML (Robotics ML) 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Impact of XML • • • Interchange mechanism between applications Rapidly becoming the database language of the Web XML client/server transactions over http: Permits web data repositories XML properties: – Scalable – Maintainable – Easy to use (spreadsheet style skills) – Interoperable (exchange business components) 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Standard Encodings • Business systems must exchange data in different formats: – Invoices, payment orders, checks, bills of lading, delivery instructions, authentication information • Need standard notation to describe transmitted data in communication protocols • BUT: what do the bits inside the tags mean? – What does 437573746 F 6 D 6572 (hex) mean? – Where do fields begin and end? – How about complex data structures (arrays, etc, )?

Abstract Syntax Notation (ASN) • ASN. 1 is a method of encoding data so that the format can be decoded from the data itself • ASN. 1 is not a programming language • It describes only data structures. No code, no logic.

Abstract Syntax Notation • ASN. 1 has primitive types: BOOLEAN, INTEGER, REAL, ENUMERATED, BIT STRING, IA 5 STRING, . . . • ASN. 1 has – SET (unordered) SEQUENCE (fixed order) of primitive types – CHOICE for selecting alternative types (integer or real) • Can define new types: Month : : = INTEGER (1. . 12) Day : : = INTEGER (1. . 31) Daily-stock-volume : : = SEQUENCE SIZE (31) OF INTEGER

Basic Encoding Rules (BER) • Define how fields described in ASN. 1 should be encoded • Units of BER are data elements • A data element is a triple: { identifier-type, length, value } • Some type codes: BOOLEAN IA 5 STRING INTEGER SEQUENCE SET 01 16 02 10 31 (8 -BIT ASCII) • The string “Customer” would be encoded as 16 08 43 75 73 74 6 F 6 D 65 72 IA 5 STRING LENGTH 8 20 -751 ECOMMERCE TECHNOLOGY HEX “C” HEX “r” HEX “u” SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Basic Encoding Rules • Content field may be primitive (value) or structured (content has subcomponents) 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Basic Encoding Rules BBCard : : = SEQUENCE { name IA 5 String (SIZE (1. . 60)), team IA 5 String (SIZE (1. . 60)), age INTEGER (1. . 100), position IA 5 String (SIZE (1. . 60)), handedness ENUMERATED {left-handed(0), right-handed(1), ambidextrous(2)}, batting-average REAL } “Casey”, “Mudville Nine”, 32, “left field”, ambidextrous, 0. 250 (47 bytes of text) C a s e y M 302 D 1605 43617365 79160 D 4 D 4 E 696 E 65 02012016 0 A 6 C 6566 01020903 80 FE 01 (47 20 -751 ECOMMERCE TECHNOLOGY u d v i 75647669 74206669 bytes in SUMMER 2003 l l e 6 C 6 C 6520 656 C 640 A BER) COPYRIGHT © 2003 MICHAEL I. SHAMOS

Using ASN. 1/BER ASN. 1 DEFINITIONS ASN. 1 COMPILER SOURCE LANGUAGE (JAVA, C) DATA STRUCTURES ENCODER/ DECODER APPLICATION CODE APPLICATION PROGRAM (JAVA, C) COMPILER APPLICATION PROGRAM NOW READS AND WRITES DATA ACCORDING TO BER 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 APPLICATION PROGRAM COPYRIGHT © 2003 MICHAEL I. SHAMOS

ASN. 1 Applications • Telephone billing information – Transferred Account Procedure (TAP 3) – UMTS (3 G phones) • • • X 9 financial services (checks, electronic funds transfer) Air-to-ground aircraft information Electric and gas utilities Automobile diagnostic monitoring systems Radio Frequency Identification (RFID) Biometric IDs (Proposed ANSI Standard X 9. 84) – Common Biometric Exchange File Format CBEFF • Smart cards (ISO 7816 -4) • MORE 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

Standards using ASN. 1 • X. 400 Message Handling System / X. 500 Directory Services – LDAP based Directories / X. 509 Digital Certificates • RSA Public Key Cryptography Standards – Storage and Transmission of Keys / Certificates (PKCS #12) • Secure Electronic Transaction (SET) – Visa / Mastercard • H. 323 / T. 120 Multimedia Communication Standards – Definition and Exchange of Multimedia Terminal Capabilities • Telecommunications Management Network (TMN) – Performance / Fault and Configuration Management Information • Unicode Worldwide Character Standard – Universal 16 -bit Character Set comprising all written languages Simple Network Transfer Protocol (SNMP) • – Management Information Base (MIB) / SNMP PDUs 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

XML and ASN. 1 • Both XML and ASN. 1 represent hierarchical (treestructured) data • Therefore, one can be translated into the other! • IBM ASN. 1/XML translator • We also have XER, the XML encoding rules 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS

The XML ‘Alphabet Soup’ XML Extensible Markup Language Defines XML documents Infoset Information Set Abstract model of XML data; definition of terms DTD Document Type Definition Non-XML schema XSD XML Schema XML-based schema language XDR XML Data Reduced An earlier XML schema CSS Cascading Style Sheets Allows you to specify styles XSL Extensible Stylesheet Language for expressing stylesheets; consists of XSLT and XSL-FO XSLT XSL Transformations Language for transforming XML documents XSL-FO XSL Formatting Objects Language to describe precise layout of text on a page

The XML ‘Alphabet Soup’ XPath XML Path Language A language for addressing parts of an XML document, designed to be used by both XSLT and XPointer Supports addressing into the internal structures of XML documents XPointer XML Pointer Language XLink XML Linking Language Describes links between XML documents XQuery DOM XML Query Language (draft) Document Object Model Flexible mechanism for querying XML data as if it were a database API to read, create and edit XML documents; creates in-memory object model SAX Simple API for XML API to parse XML documents; event-driven Data Island XML data embedded in a HTML page Data Binding Automatic population of HTML elements from XML data

Q&A 20 -751 ECOMMERCE TECHNOLOGY SUMMER 2003 COPYRIGHT © 2003 MICHAEL I. SHAMOS