A TableDriven Streaming XML Parsing Methodology for HighPerformance

A Table-Driven Streaming XML Parsing Methodology for High-Performance Web Services Wei Zhang Robert van Engelen 9/20/2006 International Conference on Web Services 2006 1

Outline o XML Performance o Related Work: Schema-Specific XML Parsing (SSP) o Table-Driven Streaming XML Parsing (TDX) o Experiment Results o Conclusion 9/20/2006 ICWS 2006 2

XML Performance o XML messaging is at the heart of Web Services o XML is widely seen as underperforming o Increasingly, XML is being used in processes that demand high-performance o Validation is even worse n n 9/20/2006 Often, validation is typically applied during debugging and testing, and is often disabled in production systems ICWS 2006 3

Why are traditional XML parsers slow? 9/20/2006 ICWS 2006 4

Traditional parsers: performance issues(1) o Three stages of XML processing n n n Well-formedness parsing Validation Application data handling application Validation Parsing XML 9/20/2006 ICWS 2006 5

Traditional parsers: performance issues(2) o Frequent access to schema o Comparison done on String (typically inefficient) o Work duplicated between validator and deserializer o Repeated data format validation and conversion (e. g. string/integer) o Data copying 9/20/2006 ICWS 2006 6

Related Work: Schema. Specific XML Parsing (1) o Idea: n n 9/20/2006 Constructing a parser that is hard-coded to process XML by exploiting schema information Merging well-formedness parsing and validation ICWS 2006 application Parsing Validation XML 7

Related Work: Schema. Specific XML Parsing(2) o Merging parsing and validation by n Constructing PDA [Chiu‘ 03] p No namespace support p Converting from NFA to DFA may result in exponentially growing space requirement n Constructing DFA [van Engelen ‘ 04] p Cannot process cyclic XML schema o g. SOAP toolkit [van Engelen ‘ 04] n Based on recursive-descent parsing n Not suitable for generic XML parsing without application data (de)serialization 9/20/2006 ICWS 2006 8

Table-Driven Streaming XML Parsing Methodology (TDX) An integrated Approach to XML Parsing, validation, deserialization, and even application-specific events for High Performance Web Services Parsing Validation application XML 9/20/2006 ICWS 2006 9

Table-Driven Streaming XML Parsing Methodology (1) o LL(1) Grammar can be generated from schema o XML well-formedness parsing can be verified through grammar productions o XML structure can be verified through grammar productions n e. g. Occurrence, enumeration simple. Type o CDATA value validation can be accomplished by semantic actions o Application-specific events can also be encoded as semantic actions 9/20/2006 ICWS 2006 10

An Illustrating Example(1) Schema (abbreviated syntax): <element name=“example” type=“example_type”/> <complex. Type name=“example_type"> <sequence> <element name=“id” type=“xsd: string”/> <element name=“value” type=“xsd: integer”/> <element name=“state” type=“state_type”/> </sequence> </complex. Type> <simple. Type name=“state_type"> <restriction base=“xsd: string”> <enumeration value=“ON”/> <enumeration value=“OFF”/> </restriction> </simple. Type> 9/20/2006 ICWS 2006 11

An Illustrating Example(1) Grammar: Schema (abbreviated syntax): <element name=“example” type=“ example_type”/> <complex. Type name=“example_type"> <sequence> (1) s -> ‘<example>’ t ‘</example>’ (2) t -> t 1 t 2 t 3 <element name=“id” type=“ xsd: string”/> <element name=“value” type=“ xsd: integer”/> <element name=“state” type=“state_type”/> </sequence> </complex. Type> (3) t 1 -> ‘<id>’ CDATA ‘</id>’ //is. Id. Type(…) (4) t 2 -> ‘<value>’ CDATA ‘</value>’ //is. Value. Type(…) (5) t 3 -> ‘<state>’ v ‘</state>’ <simple. Type name=“state_type"> <restriction base=“xsd: string”> <enumeration value=“ON”/> (6) v -> ‘ON’ EVENT //do. State. ON() <enumeration value=“OFF”/> </restriction> </simple. Type> 9/20/2006 (7) v -> ‘OFF’ ICWS 2006 12

An Illustrating Example(2) s Input: <example> <id> id_123 </id> <value> 456 </value> <state> ON </state> </example> ‘<example>’ t 1 t ‘</example>’ t 2 t 3 ‘<id>’ CDATA ‘</id>’ ‘<value>’ CDATA ‘</value>’ invoke is. Id. Type(…) invoke is. Value. Type(…) ‘<state>’ v ‘</state>’ ‘ON’ EVENT invoke do. State. On() Top-down parsing tree 9/20/2006 ICWS 2006 13

TDX Architecture(1) 9/20/2006 ICWS 2006 14

TDX Architecture(2) 9/20/2006 ICWS 2006 15

TDX Architecture(3) 9/20/2006 ICWS 2006 16

TDX Architecture(4) 9/20/2006 ICWS 2006 17

TDX Modularity o TDX parsing engine is schema-independent o Hot swap modules for SSP tokens 9/20/2006 Type-checking actions LL(1) grammar productions ICWS 2006 LL(1) parse table 18

TDX Construction Toolkit(1) o Two Code generators: WSDL 2 TDX and LL 2 Table o Given a schema or WSDL specification, the toolkit automatically generates tables for parsing engine 9/20/2006 ICWS 2006 19

TDX Construction Toolkit(2) o Why two generators? n n 9/20/2006 Application-specific events can not be generated automatically Allows insertion of application specific events ICWS 2006 20

TDX Scanner/Tokenizer o TDX scanner is also runtime tokenizer o Why tokenization? n Comparison done on tokens (more efficient) o Defined by component tags n Element names, attribute names n Classified as starting tags, ending tags n Enumeration values n CDATA, EVENT o Normarlized namespace binding n <namespace, tag_name> 9/20/2006 ICWS 2006 21

Scanner/Tokenizer example <book xmlns =“x. org" xmlns: y=“y. org"> <title> XML Bible </title> <author> <name> Bob </name> <y: title> professor </y: title> </author> </book> Part of tokens: 9/20/2006 <title> </title> <author> </author> x. org b. TITLE e. TITLE b. AUTHOR e. AUTHOR y. org b. TITLE 1 e. TITLE 1 ICWS 2006 22

Mapping Rules o Define mapping from XML schema to LL(1) grammars o Preserves structural constrains o Many types of validation constraints are incorporated in resulting grammar productions n e. g. , occurrence constraints o Some type-checking constraints are incorporated as grammar productions n 9/20/2006 e. g. , enumeration simple. Type ICWS 2006 23

Sample Mapping Rules 9/20/2006 ICWS 2006 24

Mapping Example <complex. Type name=“example”> <sequence> <element name=“id” type=“id_type” min. Occurs=“ 0”/> <element name=“value” type=“value_type” min. Occurs=“ 0” max. Occurs=“unbounded”/> </sequence> </complex. Type> 9/20/2006 ICWS 2006 example -> s 1 s 2 S 1 -> b. ID id_type e. ID S 1 -> S 2 -> b. VALUE value_type e. VALUE s 2 S 2 -> 25

TDX Table Generation Example Grammar: (1) s -> b. E t e. E (2) t -> t 1 t 2 t 3 (3) t 1 -> b. I CD e. I //is. Id. Type(…) (4) t 2 -> b. V CD e. V //is. Value. Type(…) (5) t 3 -> b. S v e. S (6) v -> c. ON EV //do. State. On() (7) v -> c. OFF 9/20/2006 ICWS 2006 26

TDX Table Generation Example(2) b. E s e. E b. I b. V e. V b. S e. S c. ON c. OFF v->c. ON v->OFF $ s->b. E t e. E t t->t 1 t 2 t 3 t 1 ->b. I CD e. I t 2 ->b. V CD e. V t 3 ->b. S v e. S v $ ACC LL(1) Parse Table 9/20/2006 ICWS 2006 27

TDX Parsing Engine Exmple Parsing Table: b. E s e. E b. I b. V e. V b. S e. S c. ON c. OFF v->c. ON v->OFF $ s->b. E t e. E t t->t 1 t 2 t 3 t 1 ->b. I CD e. I t 2 ->b. V CD e. V t 3 T 3 ->b. S v e. S v $ Input: <example> <id> id_123 </id> <value> 456 </value> <state> ON </state> </example> ACC b. E s $ TDX Parsing Engine stack 9/20/2006 ICWS 2006 28

Parsing Example (cont’d) Parsing Table: b. E s e. E b. I b. V e. V b. S e. S c. ON c. OFF v->c. ON v->OFF $ s->b. E t e. E t t->t 1 t 2 t 3 t 1 ->b. I CD e. I t 2 ->b. V CD e. V t 3 T 3 ->b. S v e. S v $ Input: <example> <id> id_123 </id> <value> 456 </value> <state> ON </state> </example> ACC b. E t e. E $ TDX Parsing Engine stack 9/20/2006 ICWS 2006 29

Parsing Example (cont’d) Parsing Table: b. E s e. E b. I b. V e. V b. S e. S c. ON c. OFF v->c. ON v->OFF $ s->b. E t e. E t t->t 1 t 2 t 3 t 1 ->b. I CD e. I t 2 ->b. V CD e. V t 3 T 3 ->b. S v e. S v $ Input: <example> <id> id_123 </id> <value> 456 </value> <state> ON </state> </example> ACC b. E b. I t e. E $ TDX Parsing Engine stack 9/20/2006 ICWS 2006 30

Parsing Example (cont’d) Parsing Table: b. E s e. E b. I b. V e. V b. S e. S c. ON c. OFF v->c. ON v->OFF $ s->b. E t e. E t t->t 1 t 2 t 3 t 1 ->b. I CD e. I t 2 ->b. V CD e. V t 3 T 3 ->b. S v e. S v $ ACC b. E b. I Input: <example> <id> id_123 </id> <value> 456 </value> <state> ON </state> </example> t 1 t 2 t 3 e. E $ TDX Parsing Engine stack 9/20/2006 ICWS 2006 31

Parsing Example (cont’d) Parsing Table: b. E s e. E b. I b. V e. V b. S e. S c. ON c. OFF v->c. ON VT v->OFF $ s->b. E t e. E t t->t 1 t 2 t 3 t 1 ->b. I CD e. I t 2 ->b. V CD e. V t 3 T 3 ->b. S v e. S v $ ACC b. E b. I Input: <example> <id> id_123 </id> <value> 456 </value> <state> ON </state> </example> b. I CD e. I t 2 t 3 e. E $ TDX Parsing Engine stack 9/20/2006 ICWS 2006 32

Parsing Example (cont’d) Parsing Table: b. E s e. E b. I b. V e. V b. S e. S c. ON c. OFF v->c. ON VT v->OFF $ s->b. E t e. E t t->t 1 t 2 t 3 t 1 ->b. I CD e. I t 2 ->b. V CD e. V t 3 T 3 ->b. S v e. S v $ ACC b. E b. I CD Input: <example> <id> id_123 </id> <value> 456 </value> <state> ON </state> </example> CD e. I t 2 t 3 e. E $ stack 9/20/2006 ICWS 2006 TDX Parsing Engine invoke is. Id. Type() 33

Parsing Example (cont’d) Parsing Table: b. E s e. E b. I b. V e. V b. S e. S c. ON c. OFF v->c. ON VT v->OFF $ s->b. E t e. E t t->t 1 t 2 t 3 t 1 ->b. I CD e. I t 2 ->b. V CD e. V t 3 T 3 ->b. S v e. S v $ ACC b. E b. I CD … $ Input: <example> <id> id_123 </id> <value> 456 </value> <state> ON </state> </example> $ TDX Parsing Engine stack 9/20/2006 ICWS 2006 34

Experiment Results o Test environment n 2. 4 GHz P 4, 512 MB RAM, Red Hat Linux 3. 2. 2 -5, GNU Compiler g++. 3. 2. 3 with option – 02 n Memory-resident XML message n Measures with elapsed real time using timeofday() for 100 runs o Compared with n DFA-based Parser n g. SOAP 2. 7 n e. Xpat 1. 2 n Xerces 2. 7. 0 9/20/2006 ICWS 2006 35

Experiment Results(cont’d) XML Schema for echo. String (abbreviated syntax): <schema> <element name="echo. String"> <complex. Type> <sequence> <element name="input" type="xsd: string“ max. Occurs=“unbounded”/> </sequence> </complex. Type> </element> </schema> 9/20/2006 ICWS 2006 36

Parsing Performance (1) XML document size: 1024 B 9/20/2006 ICWS 2006 37

Parsing Performance (2) 9/20/2006 ICWS 2006 38

Conclusions o TDX is fast n n n n 9/20/2006 Integrated approach across layers Avoid schema access at runtime Comparison done on tokens Avoid data copying Avoid format conversions Minimized function calls Optimization based on schema structure ICWS 2006 39

Conclusions (cont’d) o XML can be parsed, validated, and deserialized efficiently for high-performance Web services using table-driven methodology n Can be up to several times faster than industry-strength high-performance validating XML parsers. o Table-Driven methodology can offer high-level of modularity, and o Provides a mechanism integrating applicationspecific events, such as SOAP deserializers 9/20/2006 ICWS 2006 40

Thank You 9/20/2006 ICWS 2006 41