Active XML A datacentric perspective on Web services

Active XML: A data-centric perspective on Web services 1 Omar Benjelloun INRIA Futurs With: Serge Abiteboul, Tova Milo, and many others. Omar Benjelloun – Active XML April 30 th, 2004

2 Active XML - Outline Introduction Active XML • • Active XML documents Active XML services Novel issues • • Exchanging Active XML data Querying Active XML data Active XML Peers • • • The peer as a client The peer as a server Theoretical foundations Applications Conclusion Omar Benjelloun – Active XML

Introduction Omar Benjelloun – Active XML

4 Distributed data management in P 2 P Information is everywhere XML Web service XML services XML Internet services XML XML services Omar Benjelloun – Active XML Web service Data warehouses Databases Web sites PC, PDA, cell phones, home appliances, cars…

5 The golden triangle of distributed data management XML a standard for data representation & exchange • • • Extensible Markup Language Labeled ordered trees Types: XML Schema / tree automata Query languages • XPath, XQuery Web services standards for distributed computing • • • SOAP, WSDL, UDDI Activation of methods on remote servers Many burgeoning standard proposals (Choreography, Qo. S, user interface, etc. ) Omar Benjelloun – Active XML SOAP WSDL XQuery XPath

6 What is Active XML (AXML)? AXML is a declarative language for distributed information management and an infrastructure to support this language, in a peer-to-peer framework. Omar Benjelloun – Active XML

Active XML Omar Benjelloun – Active XML

8 Active XML documents with embedded calls to Web services Intensional • • Some of the data is given explicitly Some is given intensionally (i. e. the means to acquire data when needed are given) Dynamic • • If the external sources change, the same document will provide different information Reaction to world changes Omar Benjelloun – Active XML

9 Not a new idea in databases, nor on the Web Mixing calls to data is an old idea • • Procedural attributes in relational systems Basis of Object-oriented Databases In Web programming • Sun’s JSP, PHP+My. SQL Calls to Web services inside documents • Macromedia FLEX, Apache Jelly, Microsoft XAML What is new is the exploitation of the idea… Omar Benjelloun – Active XML

10 Web services in brief A number of standards • XML • SOAP: Exchange of messages between applications • WSDL: Description of service interfaces (e. g. input/output types) • UDDI: Advertisement and discovery of services • … other proposed standards (choreography, security, etc. ) For us: means to provide, invoke and describe remote functions with XML input/output. They make AXML documents universally understandable. Omar Benjelloun – Active XML

11 A sample AXML document <? xml version=“ 1. 0” ? > <newspaper> <title>Le Monde</title> <date>06/10/2003</date> <call svc=“Yahoo. Get. Temp”> <city>Paris</city> </call> <call svc=“Time. Out. Get. Events”> exhibits </call> </newspaper> newspaper Get. Events title date Get. Temp “Exhibits” city “ 06/10/2003” “Le Monde” “Paris” AXML documents may contain calls: • • Omar Benjelloun – Active XML to any existing Web services (e-bay. net, google. com…) to any AXML Web services (to be defined)

12 Materialization <? xml version=“ 1. 0” ? > <newspaper> <title>Le Monde</title> <date>06/10/2003</date> <temp>16°C</temp> <call svc=“Yahoo. Get. Temp”> <city>Paris</city> </call> <call svc=“Time. Out. Get. Events”> exhibits </call> </newspaper> newspaper Get. Events temp Get. Temp date “Exhibits” city “ 16°C” “ 06/10/2003” “Paris” “Le Monde” title SOAP call Y! We will see later that: • • Replacing the call by its result is not the only option Calls are not necessarily RPC-style synchronous invocations Omar Benjelloun – Active XML

13 AXML Web services Parameters: AXML data Result: AXML data Great flexibility Distribute computations: by sending as parameters data containing service calls, one can delegate some work to other peers. Partial computations: by returning data containing service calls, one can give to the receiver the control of these calls. Omar Benjelloun – Active XML

14 Calling an AXML service <? xml version=“ 1. 0” ? > <newspaper> <title>Le Monde</title> <date>06/10/2003</date> <temp>16°C</temp> <exhibits> <call svc=“Yahoo. Get. Exhibits”> <call svc=“Time. Out. Get. Events”> <city>Paris</city> exhibits </call> </exhibits> </newspaper> newspaper title temp date “ 06/10/2003” “Exhibits” Get. Exhibits “ 16°C” “Le Monde” Termination is an issue Omar Benjelloun – Active XML City “Paris” SOAP call (still…) T! Materialization is a recursive process Get. Events exhibits

15 Organization Novel issues raised by the AXML language • • Exchange of AXML data Querying AXML data Supporting infrastructure • AXML peers: – Management of persistent AXML data – Declarative AXML services Applications Omar Benjelloun – Active XML

Novel issues Omar Benjelloun – Active XML

17 Active XML - Outline Introduction Active XML • • Active XML documents Active XML services Novel issues • • Exchanging Active XML data (SIGMOD 2003) Querying Active XML data Active XML Peers • • • The peer as a client The peer as a server Theoretical foundations Applications Conclusion Omar Benjelloun – Active XML

18 To call or not to call ? newspaper Get. Events temp Get. Temp “Exhibits” city “ 06/10/2003” “ 16°C” “Le Monde” “Paris” title Y! q Materialization can be performed q by the sender, before sending a document… q or by the receiver, after receiving it. Omar Benjelloun – Active XML date

19 Why control the materialization of calls? For added functionality, e. g. • Intensional data allows to get up-to-date information. For security reasons or capabilities, e. g. • • I don’t trust this Web service/domain, I don’t have the right credentials to invoke it, It costs money, Maybe the receiver doesn’t know Active XML! For performance reasons, e. g. • A proxy can invoke all the services on behalf of a PDA. … and many more reasons you can think of! Omar Benjelloun – Active XML

20 How to control it? Using types We extend XML Schema, with intensional types: XMLSchemaint Sender Capabilities ACL Cost. . . g q f f g q r . . . q. . . g g q g f r q g f g . . . r. . . g data exchange Schema Receiver Capabilities ACL Cost. . Static analysis algorithms use signatures of services: WSDLint Omar Benjelloun – Active XML

21 The extended schema language To simplify, we use here a DTD-like syntax Data: newspaper title = data date = data temp = data city = data = title. date. (Get. Temp|temp). (Get. Events|exhibit*) newspaper Get. Events title exhibit = title. (Get. Date|date) Functions: Get. Temp(city) date Get. Temp “Exhibits” city “ 06/10/2003” “Le Monde” “Paris” -> temp Get. Events(data) -> (exhibit|performance)* Get. Date(title) -> date Rewriting: replace call(s) by an arbitrary output of the service. Omar Benjelloun – Active XML

22 Rewritings The Goal: Given • an intensional document d • a schema s, Can we rewrite d so that it matches s? Safe rewriting: one that for sure leads to s (we know without making any call). Possible rewriting: one that may lead to s (depending on the answers of services). Omar Benjelloun – Active XML

23 Difficulties Infinite search space • • Vertical Horizontal Main problem • • The result of a Web service call is unknown, We just know a signature (input/output types) We want a very efficient solution. Foundations of the problem • • String & tree automata, with existential and universal transitions. Omar Benjelloun – Active XML

24 Results The general problem is undecidable [MSS 03] Restrictions on the considered rewritings • • Left-to-right: No “going back and forth” K-depth: bound on the nesting of function calls (Search space still infinite but finitely representable) Under these restrictions • • • We have algorithms to find safe/possible rewritings. They are PTIME (for deterministic schemas). We can also do it between schemas. Implementation • demo at VLDB 2003 (customizable news syndication) Omar Benjelloun – Active XML

25 Safe rewriting algorithm Sketch • • • Deal with function parameters first, Top-down traversal of the tree, For each data node: – rewrite its children (viewed as a word), – to match the target type (a regular expression) – using regular automata techniques, and smart marking. Omar Benjelloun – Active XML

26 Safe rewriting algorithm (2) Build an FSA that accepts all k-depth rewritings of the initial word. q 0 title q 1 date q 2 q 5 Get. Temp temp q 3 q 6 Get. Events q 4 q 7 exhibit performance Build an FSA that recognizes the complement of the target type. * title p 0 * date p 1 * temp p 2 *Get. Events p 4 * p 3 * exhibit p 5 exhibit Omar Benjelloun – Active XML p 6 *

27 Safe rewriting algorithm (3) Compute the intersection of these languages: performance exhibit q 0, p 0 title q 7, p 6 q 4, p 6 Get. Events q 3, p 6 q 7, p 6 Get. Temp q 1, p 1 date q 2, p 2 q 5, p 2 q 3, p 3 temp exhibit performance Get. Events exhibit performance q 7, p 5 q 7, p 3 q 4, p 4 q 6, p 3 A smart marking determines whether a safe rewriting exists. Then run the word on the marked automaton to find an actual rewriting. Optimization: lazy construction of the automata Omar Benjelloun – Active XML q 4, p 5

28 Active XML - Outline Introduction Active XML • • Active XML documents Active XML services Novel issues • • Exchanging Active XML data Querying Active XML data (SIGMOD 2004) Active XML Peers • • • The peer as a client The peer as a server Theoretical foundations Applications Conclusion Omar Benjelloun – Active XML

29 Querying AXML Data Given a (tree pattern) query: /newspaper[temp > 18°C]/exhibits//exhibit[location=“Le Louvre”] newspaper Materialize the document? Call only the services that may data to the query answer. exhibits Get. Events temp Get. Temp contribute title “Exhibits” get. Date Get. Exhibits city “ 19°C” City “Paris” “Le Monde” “Paris” The problem: Lazy evaluation of service calls To call or not to call, this time when evaluating a query Omar Benjelloun – Active XML

30 Lazy evaluation Difficulties: • • Calls can be found everywhere in the document May appear dynamically (as a result of previous calls) May become (ir)relevant due to previous invocations Need to take signatures of calls into consideration A possible approach: modify the query processor • • • Top-down evaluation Trigger the calls found on the way Not so great: – Computation is blocked – Optimization opportunities are lost Omar Benjelloun – Active XML

31 Our solution Given a query to evaluate: newspaper temp > 18°C exhibits exhibit location “Le Louvre” newspaper Derive a set of “node-focused” queries (NFQ), that find the relevant calls when evaluated on the document. exhibits temp * * * > 18°C Need to be reevaluated, as the document evolves! Omar Benjelloun – Active XML Etc.

32 Optimizations Service calls sequencing • • Analysis of the relationship between calls (through the NFQ’s) Layering, and parallelization inside each layer. Refinement via type analysis • Matching output types of services with data expected of queries “Pushing” queries to capable services Acceleration: • • Via relaxation: – NFQ approximation – Superset of the relevant calls Via a special access structure, similar to a Data. Guide: – Restricted to paths that lead to service calls – Indexes the calls Experimental assessment • 10 x speed-up when combining optimizations Omar Benjelloun – Active XML

Active XML peers Omar Benjelloun – Active XML

34 Distributed data management in P 2 P XML AXML services Web service XML AXML services AXML Web AXML services XML XML AXML services Omar Benjelloun – Active XML AXML Web service

35 What do we need from an AXML system ? Persistent, manageable, dynamic AXML data. Easy ways to define services Control of the exchanged data (parameters & results of service calls) • • • Repository: manages persistent AXML data Client: uses (AXML) Web services Server: provides AXML services Omar Benjelloun – Active XML soap Peer-to-peer architecture, where each AXML peer: AXML peer

36 Global architecture AXML peer S 2 AXML peer S 1 Query engine query SOAP AXML engine AXML SOAP wrapper read update AXML store AXML SOAP service descriptions XML Omar Benjelloun – Active XML AXML peer S 3 SOAP service SOAP client

37 Implementation SUN’s Java SDK 1. 4 (includes XML parser, XPath processor, XSLT engine) Apache Tomcat 4. 1 servlet engine Apache Axis SOAP toolkit 1. 1 X-OQL query processor, persistent DOM repository JSP-based Web user interface, using JSTL 1. 0 standard tag library Also, a lightweight implementation for PDA/phone (J 2 ME, CLDC profile), used for [ABB 03 demo]. Omar Benjelloun – Active XML

38 Active XML - Outline Introduction Active XML • • Active XML documents Active XML services New issues • • Exchanging Active XML data Querying Active XML data Active XML Peers • • • The peer as a client The peer as a server Theoretical foundations Applications • • • P 2 P auctions News syndication Other applications Conclusion Omar Benjelloun – Active XML

39 Managing persistent AXML data “Our newspaper should have its temperature information refreshed daily. New exhibits should be fetched every week and archived for 6 months” Service call results enrich the document (calls can be kept for possible future reuse) Main issues: • • When to activate a service call? What to do with its result? Omar Benjelloun – Active XML

40 When to activate a service call? Explicit pull mode • • Daily, weekly, or after some event: e. g. , when another call occurs This aspect of the problem is related to active databases Implicit pull mode • • Detect which intensional information (the service calls) may contribute to the answer of a query (lazy evaluation) This aspect of the problem is related to deductive databases Push mode • • Based on a query subscription; the service provider pushes information to the client (E. g. , for synchronization purposes) This is related to stream and subscription queries Omar Benjelloun – Active XML

41 Managing service call results How long does the returned data remain valid? • • • Just long enough to answer a query: Mediation 1 day, 1 week, … or unbounded: Caching / Warehousing Various portions of the document may follow different policies: Hybrid For repeated service call invocations: merge policy • • append, replace, Fusion (using XML Schema-like keys), Specific merge policies can be provided as Web services Omar Benjelloun – Active XML

Example: AXML document with control attributes <? xml version=“ 1. 0” ? > <newspaper> <title>Le Monde</title> <date>06/10/2003</date> <call svc=“Yahoo. Get. Temp” mode=“lazy” valid=“ 1 day” merge=“replace” > <city>Paris</city> </call> <call svc=“Time. Out. Get. Events” mode=“every Monday morning” valid=“ 6 months” merge=“append”> exhibits </call> </newspaper> Omar Benjelloun – Active XML 42

43 Active XML - Outline Introduction Active XML • • Active XML documents Active XML services Novel issues • • Exchanging Active XML data Querying Active XML data Active XML Peers • • • The peer as a client The peer as a server Theoretical foundations Applications Conclusion Omar Benjelloun – Active XML

44 Declarative AXML services Services can be defined by queries or updates over the AXML documents of the repository (XQuery, XPath, Xupdate) let service Get. Exhibits. By. Location($loc) be for $a in document(“newspaper. xml")/newspaper/exhibits, $b in $a//exhibit where $b@name=$loc return <exhibits> {$b} </exhibits> Which (lazy) service calls may contribute to the answer? Omar Benjelloun – Active XML

45 Other means to define services Other programming languages: • • XSLT transformations (through Apache Xalan) Java classes (through Axis) Composition of existing services: • BPEL 4 WS (through IBM’s BPEL 4 J implementation) Omar Benjelloun – Active XML

46 Active XML - Outline Introduction Active XML • • Active XML documents Active XML services New issues • • Exchanging Active XML data Querying Active XML data Active XML Peers • • • The peer as a client The peer as a server Theoretical foundations (PODS 2004) Applications Conclusion Omar Benjelloun – Active XML

47 Theoretical foundations: Positive AXML Restricted framework • • Data model – set-based (unordered) AXML trees – Call results are accumulated in documents Services – Monotone – Positive: defined by conjunctive fragment of XQuery Results • • Well-defined (possibly infinite) fix-point semantics Termination, lazy evaluation… Connections to: • Regular (infinite) trees, Query-Sub-Query [AM 04], … Omar Benjelloun – Active XML

Applications Omar Benjelloun – Active XML

49 Demos Peer-to-peer auctions • Discovery of new peers/auctions through intensional answers RSS News syndication • (VLDB 2002 demo) (VLDB 2003 demo 1) Customization of services through schemas + news subscriptions Distributed workspaces (VLDB 2003 demo 2) Web warehousing (ECDL 2003 demo) A powerful framework for the fast development of distributed, data-centric applications. Omar Benjelloun – Active XML

50 Other applications E. dot, a dynamic warehouse on food risk management • Use AXML as the platform for the warehouse definition, construction and maintenance Network configuration • Use AXML exchange of information to configure hardware/software components Software distribution • Use AXML to customize distributions and keep your view of the software fresh Decentralized user profile/patient data management • Use AXML to coordinate the integration of data, and privacy enforcement services in a uniform way Omar Benjelloun – Active XML

Conclusion Omar Benjelloun – Active XML

52 AXML documents and services A simple paradigm… …that allows for new, powerful features. • • • Intensional parameters and results: AXML documents can be exchanged Support for continuous services (streams of answers) Control over the exchange of AXML data Issues Control of call activation via typing, Lazy evaluation, Replication and distribution, Security, Mobility, Termination, Implementation, Foundations, … Omar Benjelloun – Active XML

53 Current/Future work Security and privacy (with Bell Labs) Editor/browser plug-in for AXML Mass storage XML DB (with Xyleme Corp. ) P 2 P infrastructure … Omar Benjelloun – Active XML

54 To know more… http: //purl. org/net/axml • • • Implementation becomes open-source Already available for research Will be released publicly very soon. Selected publications • S. Abiteboul, O. Benjelloun, T. Milo: • • Positive Active XML, PODS, 2004. S. Abiteboul, O. Benjelloun, B. Cautis, I. Manolescu, T. Milo, N. Preda: Lazy Query Evaluation for Active XML, SIGMOD, 2004. T. Milo, S. Abiteboul, B. Amann, O. Benjelloun, F. Dang Ngoc: Exchanging Intensional XML Data, SIGMOD, 2003 (full version to appear in TODS). S. Abiteboul, O. Benjelloun, I. Manolescu, T. Milo, R. Weber: Active XML: A Data-Centric Perspective on Web Services (book chapter), In Web Dynamics, Springer, 2004. S. Abiteboul, A. Bonifati, G. Cobena, I. Manolescu, T. Milo: Dynamic XML Documents with Distribution and Replication, SIGMOD, 2003 Omar Benjelloun – Active XML

Merci Omar Benjelloun – Active XML

56 Omar Benjelloun – Active XML

Extra slides Omar Benjelloun – Active XML

58 Asynchronous/Continuous services The client subscribes and then is notified The server decides when to send data • E. g. , promotional offers Change control: • • Management of replication [ABCMM 03] What to do when a change is detected – Send the new state of data – Send the delta between old and new state – Dual of merge policies Omar Benjelloun – Active XML

59 Peer-to-peer auctions (VLDB 2002 demo) Each peer proposes auctions: • Document myauctions. xml with the Each peer knows about peer’s items and their current bids other peers’ auctions: • Services offered: • Document allauctions. xml – get. Local. Auctions(), – status(auction. Id) Each peer bids on auctions: • Document mybids. xml with the • peer’s bids Services offered: – bid(peer, auction. Id, amount) – bid. Up. To(peer, auction. Id, increment, limit) Omar Benjelloun – Active XML • contains calls to other peers that transitively retrieve their known auctions. Service offered : get. All. Auctions() When an auction closes, the winner is notified.

60 News syndication (VLDB 2003 demo) News sources: • Get. Story(id) • Get. News. About(kwd) Aggregators: • Get. News. About(kwd) • …but several versions, more or less intensional Clients: • PC, laptops, PDAs Omar Benjelloun – Active XML

61 Service customization using schemas Customizing the output of services • • • News sources/aggregators provide different versions of Get. News. About with different output schemas The output is automatically transformed into the desired schema Clients can also specify a desired output schema as a parameter Customizing the input of services • • Location-aware continuous services for mobile users The context of the user is given by intensional parameters Distributed logging mechanism • Also customizable through the use of schemas Omar Benjelloun – Active XML

62 Call parameters <temp> <call svc=“Get. Temp@weather. com”><city>“Denver”</city></call> </temp> XML <temp> <call svc=“Get. Temp@weather. com”>. . /city</call> </temp> XPath <temp> <call svc=“Get. Temp@weather. com”> <city> <call svc=“Get. Capital@us. gov”>“colorado”</call> </city> </call> </temp> AXML To call or not to call (before invoking) ? Omar Benjelloun – Active XML