Beyond the simple Pipeline managing processes over time

Beyond the simple Pipeline managing processes over time XML Prague , 17/06/2007 Geert Bormans, independent consultant geert@gbormans. telenet. be

Introduction n Who am I? q Background n 10 years team-lead and researcher q n 5 years VP professional services q q researchgroup document architectures Catholic University Leuven British/Canadian XML software vendor Independent Consultant n Mainly projects that surface the intrinsic value of (yet existing) content q q q Migrations, ‘legacy’ conversions, XML transformations Content Management, content reuse e-publishing geert@gbormans. telenet. be realising the value of your content

Introduction n What to expect? n n n n A quick introduction of the problem space Introduction of the project Introduction of the solution A word about pipelines Challenges for building pipeline solutions The solutions Architecture Wider applicability and future plans A quick demo geert@gbormans. telenet. be realising the value of your content

So we all love XML pipelines by now… Ever thought about managing them?

Managing Pipelines - Problem space geert@gbormans. telenet. be realising the value of your content

The project n The customer q q q n A financial institution working for the goverment Publishing yearly statements New legislation requires archiving the statements Current Architecture q q Complex Monolithic Code Growing with yearly changes geert@gbormans. telenet. be realising the value of your content

The project – Current Architecture geert@gbormans. telenet. be realising the value of your content

Example Statement geert@gbormans. telenet. be realising the value of your content

Next years Example Statement ID added to statement geert@gbormans. telenet. be realising the value of your content Publisher company rebrand

Original Request for Proposal geert@gbormans. telenet. be realising the value of your content

Managing XML instead of PDF. . . geert@gbormans. telenet. be realising the value of your content

. . . and version managing the pipeline Each asset in the CMS is linked (through metadata) to a specific version of the pipeline, hence assuring the possibility to reconstruct a statement as it was printed before geert@gbormans. telenet. be realising the value of your content

Pipeline benefits n Break up complexity q q n n n Simpler to develop Easier to debug and maintain Clear assignment of responsibilities Mix technologies easily Reuse components geert@gbormans. telenet. be realising the value of your content

Pipeline solutions - challenges n Execution q q n n Avoiding intermediate (de)-serialisation Allow for mixed technologies q n n Fit components together and Compile Pipeline Execute with high performance in a robust way Different technology per component possible Easy creation of pipeline Manageability geert@gbormans. telenet. be realising the value of your content

Solutions Architecture geert@gbormans. telenet. be realising the value of your content

Pipeline Component Managment System n n Processing steps are URI-addressable resources Steps pulled together at run time to create pipeline Execution by Pipeline Execution Engine Datamodel for simple pipeline geert@gbormans. telenet. be realising the value of your content

Pipeline Execution Engine n Thin layer on top of Net. Kernel q q q Resource oriented computing platform Microkernel that schedules resource requests Dynamic compilation of various scripting languages Strong support for XML processing tasks Pipeline language: DPML geert@gbormans. telenet. be realising the value of your content

DPML Example geert@gbormans. telenet. be realising the value of your content

Meeting the challenges n n n Pipeline execution by Net. Kernel Unnecessary serialisation/deserialisation avoided by notion of aspects and transreptors Net. Kernel supports pipeline components developed in various technologies PCMS serves as abstraction layer between pipeline development and execution PCMS takes care of pipeline version managment geert@gbormans. telenet. be realising the value of your content

Wider Applicability n n Architecture provides version managment of pipeline components Most important factor of complexity of XML processing is variation in time n Applications layered on this architecture will be better protected against damage by changes n A solution worthwhile considering for any complex XML processing application geert@gbormans. telenet. be realising the value of your content

Future Plans n n Non lineair pipelines (forking, looping) Dashboard for monitoring q n Import filters for q q n Timings, breakpoints XProc Stylus Studio pipes . . . geert@gbormans. telenet. be realising the value of your content

Questions and Discussion Thanks for your attention