Development Methodologies for Vocabularies Waterfall vs cyclical big

Development Methodologies for Vocabularies Waterfall vs. cyclical, ‘big bang’ vs. evolutionary—a case study of two projects Jim Gabriel 25 April 2005, OASIS Symposium, New Orleans The Future of XML Vocabularies

Agenda l l l Development methodologies Cyclical case study Waterfall case study Observations Recommendations Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Development Methodologies l Cyclical Evolutionary roll-out l Waterfall “Big Bang” roll-out Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Cyclical l When to use the Cyclical method: o o o l Benefits: o o o l Legacy High expectation of change Virgin territory Incremental roll-out Gradual user acclimatization Can (and should) be model-driven Drawbacks: o o o Needs powerful tools Needs discipline People can become bottlenecks Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Waterfall l When to use the Waterfall method: o o o l Benefits: o o l New systems (clean-sheet development) Low expectation of change Well-understood territory “Big-bang” roll-out Simple development environment infrastructure Drawbacks: o o Oligarchy Change = shock to the system Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Cyclical Case Study Customer supportal

Scope l Vocabulary required for: o o l Library Classroom Technical Support Publishing system Development team: o o o 7 stakeholders 4 departments 2 countries Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Constraints l l l l Information lifecycle spanned multiple versions Publishing system requirements constantly evolving Unpredictable document types assembled from existing components Dynamic, incremental publishing No broken links allowed All information must be single-source Proprietary plug-ins to access content not allowed Authors to be able to directly update content Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Working methods l Investigation: o o l l l l Users Producers Information mapping DTD design Data conversion, data entry, new authoring Evolve design (change DTDs) Rebuild publishing system Synchronize content Repeat (goto Evolve design) Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Was the cyclical method suitable? l l In theory, yes In practice, no o o o We developed 65+ schemas Evolution of schemas is not straightforward Evolution of content is difficult Technology unable to support evolution process Experts become bottlenecks Team quickly de-motivated Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Waterfall Case Study Next generation of vodafone Live!

Scope l l l l l Various other systems currently operating Ongoing development effort around the globe Content from 3 rd parties around the globe Roll-out to more and more Vodafone subsidiaries Support more and more content providers Cater for very different types of content Support deployment in new geographies Manage new types of content without re-engineering core components Many stakeholders, 1 Architect, 1 Schema Designer/Developer Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Constraints l l Similar in many respects to the Cyclical case study Specifically: o o Both mobile-centric and content-centric Address the current and future needs of all the Vodafone operating divisions Incorporate the latest developments in XML standards Provide fine-grained access to varied content Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Working methods l l Investigation Schema design Delivery Implementation Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Was the waterfall method suitable? l Yes, for the initial phase o o o l l Robust, comprehensive design Development straightforward 22 schemas developed Changes may force a different approach in the future Problems, if any, not visible to central group Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Observations Conclusions and lessons learned

Some conclusions l l The waterfall approach is ideal when implementation happens later The cyclical approach to schema development is necessary in any situation where: o o o l l Implementation happens simultaneously The implementation needs to stay alive throughout the process of change You have documents and application logic with a shelf-life longer than the schemas Maintenance is always cyclical The cyclical method is difficult to apply with XML Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Some reasons for the conclusions l l l XML schemas have no ‘source code’ XML schemas are not models, rather, they are expressions of a model Source control systems add little value Version management of schemas difficult to enforce Team development impractical Change management unscientific o o l Relaxing constraints = easy Tightening constraints = difficult Impact of change impossible to predict Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Some consequences l Increased risk o o l Increased costs o o o l l Increasing likelihood of failure over time Decreasing reliability (‘buggy’ behaviour) Diminished understanding of the system Vendor and supplier lock-in Unpredictable costs Resource bottlenecks Lengthening development cycles Decreased compliance with standards Inability to change existing systems Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Retro-fitting XML into a Methodology l l l Methodologies need supporting technologies Cyclical development best served by Model-Driven Architecture (MDA) Mainstream software development technologies are not designed to cater specifically for XML: o o o l UML, CASE tools… CVS et al App. Dev systems XML development tools are great for helping to create systems, but not for managing or evolving them Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Recommendations Addressing the technical shortfalls of existing methodologies for XML

Progressing the Technology to fit XML l l Team development Object-level: o o o l Modelling and development Source control Version control Impact analysis Change management Deploy and release Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Team Development l Needed for team development of large XML schema families, and the applications they constrain: o o o o o Single-user workspaces Multi-user resources Locking strategies Conflict resolution Impact analysis Coherent version management Groups, Permissions, User roles Definable releases Deployment mechanism Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Object-level… l XML = mechanism for applying names to objects l XML Schema = very OO, but… l XML declarations != objects in an OO sense l A name should be a property of an object l XML objects from a programmer’s perspective are: o o l l Schemas Transformations Instances (etc. ) …all of which provide a container for multiple references to single objects, duplicated to the nth degree Charting dependencies between objects is almost impossible Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Managing objects requires abstraction l l Rise above implementation issues Model-Driven Architecture (MDA) Conceptual External Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies Internal

Version Control A Build: 84 85 86 87 Object versions: A A 1 AA 2 A A 3 A A 4 } Payload Mappings for Order Object Model ORDER A A 5 A A 6 Payload for Ship ACCEPT Mappings for Ship 90 91 A A 7 A A 8 Payload Mappings for Invoice INVOICE SHIP A A E B B B F F E C I I F D L F A I SHIP. XSD } ACCEPT. XSD } ORDER. XSD } XML Payloads 89 A } XML Schemas and Transformations Payload Mappings for Accept 88 INVOICE. XSD ORDER*. XSL ACCEPT*. XSL SHIP*. XSL INVOICE*. XSL Order A E F G H I Accept A B F G H I Ship A B E F G H L Invoice A B C D F G H I J K M Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Pure XML Modelling l Retro-fitting XML into other models: o o Interesting Not straightforward Creates dependencies Re-invention of wheel syndrome with new Recommendations l Ergo: Create a pure XML model l Significant superset of XSD Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Necessary Technology l l l Repository User administration Container hierarchy: o o l l Cabinets Projects Tasks build release deploy Object-level version control: o o o Mix & match = nonsense! Consistency + zero conflict Context of release = the required level Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies

Necessary Process Developer Project #1 Single-user workspace. Administrator Central point of management Check in changes Task #2 Import Working on checked out objects, new objects, and imported objects *. DTD Task #1 Task #2 Task #3 Un-integrated Tasks • • Integrated Tasks Build n of object model A 1. 1 1. 2 2. 0 An Build and versioning Make a new Release Update to new build Check out objects Developer Project #2 Project #3 Project #4 *. XSD Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies *. DTD *. XSL

Summary l l l l Waterfall method is good when applicable Waterfall method often insufficient Cyclical method usually preferable Cyclical method difficult to apply Traditional technology presents challenges XML is unique Schema development = evolution management XML requires unique technology to address evolution management Jim Gabriel (jgabriel@digitalml. com) OASIS Symposium 2005 – The Future of XML Vocabularies