STF 454 DESIGN OF TDL STATUS REPORT Last

STF 454 “DESIGN OF TDL” – STATUS REPORT Last change: 2014 -01 -15 © ETSI 2011. All rights reserved

Document History 2014 -01 -15: Input to MTS#61 2013 -09 -30: Input to joint STF/SG meeting and MTS#60 2013 -05 -13: Input to MTS#59 2013 -04 -03: In preparation of the 3 rd session 2013 -03 -16: Outcome from Budapest meeting 2013 -02 -25: Reviewed during SG#2; agenda 2 nd session 2013 -02 -22: Document created 2 © ETSI 2011. All rights reserved

TDL Documentation Latest TDL standard document • https: //docbox. etsi. org/STF 454_MTS_TDL/Public/ETSI_ES_TDL. pdf Overview on TDL • https: //docbox. etsi. org/STF 454_MTS_TDL/Public/STF 454 -TDLOverview. pptx Current TDL meta-model • Requires Eclipse Modeling Distribution and Papyrus UML editor • https: //docbox. etsi. org/STF 454_MTS_TDL/Public/TDL-MM. zip 3 © ETSI 2011. All rights reserved

REPORT ON SESSIONS/MEETINGS 4 © ETSI 2011. All rights reserved

Outcome from 1 st Session (week 7) Goal: Gain common understanding in the team Discussion of TDL use cases and users of TDL Clarifying the scope of TDL for • Conformance (functional) testing • Interoperability testing Discussed examples of test specs • 3 GPP UE conf tests • IMS iop tests Any input to TDL must be given by convincing examples Walkthrough of TDL feature list (not completed) • Done: Structure – Interactions – Behavior • Open: Data – Time – Misc. 5 © ETSI 2011. All rights reserved

Outcome from 2 nd Session (week 10) Goal: Establishing the foundations of TDL Closer look at examples • IMS iop testing, Ericsson IMS, IPv 6 iop testing • 3 GPP conf testing, OMA conf testing • Generated tests from MBT tools (other STF) • HLTD requirements (pre-TDL work) • Test spec languages CCDL Consolidation of gathered TDL features Further discussion on TDL features – with action items Result and next steps • Initial feature list mostly completed (can be always extended) • Further discussions needed, addressed in Action Items 6 © ETSI 2011. All rights reserved

Outcome from 3 rd Session (week 15) Goal: Laying the foundation of the TDL standard First and complete walk-through and discussion of the metamodel covering • Structure of a TDL model • Packaging concept • Test architecture • Behavioural expressions • Data values • Time concepts • Miscellaneous concepts Fixing the structure of the TDL document 7 © ETSI 2011. All rights reserved

Milestone A: Early Draft on TDL Submitted on 29 -April-2013 (week 18) Approved at MTS#59 Contents • Structure and To. C of TDL document • Rough description of the meta model (abstract syntax) 8 © ETSI 2011. All rights reserved

Outcome from 4 th Session (week 29) Goal: Close gaps in the TDL meta-model; add tool support Addressing issues raised during the joint STF/SG meeting Review of the meta-model from the early draft • Detailing data types and data • Detailing time concepts Setup of a proper tool environment to design the meta-model • Eclipse Modeling Distribution, Kepler edition • Papyrus UML editor plug-in • Extensions by MFW to support embedded documentation Results • Design tool for the meta-model • Next iteration of the TDL meta-model 9 © ETSI 2011. All rights reserved

Outcome from 5 th Session (week 36) Goal: Stabilize the TDL meta-model Internal review of the current meta-model to fix inconsistencies and address shortcomings Reaching consensus within the STF on features and their semantics Introducing documentation generation from the meta-model project within Eclipse Preparation of the next stable draft Results • Stable meta-model for next draft • Validation of meta-model with available tools (EMF editor, x. Text) 10 © ETSI 2011. All rights reserved

Milestone B: Stable Draft on TDL Submitted on 18 -Sep-2013 (week 38) Approved at MTS#60 Contents • All remarks from SG addressed, but one Definition of sub- • configurations postponed to future TDL version Detailed description of the meta-model • All packages • All classes contained within • All of their attributes Introduction and motivation updated • Examples on concrete syntaxes for TDL submitted as separate contributions to MTS#60, e. g. proposal for 3 GPP 11 © ETSI 2011. All rights reserved

Outcome from 6 th Session (week 45) Goal: Addressing feedback from SG#6/MTS#60 meeting Review of all parts of the MM and discussion according to feedback from SG Discussion of results from MM validation performed by STF members Editing of the MM description Results • Updated MM • Improved documentation 12 © ETSI 2011. All rights reserved

Outcome from 7 th Session (week 49) Goal: Finalization of TDL document Further improvements of the MM, mainly parameterization Review and improvement of the MM description Discussion on TDL concrete syntax and examples from 3 GPP and IMS Results • Further consolidation of MM • Improved readability of MM and its description • Text ready for delivering the Final Draft 13 © ETSI 2011. All rights reserved

Milestone C: Final Draft on TDL Submitted on 20 -Dec-2013 (week 51) Approval at MTS#61 planned Contents • Validated(!) description of the meta-model • Improved introduction (Clause 4) • Informative annex on an example of a concrete textual syntax and it • 14 application Meta-model provided as a technical document (Papyrus UML project) © ETSI 2011. All rights reserved

Next Steps Joint SG/STF meeting, 2014 -01 -17 • Discussion on submitted final draft MTS#61 meeting, 2014 -01 -29/30 • Approval of final draft • Discussion on TDL phase 2 (2014) 15 © ETSI 2011. All rights reserved

Final Session Plan, 1 st Period 16 from to 21. 01 28. 01 04. 02 11. 02 18. 02 25. 02 04. 03 11. 03 18. 03 25. 03 01. 04 08. 04 15. 04 22. 04 29. 04 06. 05 13. 05 20. 05 27. 05 25. 01 01. 02 08. 02 15. 02 22. 02 01. 03 08. 03 15. 03 22. 03 29. 03 05. 04 12. 04 19. 04 26. 04 03. 05 10. 05 17. 05 24. 05 31. 05 week 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 © ETSI 2011. All rights reserved GA FK PM AU MFW CTI E 2, 5 H 1, 0 C 2, 5 H 1, 0 E 2, 5 C 2, 5 H 1, 0 H 2, 0 E 2, 5 E 4, 0 E 2, 5 E 4, 0 H 2, 0 C 2, 5 H 1, 0 H 2, 0 E 4, 0 H 2, 0 H 2, 5 H 0, 5 Other Notes E 2, 5 E 1, 5 E 1, 0 E 4, 0 E 2, 8 H 2, 0 H 1, 0 H 4, 0 H 0, 5 H 2, 0 E 2, 0 Prep. meet. S#1: Wed-Thu full days, Fri 1/2 day S#2: Budapest, Wed, Thu, Fri morning G 2 M, Wed Easter: 29. 3. --1. 4. S#3: ETSI, Tue, Wed, Thu, Fri early draft ready MTS#59, 14. -15. 5. G 2 M, Wed, Thu

Session Plan, 2 nd Period 17 from to 15. 07 22. 07 29. 07 05. 08 12. 08 19. 08 26. 08 02. 09 09. 09 16. 09 23. 09 30. 09 07. 10 14. 10 21. 10 28. 10 04. 11 11. 11 18. 11 25. 11 02. 12 09. 12 16. 12 19. 07 26. 07 02. 08 09. 08 16. 08 23. 08 30. 08 06. 09 13. 09 20. 09 27. 09 04. 10 11. 10 18. 10 25. 10 01. 11 08. 11 15. 11 22. 11 29. 11 06. 12 13. 12 20. 12 week © ETSI 2011. All rights reserved 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 GA FK PM AU C 4, 0 MFW C 2, 0 CTI E 2, 0 Other Notes E 4, 0 E 4, 0 E 2, 0 ? ? C 4, 0 C 4, 0 E 2, 0 E 4, 0 E 4, 0 S#4 Munich, Tue-Fri E 2, 0 phone conf on Mon or Tue S#5, ETSI, Tue-Fri stable draft ready MTS-TDL, 30. 9. , MTS#60, 1. /2. 10. 22. -24. 10. UCAAT S#6, Berlin, Tue-Fri S#7, ETSI, Tue-Fri final draft ready

STF REPORTING 18 © ETSI 2011. All rights reserved

STF Technical Risks Risk Severity Likelihood Mitigation Scope and contents of TDL draft moving or changing due to “green field” development High Low Move insufficiently discussed topics to a follow-up STF on TDLv 2. Validity and consistency of Medium developed meta-model cannot be guaranteed Low Introduced tool support for meta-model design and doc generation is faulty, which causes heavy manual rework of the document Happened Work out TDL concrete syntax and its mapping to the MM; enforce validation activities outside the STF. Fall-back to document-centric work to produce the next draft; put more efforts in the tool support. Low end-user acceptance due Medium to ill-designed TDL concrete syntax 19 © ETSI 2011. All rights reserved Medium Keep close contact especially with ETSI end-users; keep MM extensible; plan for a follow-up activity

Very Tight STF Resources Equal distribution of budget over 5 experts (opposed to the suggested 4) dilutes overall resources • Voluntary work shot up to 30% • Initial ramp-up to get all experts on same ground required • Parallelisation of work difficult to achieve • Mainly for the analysis and writing part • But not for the design In total 179 person days for contracted experts • 129. 5 PD spent up to milestone B • Only 49. 5 PD to achieve milestone C High voluntary efforts for MM validation 20 © ETSI 2011. All rights reserved

DISCUSSION OF TDL PHASE 2 Review of the related To. R 21 © ETSI 2011. All rights reserved

TDL Roadmap 2013 TDL 1 2014 TDL 2 2015 TDL 3 22 • Foundation, manual creation of test descriptions, representation of test cases, documentation of tests • Automation support for generating concrete tests • Concrete syntaxes for end-users and tool interop • Features for the generation of abstract tests • Support of further use cases, e. g. performance tests © ETSI 2011. All rights reserved

Goal and Objectives of TDL 2 Goal • Supporting ETSI and industrial users in using TDL Objectives • Extended TDL MM for supporting test automation • Default concrete syntaxes • Graphical syntax for end-users • Textual exchange syntax for tool interoperability • Analysis on the needs for a textual syntax to support ETSI use cases 23 © ETSI 2011. All rights reserved

Deliverables from TDL 2 Work Item code Standard number Working title Scope D 1 RES/ES 203 119 -1 V 1. 2. 1 Test Description Language; Meta-Model and Semantics Scope: common concepts, meta-model, semantics D 2 DES/ ES 203 119 -2 V 1. 1. 1 Test Description Language; Graphical Syntax Scope: TDL graphical concrete syntax for end users D 3 DES/ ES 203 119 -3 V 1. 1. 1 Test Description Language; Exchange Format Scope: TDL exchange format for tool interoperability Del. Optionally ES part 4 on TDL textual concrete syntax (no WI created yet) 24 © ETSI 2011. All rights reserved

TDL 2 organization of work Start: Feb 2014 • Task 0: Project management • Task 1: Extension of TDL MM (02— 12/2014) • Task 2: Graphical concrete syntax (02— 05— 12/2014) • Task 3: Exchange syntax (06— 12/2014) • Task 4 a: Analysis on ETSI concrete syntax (02— 05/2014) Potential STF extension: Jun 2014 (decision at MTS#62) • Task 4 b: ETSI concrete syntax (06— 12/2014) End: Dec 2014 • WI: updated MM description + semantics • WI: concrete syntax + MM mapping • WI: exchange syntax + MM mapping 25 © ETSI 2011. All rights reserved

TDL 2 milestones M 0: 02/2014 • Start of work of Tasks 0, 1, 2, 4 M 1: 05/2014 • • Discussion at MTS#62 (05/2014) (T 1) Early draft: updated MM (T 2) Early draft: graphical syntax (T 3) Start of Task 3 (T 4) Decision paper on text syntax M 2: 09/2014 Discussion at MTS#63 (10/2014) • (T 1) Stable draft: updated MM • (T 2) Stable draft: graphical syntax • (T 3) Early draft: exchange syntax M 3: 12/2014 • (T 1) Final draft: updated MM • (T 2) Final draft: graphical syntax • (T 3) Final draft: exchange syntax 26 © ETSI 2011. All rights reserved Approval at MTS#64 (02/2015)

TDL task efforts and budget Efforts for experts • Task 0: Management 6 PD (STF leader) • Task 1: Extended MM 60 PD • Task 2: Graphical syntax 84 PD • Task 3: Exchange syntax 48 PD • Task 4 a: Textual syntax 10 PD Expert voluntary contribution of 20% Additionally, 25 PD support from CTI (voluntary!) Budget: 99, 600 € + 4000 € travel costs • All initial resources granted, no cut! 27 © ETSI 2011. All rights reserved

Features of extended TDL MM Test Configuration • Specification of sub test configurations (subsets, renaming etc. ) Types and Data, Time • Expressions over user data types (e. g. guards) and time constraints • Support for variables and data functions • Support for arrays of data types and (possibly) component and gate types Behaviour • Refinement of interactions for different interface types • Extensions of interaction flow operations, e. g. MSC co-regions Support of abstraction levels in TDL • A) Test description level descriptive specification of tests • B) Test implementation level abstract description of concrete (TTCN-3) tests • Automatic or guided refinement from A to B 28 © ETSI 2011. All rights reserved

Features of TDL concrete syntaxes Graphical syntax • Foundation: OMG Diagram Definition (DD) architecture • Certainly inspired by UML Exchange syntax • Foundation: OMG Test Information Interchange Format (Test. IF) • Human-readable syntax vs. XML-based syntax? Mapping rules for all syntactical elements to MM elements need to be defined 29 © ETSI 2011. All rights reserved

TDL FEATURE DISCUSSION 30 © ETSI 2011. All rights reserved

Communication STF – SG Ensure overall direction of TDL design is OK Resolve deadlocks in STF internal discussions by providing guidance and advice Develop a roadmap for TDL feature list with mapping to TDL use cases • STF will make proposals • Feedback expected from SG 31 © ETSI 2011. All rights reserved

TDL Use Cases Purpose: Promote TDL features based on their potential usage 32 UC Short description Example A TDL for documentation (incl. informal parts) 3 GPP test specs B TDL for generation of tests that can be made executable (i. e. all parts are formal) Automatic mapping of a TDL spec to partial TTCN-3 code C TDL for representation of generated tests (i. e. output from MBT tools) Test cases generated from system models D TDL for representation of test logs Test execution log of a TTCN-3 tool E TDL for test generation (i. e. input to MBT tools) Test models as activity diagrams F TDL for performance testing On-the-fly testing from a TDL spec G TDL for interoperability testing Use case models, from which tests are derived © ETSI 2011. All rights reserved

Internal TDL Feature Discussion For use in STF, communication with SG Proposed structure (template) • • 33 TDL feature name Covered TDL use cases Motivation Description (abstract syntax, meta-model) Semantics Proposed textual concrete syntax Proposed graphical concrete syntax Examples of use © ETSI 2011. All rights reserved

TDL Feature Description in Standard Structure of TDL feature description • TDL feature name (title of sub-clause) • Overview (covered in early draft) • Free description of the feature • Abstract syntax • Representation of the feature and its elements in the meta-model • Semantics • Preferably formal description of the semantics of the feature • Constraints on the feature that can be statically analyzed • Classifier description • Description of all elements contained in the meta-classes Note 1: Concrete syntax moved to annex! 34 © ETSI 2011. All rights reserved