Overview of the ETSI Test Description Language 2

Overview of the ETSI Test Description Language 2 nd MBT User Conference 2012 Tallinn, Estonia September 25– 27, 2012 Andreas Ulrich (Rapporteur) Siemens AG andreas. ulrich@siemens. com © ETSI 2012. All rights reserved

Overview 1. What can I do with TDL? 2. What are the concepts behind TDL? 3. How does TDL compare to other approaches? 4. How does TDL look like to users? 5. What comes next? 2 nd MBTUC 2012, Tallinn, Estonia 2

What can I do with TDL? 2 nd MBTUC 2012, Tallinn, Estonia 3

Purpose of TDL q Scope Ø Black-box testing of distributed communicating real-time systems Ø Design/representation of test descriptions before implementing them • Using different representation syntaxes Ø Indifferent to application domains • Telecommunication / automotive / avionics / automation / others q Application scenarios Ø Manual specification of tests from use cases, user stories etc. Ø Intermediate representation of automatically generated tests from • MBT tools, simulators, existing test logs, . . . Ø Generation of • Executable tests • Test documentation Ø Visualization of test logs 2 nd MBTUC 2012, Tallinn, Estonia 4

An Application Scenario Example for TDL Test Purpose Spec (Ex. TRA) System Model (State Chart) Automatic Test Case Generation Manual Derivation In Syntax #1 Test Description (TDL) Generation In Syntax #2 Test Documentation (MS Word) Automatic Code Generation Executable Tests (TTCN-3) 2 nd MBTUC 2012, Tallinn, Estonia 5

TDL Specifies… q Interactions between SUT/tester at observable interfaces (only) Ø System inputs/outputs: messages, calls, signals q Expected flow of interactions Ø Sequential, alternative, concurrent, recurrent behavior, etc. Ø Mock-up (default) behavior, others q Timing constraints Ø System response time (system in/out, out/out constraints) Ø Tester reaction time (system out/in constraints) q Abstract test data Ø Relevant values Ø Constraints on test data, i. e. ranges or intervals q Parameterization of test descriptions 2 nd MBTUC 2012, Tallinn, Estonia 6

What are the concepts behind TDL? 2 nd MBTUC 2012, Tallinn, Estonia 7

Ingredients of a Software Language Abstract Syntax (abstract syntax graph, meta-model) Concrete Syntax (syntax tree, BNF grammar) Semantics (transitional or operational sem. ) 2 nd MBTUC 2012, Tallinn, Estonia 8

Coverage by TDL Abstract Syntax Standardized, incl. extension capabilities Concrete Syntax At least one standardized representation 2 nd MBTUC 2012, Tallinn, Estonia Semantics Standardized 9

Using TDL Static analysis Consistency checks, visualization Common TDL meta-model Test concepts expressed in a single abstract syntax Test generation Transformation Abstract test descriptions Various user-defined syntaxes, e. g. graphical, tabular, textual Executable tests Various test environments and languages, e. g. TTCN-3, C# 2 nd MBTUC 2012, Tallinn, Estonia 10

TDL is Adjustable by User q TDL concrete syntax may cover only parts of the meta-model q TDL meta-model can be extended by a user if need arises q User extensions of the TDL meta-model can be subjected to further TDL standardization and maintenance Extends Partially implements User-defined concrete syntax TDL meta-model w/ user extensions 2 nd MBTUC 2012, Tallinn, Estonia 11

How does TDL look like to users? 2 nd MBTUC 2012, Tallinn, Estonia 12

Examples in Possible TDL Concrete Syntax Sequential test with multiple SUT ports Structured test with default behavior Note: Examples only, concrete syntax is definable by user! 2 nd MBTUC 2012, Tallinn, Estonia 13

Examples in Possible TDL Concrete Syntax testdescription join. TD() { UCAROL ! join(Carol, room 1); par { { BOB ? join. PDU(Carol, ? ); BOB ! join. PDU(Bob, room 1) }, { ALICE ? join. PDU(Carol, ? ); ALICE ! join. PDU(Alice, room 1) } }; UCAROL ? joined() } A concurrent test in graphical and textual notation A test scenario model Note: Examples only, concrete syntax is definable by user! 2 nd MBTUC 2012, Tallinn, Estonia 14

How does TDL compare to other approaches? 2 nd MBTUC 2012, Tallinn, Estonia 15

TDL vs. UML Testing Profile TDL UTP q Focus on test descriptions of tester/SUT interactions q Various concrete syntaxes q Common meta-model with semantics fosters reuse of tests and tools q Amendable to user’s needs – extension, reduction of scope q Wide scope inherited from UML, covers all kinds of test modeling q Graphical language q Tool-specific solutions, models hardly transferrable q Captures not all needs, further profiles needed, e. g. MARTE The semantic fuzz of UML 2 nd MBTUC 2012, Tallinn, Estonia 16

TDL vs. TTCN-3 TDL TTCN-3 q Abstract test specification level – low level of detail q Lower test execution level – high level of detail q Captures requirements on a tester w. r. t. SUT behavior q Detailed tester implementation, incl. tester internal behavior q Leaves room for various tester code generation strategies q Captures the user intention to implement an executable tester 2 nd MBTUC 2012, Tallinn, Estonia 17

Business Value of TDL q Support to test design before rushing to test code Ø Better quality of tests Ø Higher productivity in testing q Common meta-model for all kind of tests descriptions Ø Enabled development of specialized tools for • Generation of executable test code using different strategies • Test documentation and visualization • Consistency verification and further analysis of test descriptions Ø Offers a way to harmonize test descriptions coming from different sources q Support for customized test languages (concrete TDL syntax) Ø Graphical Ø Textual Ø Tabular representation of tests 2 nd MBTUC 2012, Tallinn, Estonia 18

What comes next? 2 nd MBTUC 2012, Tallinn, Estonia 19

Status & Outlook q Achievements so far Ø Identification of the purpose and business value of TDL Ø Collection of related work Ø Discussion on TDL features q Next steps Ø First draft of TDL covering core features Ø Definition of the TDL meta-model Ø Validation of TDL in a reference implementation Make a stable draft of the ETSI standard on TDL available in 2013! 2 nd MBTUC 2012, Tallinn, Estonia Do you want to join? 20

Acknowledgments So far, the TDL initiative has attracted quickly 30+ persons and different organisations from industry, tool providers, and research from inside and outside of ETSI. The rapporteur of this activity is thankful for the many stimulating and controversial discussions by various people to lay out the cornerstones of TDL! Siemens AG acknowledges partial funding of this activity from the ARTEMIS Joint Undertaking, grant agreement no. 269335 and the German BMBF. 2 nd MBTUC 2012, Tallinn, Estonia 21

Thank you! Your questions?