Overview of the ETSI Test Description Language 2

Overview of the ETSI Test Description Language 2 nd MBT User Conference 2012 Tallinn, Estonia September 25– 27, 2012 Andreas Ulrich (Rapporteur) Siemens AG andreas. ulrich@siemens. com © ETSI 2012. All rights reserved

Overview 1. What can I do with TDL? 2. How is TDL designed? 3. How does TDL compare to other approaches? 4. How does TDL look like to users? 5. What comes next? 2 nd MBTUC 2012, Tallinn, Estonia 2

What can I do with TDL? 2 nd MBTUC 2012, Tallinn, Estonia 3

Purpose of TDL q Scope Ø Distributed, concurrent, reactive real-time systems Ø Black-box testing Ø Indifferent to application domains • Telecommunication / automotive / avionics / automation / others Ø Different, user-definable testing languages q Application scenarios Ø Manual specification of tests from use cases, user stories etc. Ø Intermediate representation of automatically generated tests • Representing output from MBT tools Ø Generation of • Executable tests • Test documentation Ø Visualization of test logs 2 nd MBTUC 2012, Tallinn, Estonia 4

TDL Specifies… q Interactions between SUT/tester at observable interfaces (only) Ø System inputs/outputs: messages, calls, signals q Expected flow of interactions Ø Sequential, alternative, concurrent, recurrent behavior, etc. Ø Mock-up (default) behavior, others q Timing constraints Ø System response time (system in/out, out/out constraints) Ø Tester reaction time (system out/in constraints) q Test data Ø Concrete values Ø Constraints on valid test data, i. e. ranges or intervals q Parameterization of test specifications Ø Individual test cases Ø Entire test suites 2 nd MBTUC 2012, Tallinn, Estonia 5

An Application Scenario Example for TDL Test Purpose Spec (Ex. TRA) System Model (State Chart) Automatic Test Case Generation Manual Derivation In Syntax #1 Test Case Spec (TDL) Generation In Syntax #2 Test Documentation (MS Word) Automatic Code Generation Executable Tests (TTCN-3) 2 nd MBTUC 2012, Tallinn, Estonia 6

How is TDL designed? 2 nd MBTUC 2012, Tallinn, Estonia 7

Ingredients of a Software Language Abstract Syntax (abstract syntax graph, meta-model) Concrete Syntax (syntax tree, BNF grammar) Formal Semantics (transitional or operational sem. ) 2 nd MBTUC 2012, Tallinn, Estonia 8

Coverage by TDL Abstract Syntax Standardized, incl. extension capabilities Concrete Syntax Formal Semantics Standardized Informative, by example only 2 nd MBTUC 2012, Tallinn, Estonia 9

Using TDL Static analysis Consistency checks, visualization Common TDL meta-model Test concepts expressed in a single abstract syntax Test generation Transformation Abstract test descriptions Various user-defined syntaxes, e. g. graphical, tabular, textual Executable tests Various test environments and languages, e. g. TTCN-3, C# 2 nd MBTUC 2012, Tallinn, Estonia 10

Definition of the TDL Meta-Model q In fact, two approaches exist Ø OMG’s UML/MOF • International standard • Complex Ø Eclipse Modeling Framework (EMF) Ecore • Simpler than MOF • Good tool support q For practicability reasons, TDL meta-model will be based on a subset of Ecore 2 nd MBTUC 2012, Tallinn, Estonia 11

TDL is Adjustable by User q TDL meta-model can be extended by a user if need arises q TDL concrete syntax may cover only parts of the meta-model q User extensions of the TDL meta-model can be subjected to further TDL standardization and maintenance Extends Partially implements User-defined concrete syntax TDL meta-model w/ user extensions 2 nd MBTUC 2012, Tallinn, Estonia 12

How does TDL look like to users? 2 nd MBTUC 2012, Tallinn, Estonia 13

Examples in Possible TDL Concrete Syntax Sequential test with multiple SUT ports Structured test with default behavior Note: Examples only, concrete syntax is defined by user! 2 nd MBTUC 2012, Tallinn, Estonia 14

Examples in Possible TDL Concrete Syntax testcase join. TC() { UCAROL ! join(Carol, room 1); par { { BOB ? join. PDU(Carol, ? ); BOB ! join. PDU(Bob, room 1) }, { ALICE ? join. PDU(Carol, ? ); ALICE ! join. PDU(Alice, room 1) } }; UCAROL ? joined() } A concurrent test in graphical and textual notation A test suite model Note: Examples only, concrete syntax is defined by user! 2 nd MBTUC 2012, Tallinn, Estonia 15

How does TDL compare to other approaches? 2 nd MBTUC 2012, Tallinn, Estonia 16

TDL vs. UML Testing Profile TDL UTP q Focus on test descriptions of tester/SUT interactions q Various concrete syntaxes q Precise semantics q Common meta-model for tests fosters reuse of tests and tools q Amendable to user’s needs – extension, reduction of scope q Wide scope inherited from UML, covers all kinds of test modeling q Graphical language q Semantics relies on UML q Tool-specific solutions, models hardly transferrable q Captures not all needs, further profiles needed, e. g. MARTE The semantic fuzz of UML 2 nd MBTUC 2012, Tallinn, Estonia 17

TDL vs. TTCN-3 TDL TTCN-3 q Test specification level q Test execution level q Captures requirements on a tester w. r. t. SUT behavior q Detailed tester implementation, incl. tester internal behavior q Leaves room for various tester code generation strategies q Captures the user intension of an executable tester q Clean test spec possible – level of detail adjustable by user q Tester code sometimes cluttered, unreadable q Supports validation of test spec due to concise semantics q Validation hard due to complex semantics 2 nd MBTUC 2012, Tallinn, Estonia 18

Business Value of TDL q Provides guidance to users to define ‘their’ test language (concrete syntax) based on common concepts (TDL abstract syntax) Ø Graphical UML-like / textual / tabular representation of tests q Enables tool vendors to develop specialized test generation tools from common abstract test model Ø Generation of TTCN-3 / Java/JUnit test code / test documentation Ø Generation of sequential tests / concurrent tests q Enables provision of analysis tools independent from a concrete user language Ø Consistency checker Ø Analysis for system input/output races q Offers a way to exchange of test specs at an abstract level Ø Common meta-model for all kind of tests 2 nd MBTUC 2012, Tallinn, Estonia 19

What comes next? 2 nd MBTUC 2012, Tallinn, Estonia 20

Status & Outlook q Achievements so far Ø Identification of the purpose and business value of TDL Ø Collection of related work Ø Discussion on TDL features q Next steps Ø Agreement on TDL core features Ø Design of the TDL meta-model Ø Validate the TDL concepts in a tool Make a (draft) ETSI standard on TDL available in 2013! 2 nd MBTUC 2012, Tallinn, Estonia Do you want to join? 21

Acknowledgments So far, the TDL initiative has attracted quickly 30+ persons and different organisations from industry, tool providers, and research from inside and outside of ETSI. The rapporteur of this activity is thankful for the many stimulating and controversial discussions by various people to lay out the cornerstones of TDL! Siemens AG acknowledges partial funding of this activity from the ARTEMIS Joint Undertaking, grant agreement no. 269335 and the German BMBF. 2 nd MBTUC 2012, Tallinn, Estonia 22

Thank you! Your questions?