About ETSI Protocol and Testing Competence Centre TTCN3

About ETSI Protocol and Testing Competence Centre TTCN-3 Tutorial

What is TTCN-3? q Testing and Test Control Notation Version 3 q Internationally standardized testing language Ø Product of the ETSI Technical Committee MTS (Methods for Testing and Specification) q A programming language that has been used for more than 15 years in standardization as well as industry Ø Specifically designed for black box testing and certification Ø Constantly developed and maintained at ETSI by a team of leading testing experts from industry, institutes, and academia q A testing technology that applies to a variety of application domains and types of testing Ø Knowledge of TTCN-3 is valuable both for employees as well as employers due to its wide applicability Ø Offers potential for reducing training and test maintenance costs significantly Ø Proven to work in very large and complex industrial tests, e. g. , 3 G network elements TTCN-3 Tutorial 2

The TTCN-3 Standards (free download at http: //www. ttcn-3. org/Specifications. htm) q ES 201 873 -1 (Z. 140) Ø TTCN-3 Core Language q ES 201 873 -2 (Z. 141) Ø TTCN-3 Tabular Presentation Format (TFT) q ES 201 873 -3 (Z. 142) Ø TTCN-3 Graphical Presentation Format (GFT) q ES 201 873 -4 (Z. 143) Ø TTCN-3 Operational Semantics q ES 201 873 -5 Ø TTCN-3 Runtime Interface (TRI) q ES 201 873 -6 Ø TTCN-3 Control Interfaces (TCI) q ES 201 873 -7 and onwards (under development) Ø Using ASN. 1, XML, IDL, C/C++ with TTCN-3 Tutorial 3

What makes TTCN-3 different … q From conventional programming or scripting languages? Ø Rich type system including native list types and support for subtyping Ø Embodies powerful build-in matching mechanism Ø Snapshot semantics, i. e. , well defined handling of port and timeout queues during their access Ø Concept of verdicts and a verdict resolution mechanism Ø Support for specification of concurrent test behaviour Ø Support for timers Ø Allows test configuration at run-time Ø Tests focus only on implementation to be tested q From a test tool or vendor proprietary testing language? Ø Not tied to a particular application or its interface(s) Ø Not tied to any specific test execution environment, compiler or operation system Ø TTCN-3 as such is not executable and requires a compiler/interpreter, adapter as well as codec implementations TTCN-3 Tutorial 4

How does TTCN-3 relate to TTCN-2? q TTCN-3 builds on top of TTCN-2 but extends it significantly Ø Core language has now look and feel of a regular programming language: much easier to learn Ø No longer uses protocol specific terminology like PCO, ASP, PDU, etc Ø Different presentation formats: tabular, graphical, … Ø Completely dynamic test configurations Ø Support for synchronous communication Ø Support for testing distributed systems Ø Standardized test system interfaces (TRI & TCI) Ø Improved text string matching: regular expressions Ø Better harmonisation with ASN. 1 Ø Extension mechanism to integrate other type systems, e. g. , XML, ASN. 1, C, … TTCN-3 Tutorial 5

TTCN-3 test systems in a nutshell q TTCN-3 specifies a test but a test system is needed for test execution q TRI and TCI standards define test system architecture Ø TTCN- 3 tools are required to support internal interfaces Ø Allows reuse of test platforms with different tools but also for different SUTs q A test system requires Ø A TTCN-3 tool = TTCN-3 compiler and execution environment ( ) Ø A test platform for a specific device under test ( s+ s) Note: Tools come with default Test Control & Logging Test System Executor TTCN-3 Test System Test Control Logging Codecs TCI [compiled] TTCN-3 Executable TRI SUT Adapter Platform Adapter System Under Test (SUT) TCI = TTCN-3 Control Interface TRI = TTCN-3 Runtime Interface TTCN-3 Tutorial 6

An example adaptation: A IPv 6 test system Test System Executor TTCN-3 Test System Parameter File TCI [compiled] IPv 6 ATS Tool Logging IPv 6 Codecs TTCN-3 Executable TRI Ethernet Adapter Real-time Adapter Open Source Free. BSD Router TTCN-3 Tutorial 7

TTCN-3 Benefits q TTCN-3 is easy to learn Ø Look and feel of a regular programming language q Unambiguous specification and execution of tests Ø Well defined syntax, static - and operational semantics Ø Enables completely automated test execution q Off-the-shelf tools and test systems are readily available Ø Five different commercial TTCN-3 tools on the market q Open source community now taking shape Ø Tools as well as test suites and useful modules q Can be used to specify tests for standardization as well as proprietary product features q Flexible testing technology Ø Virtually no limits to adapt a test system to your needs Ø Scalable – allows test systems to grow over time TTCN-3 Tutorial 8

TTCN-3 Success stories q At ETSI Ø Used for development of any new conformance test suite, e. g. , SIP (Vo. IP), IPv 6 (Core, Mobility, Security), Hiper. MAN / Wi. Max, 3 GPP IP Multimedia Subsystem, … q In industry Ø Applied in a variety of application domains, e. g. , telecom, automotive, financial, … (see www. tt-medal. org) Ø Ericsson reported 1, 000 active licenses at TTCN-3 User Conference 2006 Ø Nokia experiences captured in IEEE Software 23(4) 2006 Ø Motorola reports doubling of testing productivity q Also used beyond Europe Ø Strong community in China TTCN-3 Tutorial 9

Expansion of TTCN-3 Use e p y T t s Te Test Domain Laboratory Integration Telecom Layer Distributed Internet Transport Unit Module TTCN-3 Tutorial 10

TTCN-3 can automate Conformance and Interoperability Testing TTCN-3 Tutorial 11

Main Capabilities of TTCN-3 q Dynamic concurrent testing configurations q Various communication mechanisms (synch and asynch) q Data and signature templates with powerful matching mechanisms (including regular expressions) q Attributes for encoding, display or user-defined information q Test suite parameterization q Control of Test Case execution and selection mechanisms q Control of complex test configurations q Assignment and handling of test verdicts q Harmonized with ASN. 1 (XML and IDL coming) q Different presentation formats q Well-defined syntax, static - and operational semantics TTCN-3 Tutorial 12

The Core Language and Other Presentation Formats Text format Tabular Format TTCN-3 Core Language Graphical Format Presentation Format 3 Presentation Formatn TTCN-3 Tutorial q Core format is text based (most popular) q TTCN-3 can be edited or viewed in other formats Ø Tabular format (for TTCN-2 people) Ø Graphical format (good for visual overview) Ø Other standardized formats in the future? Ø Proprietary formats possible 13

Example Core (Text) Format testcase TC_resolve. Etsi. Www() runs on Dns. Client { timer t_ack; server. Port. send(m_dns. Question("www. etsi. org")); t_ack. start(1. 0); alt { [] server. Port. receive(mw_dns. Answer("172. 26. 1. 17")) { setverdict (pass); } [] server. Port. receive { // any other message setverdict(fail); } [] t_ack. timeout { setverdict(inconc); } } t_ack. stop; } TTCN-3 Tutorial 14

Example Graphical Format testcase TC_resolve. Etsi. Www() runs on Dns. Client mtc server. Port Dns. Client Dns. Port timer t_ack m_dns. Question("www. etsi. org") t_ack alt mw_dns. Answer("172. 26. 1. 17") pass fail ? t_ack inconc t_ack TTCN-3 Tutorial 15

Example Tabular Format Testcase Name TC_resolve. Etsi. Www() Group Purpose System Interface MTC Type Dns. Client Comments Local Def Name Type Initial value t_ack timer Comments Behavior server. Port. send(m_dns. Question("www. etsi. org")); t_ack. start(1. 0); alt { [] server. Port. receive(mw_dns. Answer("172. 26. 1. 17")) { setverdict (pass); } [] server. Port. receive // any other message { setverdict(fail); } [] t_ack. timeout { setverdict(inconc); } } t_ack. stop; Detailed Comments: TTCN-3 Tutorial 16

Use of TTCN-3 With Other Languages q TTCN can be integrated with types systems of other languages ASN. 1 Types & Values XML Types & Values IDL Types & Values TTCN-3 Core Language q Fully harmonized with ASN. 1 (1997) q Harmonized with other languages Ø IDL, XML, C/C++ Other types & Valuesn TTCN-3 Tutorial 17

Minimal Test Configuration TTCN-3 Test System TCI TTCN-3 Executable Test Control Logging MTC Communication with IUT Codecs TRI SUT Adapter Communication with SUT Communication with IUT Platform Adapter System Under Test (SUT) Implementation Under Test q All test behavior is executed on one (main) test component TTCN-3 Tutorial 18

Example Concurrent Test Configuration TTCN-3 Test System Internal Communication TTCN-3 Executable MTC PTC TCI Test Control Logging Codecs TRI SUT Adapter Platform Adapter System Under Test (SUT) Implementation Under Test q A test involves execution of many parallel test components q Dynamic instantiation of components and communication links TTCN-3 Tutorial 19

Building blocks of a TTCN-3 Test Suite Test Data Types Data types which specify • Structure of messages or calls and their information elements (fields, parameters) • Internal data structures (e. g. , for computation) • Possibly encoding or display information Built-in basic types integer, boolean, float, bitstring, hexstring, octetstring, charstring, universal charstring . . . and structured types record, record of, set of union, enumerated . . . and special types such as component, port, verdicttype, default, etc TTCN-3 Tutorial 20

Building blocks of a TTCN-3 Test Suite Test Data Types Actual Test Data Actual test data (values) used during testing • Constants or Templates for specific message or call parameter values • Matching expressions for allowing multiple message or call parameter values • value range, value list, wildcards, presence, length, size, permutation • regular expressions • Using also template decomposition, parameterization and modification TTCN-3 Tutorial 21

Building blocks of a TTCN-3 Test Suite Test Data Types Actual Test Data Test Configuration Static aspects • Test component and port types Dynamic aspects • Dynamic instantiation and management of test components • Mappings of test components to abstract test system interfaces • Connections between test component interfaces • Management of test components TTCN-3 Tutorial 22

Building blocks of a TTCN-3 Test Suite Test Data Types Actual Test Data Test Configuration Test Behaviour test cases • specify sending/receiving messages, computation (e. g. , checksums), and verdict assignment • can be decomposed with functions and altsteps • can (re)use default behaviour • can use timers and timeouts test execution control (optional) • order, repetition, conditions, etc TTCN-3 Tutorial 23

TTCN-3 Module (…) module Etsi. Dns. Tests { // Test definition part Module Definitions Module Control control { // Test execution part // (optional) } } TTCN-3 Tutorial 24

Definitions Part module Etsi. Dns. Tests { // Message structure // Actual test data // Test configuration // Test Case definitions } TTCN-3 Tutorial 25

Structured Definitions Part module Etsi. Dns. Tests { group Message. Structure { // Defintions of message types } group Test. Data { // Templates for messages instances } group Test. System. Configuration { // Port and component types and mappings } group Test. Cases { // Test case definitions } } TTCN-3 Tutorial 26

Message Structure and Test Data type record Dns. Msg // simplified message structure! { Dns. Msg. Kind kind, charstring question, charstring answer optional } type enumerated Dns. Msg. Kind {e_query, e_response} template Dns. Msg m_dns. Question( charstring p_question ) { kind : = e_query, question : = p_question, answer : = omit // no answer } template Dns. Msg mw_dns. Answer( charstring p_answer ) { kind : = e_answer, question : = ? , // any question ok answer : = p_answer } TTCN-3 Tutorial 27

Test Configuration type port Dns. Port message { inout Dns. Msg } // Note: port types may also allow multiple different // message types or restrict the direction type component Dns. Client { port Dns. Port server. Port } // Note: component types can also define multiple port // instances of the same or different port type and // declare timers, constants or variables TTCN-3 Tutorial 28

Test Behaviour query mtc server. Port response testcase TC_resolve. Etsi. Www() runs on Dns. Client { timer t_ack; server. Port. send(m_dns. Question("www. etsi. org")); t_ack. start(1. 0); alt { [] server. Port. receive(mw_dns. Answer("172. 26. 1. 17")) { setverdict(pass); } [] server. Port. receive { // any other message setverdict(fail); } [] t_ack. timeout { setverdict(inconc); } } t_ack. stop; } TTCN-3 Tutorial 29

The Control Part Module (…) module Etsi. Dns. Tests { // Test definition part modulepar boolean mp_example; Module Definitions testcase TC_resolve. Etsi. Www() runs on Dns. Client { //. . as in previous slide } Module Control // Test execution part control { if (mp_example) { execute(TC_resolve. Etsi. Www()); } } } TTCN-3 Tutorial 30

Where can I learn more? q Visit ETSI’s official TTCN-3 web site (www. ttcn-3. org) Ø Public TTCN-3 test suites, useful TTCN-3 modules Ø Links to commercial as well as open source tools q q Read well written TTCN-3 standard suite Join the ETSI mailing list (list. etsi. org/TTCN 3. html) Take a course (see www. tt-medal. org) Read publications Ø Proceedings of Conference for Testing of Communicating Systems (TESTCOM) Ø Presentations of yearly TTCN-3 User Conferences in Europe or Asia (see www. ttcn-3. org) Ø Get a text book http: //www. wiley. com/legacy/wileychi/ttcn-3/ q Register for the next TTCN-3 user conference! TTCN-3 Tutorial 31

Thank You! Phone +33 (0)4 92 94 49 64 e-mail ptcchelp@etsi. org Website www. etsi. org/ptcc TTCN-3 Tutorial