Test Implementation using TTCN-3 Libraries TTCN-3 User Conference,

Test Implementation using TTCN-3 Libraries TTCN-3 User Conference, 7 th of June 2005 1 Stephan Schulz Sebastian Müller Validation & Testing Technologies Computing Architectures Lab Nokia Research Center Helsinki FSCOM Sophia Antipolis France © NOKIA Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

Presentation Outline • About the ETSI IPv 6 Library • Motivation • TTCN-3 Libraries • Library-based Test Suite Implementation • An Example TTCN-3 Library: ETSI Common • Conclusions 2 © NOKIA Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

About IPv 6 Library • Implemented at ETSI by STF 276 • Conformance and interoperability test specs for IPv 6 • Joint e-Europe/ETSI project • • Promote deployment of IPv 6 technology within Europe Establish European player in IPv 6 testing • Developed to support IPv 6 Logo program by IPv 6 Forum • Provide faster access to IPv 6 market • Test implementations are publicly (open+free) available • • 3 © NOKIA Insure high deployment Make tests transparent Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

Motivation • Common problem in TTCN coding is the “copy & paste” mentality of test engineers ÄLots of duplicated code + large code size = hard to maintain • Another problem: “The reinvention of the wheel“ • We desire a test suite implementation which allows as much reuse of TTCN-3 code within as well as between different test suites as possible ÄMuch of the code developed for the IPv 6 Logo certification should be reusable also in a IPv 6 test suite for, e. g. , 3 GPP! • A test suite should also be useful for additional test case development by other, external parties 4 © NOKIA Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

The Idea • Create a test suite using TTCN-3 libraries which can then be reused to compose also other related test suites ÄSo just use TTCN-3 like a regular programming language! • Ability to reuse depends on making important TTCN-3 code structuring decisions early on in test suite development ÄStrong 5 © NOKIA need for a sound methodology! Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

Benefits of TTCN-3 libraries • Less test suite specific TTCN-3 code • Less room for errors in test implementations • Use of validated TTCN-3 code segments • “More readable” test implementations • Less time needed for test case implementation and validation • Avoids the “reinvention of the wheel” Ä Summary: 6 © NOKIA Higher quality test suite implementations! Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

Keys to Reusable TTCN-3 Code • Follow the TTCN-3 Framework • • Methodology for Test Suite Implementation Modularization Rules on how to implement TTCN-3 behavior Naming conventions • Specify well defined interfaces • Use component type compatibility • Document TTCN-3 code • • 7 © NOKIA Standard practice in any programming language Free documentation tools are available Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

Libraries in TTCN-3 • A TTCN-3 library consists of modules separating • • • behavior(s) types and constants templates • Example: Libraries in the IPv 6 test suite Common IPv 6 independent code (e. g, Synchronization) Common IPv 6 code (e. g. , Echo Procedure) Common IPv 6 test suite code (e. g. , Ipv 6 Host Preamble ) But then (not reusable): • Specific IPv 6 test suite code (e. g. , IPv 6 Core Test Cases) • • • 8 © NOKIA Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

Basic Structuring of IPv 6 Test Cases • Decompose test case behavior based on well established methodology • Configuration (CF) function • • Preamble (PR) function • • get IUT into testing state Test body (TP) function • • for test configuration set up perform test as specified by test purpose Postamble (PO) function • ÄAll return IUT to its initial state of the above implemented by invoking test suite, IPv 6, or common library functions! • Implement test case or test case (TC) functions by simply invoking PR, TP, PO functions 9 © NOKIA Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

create/start preamble sync (PO) function (MTC) (PR) function (TP) function (PO) function (PTC 1) No Test case (TC) functions needed 10 © NOKIA (PR) function (TP) function (PO) function (PTCn) • Specify test case (TC) functions for each test component • Use MTC in concurrent test cases only to spawn and synchronize PTCs Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller (TC) function (TP) function (MTC) (TC) function (PR) function testcase TP sync testcase TC functions in Single and Multiple Interfaces

Rules for reusable TTCN-3 Behavior • Function should not contain any stop statement since it prevents the execution of postambles! • Test suite independent functions should avoid to use the setverdict operation • Instead use the return value of a function! • Verdict setting harms reuse, e. g. , testing of valid versus invalid behavior • A module which defines reusable behavior should contain a test component type definition tailored its needs 11 © NOKIA Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

About TTCN-3 Code Documentation • Without proper documentation reuse is questionable • Documentation of library code is well established standard practice in other programming languages (e. g. , man pages) • It should be possible to use a TTCN-3 library or modules without ever reading source files • Any TTCN-3 code should be documented regardless of its reusability! • Guidelines part of framework document • A free documentation tool t 3 doc is available: • • • 12 © NOKIA Follows ideas of similar tools for C and Java, e. g. , doxygen Generates HTML pages from TTCN-3 code comments WWW tool download: www. farodyne. comt 3 doc Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

A Library Example: ETSI Common • Three modules with common type and value definitions • Common basic, data string and text string constant and type definitions, e. g. , c_CRLF, UInt, etc. • Module for verdict control handling • Definition of function return code and verdict setting functions • Module for timing related functions, e. g. , f_sleep • Modules which implement a generic test component synchronization as well as sync examples • 13 © NOKIA Our classic “reinvention of the wheel” Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

The Synchronization Library Module • Implements an easy-to-use synchronization handling for both single as well as multi component test cases • Key concepts • • Named synchronization points TTCN-3 default used for handling shutdown • Uses extensively TTCN-3 log statement • Results in more readable execution traces! • Current main user API: f_self. Sync(sync. Point. Name, ret. Code) f_server. Sync. Clients(#, sync. Point. Name) f_server. Sync. Clients. Timed(#, sync. Point. Name, time) f_server. Wait. For. All. Clients. To. Stop() f_client. Sync(sync. Point. Name, ret. Code) [a_dummy. Shut. Down()] // shutdown example 14 © NOKIA Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

Example of Nice Multi-Component Sync Ready! f_server. Sync. Clients(2, “pr. Done”) f_server. Sync. Clients(2, “tp. Done”) Ready. f_server. Wait. For. All. Clients. To. Stop() (PTC 1) f_PR_tc 1_ptc 2() f_client. Sync(“tp. Done”, e_success); f_PO_tc 1_ptc 1() 15 © NOKIA f_tc 1_ptc 1() f_TP_tc 1_ptc 1() Ready! Ready. (PTC 2) f_PR_tc 1_ptc 1() f_client. Sync(“pr. Done”, e_success); Ready! f_client. Sync(“pr. Done”, e_success); f_TP_tc 1_ptc 2() f_client. Sync(“tp. Done”, e_success); f_PO_tc 1_ptc 2() Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller f_tc 1_ptc 2() Ready! create/start PTC 1 and PTC 2 tc 1() (MTC)

Example of Sync Problem Handling f_server. Sync. Clients(2, “pr. Done”) f_wait 4 All. Clients. To. Shut. Down() STOP! f_PR_tc 1_ptc 1() f_client. Sync(“pr. Done”, e_error); a_shut. Down_if 1() (calls stop!) 16 © NOKIA STOP! (PTC 2) f_tc 1_ptc 1() (PTC 1) Ready! f_PR_tc 1_ptc 2() f_client. Sync(“pr. Done”, e_success); a_shut. Down_if 2() (calls stop!) Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller f_tc 1_ptc 2() STOP! create/start PTC 1 and PTC 2 tc 1() (MTC)

Example of Self Synchronization (MTC) f_PR_tc 2() f_TP_tc 2() STOP! f_self. Sync(“tp. Done”, e_error); tc 2() f_self. Sync(“pr. Done”, e_success); a_shut. Down_if 1() (calls stop!) • Note that shutdown defaults are neither test case nor component but only interface specific (e. g, IPv 6, SIP, etc. ) • Therefore the same default can be used to shutdown any test component (which uses that interface) in any test case. 17 © NOKIA Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

Conclusions • This really works in practice • • The ETSI common TTCN-3 library has already been successfully deployed within STF 276 as well as a number of different protocol test suites within Nokia First versions of the generic IPv 6 and IPv 6 Core libraries have been successfully used in the execution of the ETSI IPv 6 test suite prototype against a Linux Ipv 6 SUT • At the end the real key to implementing truly reusable TTCN-3 code are • • • 18 © NOKIA TTCN-3 framework Well-defined interfaces TTCN-3 code documentation Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

References • http: //www. ipt. etsi. org • More info about this STF and documents • Test suite and executable download • ETSI DTS/MTS-IPT-0001, “Methods for Testing and Specification (MTS); Internet Protocol Testing (IPT); IPv 6 Testing: Methodology and Framework”, Sophia-Antipolis, 2005. • S. Moseley et al. , “IPv 6 Test Specifications from ETSI”, Global Ipv 6 Summit, Barcelona, (to appear), June, 2005. 19 © NOKIA Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller

About Component Type Compatibility • Key to create self contained libraries is the runs on clause with TTCN-3 functions • An IPv 6 preamble may invoke, e. g. , a synchronization function, if its component type definition is compatible type component Ipv 6 Comp { Ip. Port pt_ip; Sync. Port pt_sync; timer t_sync; } type component Sync. Comp { Sync. Port pt_sync; timer t_sync; } function f_PR_Echo. Proc. Up(…) runs on Ipv 6 Comp { var Fnc. Ret. Code v_ret; v_ret : = f_echo. Proc(); f_client. Sync(“pr. Done”, v_ret); } 20 © NOKIA Test Implementation using TTCN-3 Libraries. ppt/ 07. 6. 2005 /S. Schulz and S. Müller function f_client. Sync(…) runs on Sync. Comp { // some code }