Towards a KeywordDriven Test Presentation Format using Test

Towards a Keyword-Driven Test Presentation Format using Test. Frame Erik Altena Jos van Rooyen Leon Wolters © Logica. CMG 2006. All rights reserved

Agenda Introduction Test. Frame method and TTCN-3 Test. Frame Language keyword-driven Mapping on TTCN-3 mapping language elements Translation demo the bigger picture Further work future, conclusions Towards a Keyword-Driven Test Presentation Format using Test. Frame 2

Introduction WHY WHERE WHEN test management Test. Frame methodology for structured testing WHAT: analysis HOW: automation Test. Frame is a methodology for structured software testing. Its basic premise is the separation between what to test and how to test it Test analysis (what) is separated from its technical implementation (how) Towards a Keyword-Driven Test Presentation Format using Test. Frame 3

Introduction Test. Frame Language (TFL) transfer money deposit cash withdraw cash Test analysis focuses on the test logic; specified with keywords. Written down in its own Test. Frame Language (TFL). withdraw transfer cash deposit money cash Keywords can be reused with different test data. Towards a Keyword-Driven Test Presentation Format using Test. Frame 4

Introduction Test. Frame Language (TFL) TFL is a test tool independent test language. transfer money deposit cash withdraw cash Test execution tool engine function transfer_money deposit_cash withdraw_cash {{ { transfer “ “; deposit withdraw“ “ ”; ”; from “you”; on account from account to “me”; “you”; }} } During test automation each keyword receives its technical implementation in the chosen test tool. Processing requires an off-the-shelf or tool-specific Test. Frame Engine Towards a Keyword-Driven Test Presentation Format using Test. Frame 5

Introduction Large core language Small core language Detailed specification High-level coordination Distributed set-ups User friendly specification format Test. Frame and TTCN-3 are both used for test specification. Each has its strengths… …that suit some domains better than others. When combined they will appeal to more types of testers from more domains. Towards a Keyword-Driven Test Presentation Format using Test. Frame 6

Agenda Introduction Test. Frame method and TTCN-3 Test. Frame Language keyword-driven Mapping on TTCN-3 mapping language elements Translation demo the bigger picture Further work future, conclusions Towards a Keyword-Driven Test Presentation Format using Test. Frame 7

Test. Frame Language cluster author version Test. Frame test scripts are usually written in a spreadsheet program. TFL_basics Erik Altena 1. 0 Application start application Home Bank User log in EA 1234 test case get balance transfer money check balance Password welcome TFB_1 Balance &Keep(start_bal) To account Amount 123. 45. 678 25, 00 234. 56. 789 653, 88 Balance &startbal - 678, 88 Description Borrowed Rent log out These sheets are translated to tabdelimited text files, called clusters. Clusters are written in Test. Frame Language (TFL). Close the application manually after failure! Application stop application Home Bank Towards a Keyword-Driven Test Presentation Format using Test. Frame 8

Test. Frame Language Each test line starts with a keyword. All following fields contain parameters. Test lines without a keyword are ignored or used for parameter names. Towards a Keyword-Driven Test Presentation Format using Test. Frame 9

Test. Frame Language reporting Pre-defined keywords for test control: user-defined • reporting • structuring • calling subclusters user-defined • flow-of-control • data transfer, etc. Keywords that are user-defined describe the test procedure. Towards a Keyword-Driven Test Presentation Format using Test. Frame 10

Agenda Introduction Test. Frame method and TTCN-3 Test. Frame Language keyword-driven Mapping on TTCN-3 mapping language elements Translation demo the bigger picture Further work future, conclusions Towards a Keyword-Driven Test Presentation Format using Test. Frame 11

Mapping on TTCN-3 Basic mapping: cluster TFL_basics module TFL_basics { authorimport from Erik Keywords Altenaall; version 1. 0 testcase TFB_1() { var float Application start_bal; start application Home Bank get_balance(); get_balance(start_bal); User Password log in transfer_money(); transfer_money(“ 123. 45. 678”, EA 1234 welcome“ 25, 00”, “Borrowed); transfer_money(“ 234. 56. 789”, “ 653, 88”, “Rent”); test case check_balance(start_bal check_balance(); TFB_1 – 678, 88); log_out(); Balance get balancestop_application(“Home stop_application(); &Keep(start_bal) Bank”); } To account Amount Description transfer money 123. 45. 678 25, 00 Borrowed transfer control money () { 234. 56. 789 653, 88 Rent tfl_header_cluster(); tfl_header_cluster(“TFL_basics”); Balance check balance tfl_header_author(); tfl_header_author(“Erik &startbal - 678, 88 Altena”); tfl_header_version(“ 1. 0”); log out start_application(); start_application(“Home Bank”); log_in(“EA 1234”, “welcome”); execute(TFB_1()); Close the application manually after failure! } } Application stop application Home Bank • text file (cluster) = module • test case = test case • keyword = function call • literal parameter = actual value • parameter variable = variable with its definition Towards a Keyword-Driven Test Presentation Format using Test. Frame 12

Mapping on TTCN-3 implemented in separate Some challenges: module • implicit ends of structures • no real support for distr. testing distributed testing explicit data type • implicit data types • pre-defined keywords support for distr. testing • clusters can call explicit end each other explicit end • data transfer Towards a Keyword-Driven Test Presentation Format using Test. Frame 13

Mapping on TTCN-3 Mapping clusters on modules gave a problem in clusters calling each other. module Work { import from Keywords all; import from Test. System. Def all; import from Support. TFL all; } module Attend_meetings { import from Keywords all; import from Test. System. Def all; import from Support. TFL all; Control. From. Work() { tfl_header_cluster(“Work”); tfl_header_author(“Erik Altena”); tfl_header_version(“ 1. 0”); Control. From. Get_coffee(); Control. From. Change. Spreadsheet(); Control. From. Lunch(); Control. From. Get_coffee(); Control. From. Attend_meetings(); } testcase Progress runs on Pr. TC system Pr. SC { … } control () { Control. From. Work(); } control () { Control. From. Attend_meetings(); } Control. From. Attend_meetings() { … execute(Progress()); } Module control parts cannot call each other, but functions can. When each module control only calls a control function, these control functions can call each other. } Towards a Keyword-Driven Test Presentation Format using Test. Frame 14

Mapping on TTCN-3 TFL additions: module TFL_basics { cluster TFL_basics all; authorimport from Actionwords Erik Altena import from Test. System. Def all; version 1. 4 import from Support. TFL all; testcase TFB_1() Application runs on HBTC start application Home Bank system HBSC User { log in EA 1234 var float start_bal; test case map get balance } • explicit ends of structures Password welcome map(self: op. Port. Client, system: op. Port. System); TFB_1 HBTC HBSC get_balance(start_bal); Component Runs on System transfer_money("123. 45. 678", "25, 00", "Borrowed"); self op. Port. Client op. Port. System transfer_money("234. 56. 789", "653, 88", "Rent"); Balance check_balance(startbal - 678, 88); unmap(self: op. Port. Client, system: op. Port. System); &Keep([float] start_bal) To account Amount transfer money 123. 45. 678 25, 00 function Control. From. TFL_basics() transfer money 234. 56. 789 653, 88 { tfl_header_cluster("TFL_basics"); Balance tfl_header_author("Erik Altena"); check balance &startbal - 678, 88 tfl_header_version("1. 4"); Component Runs on start_application("Home Bank"); unmap self op. Port. Client log_in("EA 1234", "welcome"); end test caseexecute(TFB_1()); Description Borrowed Rent System op. Port. System • explicit data type declaration • incorporating test system info (extra test case parameters and new pre-defined keywords) log_out(); stop_application("Home Bank"); log out } control { } Close the application manually after failure! Control. From. TFL_basics(); Application stop application Home Bank } Towards a Keyword-Driven Test Presentation Format using Test. Frame 15

Agenda Introduction Test. Frame method and TTCN-3 Test. Frame Language keyword-driven Mapping on TTCN-3 mapping language elements Translation demo the bigger picture Further work future, conclusions Towards a Keyword-Driven Test Presentation Format using Test. Frame 16

Translation demonstration cluster author version module TFL_basics { TFL_basics from Actionwords all; Erikimport Altena import from Test. System. Def all; 1. 4 import from Support. TFL all; testcase TFB_1() Application runs. Bank on HBTC start application Home system HBSC User Password { log in EA 1234 var float welcome start_bal; map(self: op. Port. Client, system: op. Port. System); TFB_1 HBTC HBSC get_balance(start_bal); Component Runs on System transfer_money("123. 45. 678", "25, 00", "Borrowed"); self op. Port. Client op. Port. System transfer_money("234. 56. 789", "653, 88", "Rent"); Balance check_balance(startbal - 678, 88); unmap(self: op. Port. Client, system: op. Port. System); &Keep([float] start_bal) } To account Amount Description 123. 45. 678 25, 00 Borrowed function Control. From. TFL_basics() 234. 56. 789 653, 88 Rent { Balance tfl_header_cluster("TFL_basics"); Altena"); &startbal - tfl_header_author("Erik 678, 88 tfl_header_version("1. 4"); Component Runs on System start_application("Home Bank"); self op. Port. Client op. Port. System log_in("EA 1234", "welcome"); test case map get balance transfer money check balance unmap end test case execute(TFB_1()); log_out(); stop_application("Home Bank"); log out } Close the application manually after failure! control { Application. Control. From. TFL_basics(); } stop application Home Bank Test. Frame clusters can automatically be translated into TTCN-3 modules. Pre-defined keywords are provided by including a TFLmodule. User-defined keywords have their user-defined function implementation. } Towards a Keyword-Driven Test Presentation Format using Test. Frame 17

Agenda Introduction Test. Frame method and TTCN-3 Test. Frame Language keyword-driven Mapping on TTCN-3 mapping language elements Translation demo the bigger picture Further work future, conclusions Towards a Keyword-Driven Test Presentation Format using Test. Frame 18

Further work Tabular format ASN. 1 IDL Core Notation Graphical format XML Test. Frame Keyword Language format other types & values other presentation formats For a complete keyword-driven test presentation format the language mapping must be bi-directional. All TTCN-3 language elements could be supported in the keyword format. A subset however will increase readability. Towards a Keyword-Driven Test Presentation Format using Test. Frame 19

Further work ? component handler test management test logging codec test executable platform adapter Test. Frame test specification is on a higher abstract level. After translation to TTCN-3 and execution, the logging/reporting should have the same higher abstract level. SUT adapter Towards a Keyword-Driven Test Presentation Format using Test. Frame 20

Further work test logicspecification test tool specification tool implementation Test. Frame separates test logic from its technical implementation. TTCN-3 separates test specification from its implementation. Combined, TTCN-3 could separate test logic from technical specification and implementation. Towards a Keyword-Driven Test Presentation Format using Test. Frame 21

Conclusions • TTCN-3 can have a presentation format on a high-level of abstraction. • A keyword-driven presentation format would increase re-usability. • More work needs to be done on the bi-directional mapping. • More work needs to be done on the logging format. Towards a Keyword-Driven Test Presentation Format using Test. Frame 22

Thank you! E-mail Erik. Altena@Logica. CMG. com Jos. van. Rooyen@Logica. CMG. com Towards a Keyword-Driven Test Presentation Format using Test. Frame 23