Coveragedriven Automatic Test Generation for UML Activity Diagrams

Coverage-driven Automatic Test Generation for UML Activity Diagrams Mingsong Chen, Prabhat Mishra CISE Department University of Florida, USA Dhrubajyoti Kalita Intel Corporation Folsom, CA, USA

Motivation Hardware Components Hardware Advantages of UML Specification HW/SW Partitioning CIntuitive n CFriendly sig …) , De t u CSemi-formal ayo L , s i es CTool support nth (Sy UML Validation De sig (C n om pil ati on , … ) Estimation Exploration Software Components Major Design Bottleneck DError prone DTime consuming Software Automated Techniques 2

Related Work l There are two kinds of methods to validate the UML activity diagrams. u Model checking based method. Ø Consistency checking (Eshuis, TOSEM 2006). Ø Timing verification (Guelfi et al. , ASPEC 2005; Das et al. , ASPEC 2006). u Model driven testing. Ø Gray-Box method (Wang et al. , ASPEC 2004) Ø Random method (Chen et al. , TCJ 2007) l Currently, there are no approaches that can automatically generate the directed tests from the UML activity diagrams. 3

The Framework of Our Approach Testing criteria Activity diagram translate Formal Model UML activity diagram Parser and Analyzer extract Coverage data (Nu. SMV input) Model checker Fault model generator transform Test cases (Counter example) Properties (CTL, LTL formulas) 4

An Example of ATM start t 1 Verify access code a b [incorrect] [correct] Handle incorrect access code t 2 t 3 1. Activity e. g. Dispense_cash [resolved] Ask for amount c 2: Transition e. g. t 7 with guard [amount available] t 4 t 6 [amount not available] [amount available] d t 7 [not resolved] f Dispense cash Prepare to print receipt t 9 e Generate receipt content t 5 t 8 t 10 g 3. Key path e. g. key path 2 {start} -> {a}->{c} -> {d, f}->{e, f} -> {g} -> {end} Finish transaction and print receipt t 11 end 5

Extraction and Translation l Activity Diagram Extraction: u Prepare necessary information for translation. ØCollect all the graphic elements. Ø Parse the relation between the graphic elements. Ø Analysis the data in activities and conditions. l Translation to NUSMV: u Specify transitions for each element. Ø Activity has 3 states: 0 -unvisited, 1 -visiting and 2 -visited. ØTransition has 2 states: 0 -unfired, 1 -fired. u Testing oriented translation rules. Ø We define 7 rules to describe various cases. 6

Skeleton of Translated ATM Model MODULE state VAR access_code: { A 1, B 1, C 1 }; access_code_input: { A 1, B 1, C 1, D 1 }; . . . start: 0. . 2; syn_1: 0. . 2; verify_access_code: 0. . 2; . . . t 2_cond: 0. . 1; t 3_cond: 0. . 1; . . . ASSIGN init(start): =1; init(syn_1): =0; init(verify_access_code): =0; . . . MODULE ATM(st) ASSIGN next(st. start): = next(st. t 2_cond): =. . . MODULE prepare_print(st) ASSIGN next(st. prepare_print_receipt): =. . . MODULE check_amount(st) ASSIGN next(st. dispense_cash): = next(st. t 7_cond): =. . . MODULE main VAR st: state; atm: ATM(st); print: process prepare_print(st); check: process check_amount(st); The definition of the data, activities and transitions. Specify the initial state of each element. Specify the state transitions according to the translation rules. Initialization of the modules. 7

Fault Models and Examples l Activity fault model. For each activity of AD, the model assumes that such activity is not reachable. u Can be used to check the reachability of each activity. Ø e. g. The activity dispense_cash can not be visited. CTL formula: ! EF ( st. dispense_cash = 2 ) l Transition fault model. For each transition of AD, the model assumes that such transition can not be fired. u Can be used to check the execution order of the activities. u It also can be used to check whether the condition guard is satisfied. Ø e. g. The transition with condition [amount available] can not be fired. CTL formula: !EF( t 7=1 & st. amount_input <= st. amount_available ) l Key path fault model. For each key path of AD, there is no corresponding executable path. u Can be used to check the dynamic behavior of the system. Ø e. g. Key path 2 can not be covered. CTL formula: ! EF ( st. start = 2 & st. verify_access_code=2 & st. ask_for_amount = 2 & st. prepare_print_receipt = 2 & st. dispense_cash = 2 & st. generate_receipt_content = 2 & st. finish_transction_print_receipt = 2 & st. end = 2 & st. t 3_cond=1 & st. t 7_cond=1) 8

Case Study: A Control System l The first case is a small control system. The activity diagram consists of 17 activities, 23 transitions and 6 key paths. u Specification Coverage. Method Coverage (%) Activity Transition path Time (second) Random 30 90 85 50 1. 33 Random 50 95 93 67 2. 35 Random 100 100 83 5. 13 Random 150 100 100 8. 83 Our method 100 100 0. 91 u Implementation Coverage. Package Class Method Block Line 100% 90% 88% 93% u This case study shows an order of magnitude improvement compared with the random method. The generated tests have a good functional coverage in the implementation. 9

Case Study: On-line Stock Exchange System l The second case is a on-line stock exchange system. The activity diagram consists of 27 activities, 29 transitions and 18 key paths. The implementation consists of 7 packages, 39 classes, 372 methods and 2510 lines. u Specification Coverage. Method Coverage (%) Activity Transition path Time (minute) Random 800 96 83 89 19. 06 Random 1000 96 96 94 24. 26 Random 1500 100 100 30. 25 Our method 100 100 7. 08 u Implementation Coverage. Package Class Method Block Line 100% 58% 55% 51% u Our approach reduce the validation effort by 4 times compared to the random method. 10

Conclusions l Automatic test generation and design validation from UML activity diagram is promising. u Time efficient. u Guarantee the correctness of the specification. u Test reusing Ø Consistency checking between different abstraction levels. Ø Good implementation level coverage. l Three contributions of our work u Propose a set of testing oriented translation rules. u Derive the properties from the specification based on fault model. u Develop a model checking based test generation methodology. l Future work u Design and property decomposition. u Property compaction. 11

References l OMG. UML 2. 0 Superstructure Specification. Available at http: //www. omg. org/#UML 2. 0, October 2004. l Grant Martin. UML for Embedded Systems Specification and Design: Motivation and Overview. DATE, 2002. l Rik Eshuis. Symbolic model checking of UML activity diagrams. ACM Transactions on Software Engineering and Methodology , 15(1), 2006. l Dipankar Das, Rajeev Kumar and P. P. Chakrabarti. Timing Verification of UML Activity Diagram Based Code Block Level Models for Real Time Multiprocessor System-on-Chip Applications. APSEC, 2006. l N. Guelfi and A. Mammar. A Formal Semantics of Timed Activity Diagrams and its PROMELA Translation. APSEC, 2005. l L. Wang, J. Yuan, X. Yu, J. Hu, X. Li, and G. Zheng. Generating Test Cases from UML Activity Diagram based on Gray-Box Method. APSEC, 2004. l M. Chen, X. Qiu, W. Xu, L. Wang, J. Zhao and X. Li. UML Activity Diagram Based Automatic Test Case Generation for Java Programs. The Computer Journal, 2007. l A. Cimatti, E. M. Clarke, F. Giunchiglia, M. Roveri. NUSMV: A New Symbolic Model Verifier. CAV, 1999. 12