Modelbased Testing Modelbased Testing n n n Finite

Model-based Testing

Model-based Testing n n n Finite state machines Statecharts Grammars Markov chains Stochastic Automata Networks

Model-based Testing

Finite State Machine n n Finite state machines have the state changed according to the input. They are different from event flow graphs.

Finite State Machine Test case: {<turn on>, <decrease intensity>, <increase intensity>, <turn off>} off dim off normal bright <turn on> dim <turn off> normal <turn off> bright <turn off> <incr. int. > <decr. Int. > <incr. int. > <decr. int. >

Statecharts n n Statecharts specify state machines in a hierarchy. states: AND, OR, basic states AND: {B 1, B 2} OR: {b 11, b 12} basic state: {A}

Statecharts n configuration: set of states in which a system can be simultaneously. n C 1={CVM, OFF} n C 2={CVM, ON, COFFEE, IDLE, MONEY, EMPTY} n C 3={CVM, ON, COFFEE, BUSY, MONEY, EMPTY}

Statecharts n transition: tuple (s, l, s’) s: source, s’: target, l: label defined as e[g]/a e: trigger g: guard a: action n t 3: coffee[m>0]/dec n n

Statecharts Normal form specification: C 1: {CVM, OFF} C 2: {CVM, ON, COFFEE, IDLE, MONEY, EMPTY} C 3: {CVM, ON, COFFEE, BUSY, MONEY, EMPTY} C 4: {CVM, ON, COFFEE, IDLE, MONEY, NOTEMPTY} C 5: {CVM, ON, COFFEE, BUSY, MONEY, NOTEMPTY} n

Grammars n Context-free grammars to generate test cases. n Example of TC: 1+2*3 n Problem: The test cases may be infinitely long. Weights must be inserted in the rules.

Markov Chains n n n Markov chains are structurally similar to finite state machine, but can be seen as probabilistic automata. arcs: labeled with elements from the input domain. transition probabilities: uniform if no usage information is available.

Markov Chains input domain: {Enter, up-arrow, down-arrow} n variables: cursor location = {“Sel”, “Ent”, “Anl”, “Prt”, “Ext”} project selected = {“yes”, “no”} n states: {(CL = “Sel”, PD = “No”), (CL = “Sel”, PD = “Yes”), . . . } n

Markov Chains n test case: invoke Enter select down-arrow Enter analyze down-arrow Enter

Markov Chains

Markov Chains n n Analysis of the chain: Example 1: Expected length and standard deviation of the input sequences. length: 20. 1 standard deviation: 15. 8

Markov Chains Example 2: Estimate the coverage of the chain states and arcs. n 81. 25% of states appear in the test after 7 input sequences.

Markov Chains Problems with Markov Chains: n Transition matrix may become very large. n The growth of the number of states and transitions impacts in the readability. n Maintainability – it is hard to find all transitions that should be included to keep the model consistent when a new state is added.

Stochastic Automata Networks n n n SAN represents the system by a collection of subsystems: individual behavior (local transitions) and interdependencies (synchronizing events and functional rates). SAN may reduce the state space explosion by its modular way of modeling.

Stochastic Automata Networks Definition of SAN: tuple (G, E, R, P, I) n G = {G 1, . . . , Gm} global states, composed by A 1 x A 2 x. . . x An (Ai is an automaton). n E = {E 1, . . . , Ek} set of events. n R = {R 1, . . . , Rk} set of event rate functions (rate of occurrence of the event). n P = {P 1, . . . , Pk} transition probability functions, one for each pair (event, global state). n I: set on initial states.

Stochastic Automata Networks Example: n n n Automata: {Navigation, Status} Navigation = {Start, Password, Menu} Status = {Waiting, POK, PNot. OK} Events n E = {ST, QT, S, g, f} n ST = {(Start, Wait) → (Pass, Wait)} n S = {(Pass, Wait) → (Menu, POK)}

Stochastic Automata Networks n n n QT = {(Pass, Wait) → (Start, Wait), (Menu, POK) → (Start, Wait)} g = {(pass, wait) → (pass, PNot. Ok)} f = {(pass, PNot. Ok) → (pass, wait)} Initial State n I={(Start, Waiting)}

Markov Chain vs SAN n Test case samples generated using Markov chain and stochastic automat networks. Experiments: n Generation time analysis n Quality of test suite

Markov Chain vs SAN Simple counter navigation MC: 9 states and 24 transitions SAN: 3 automata (2 x 5 x 6) total of 60 states, 9 global reachable states.

Markov Chain vs SAN Calendar Manager MC: 16 states and 67 transitions SAN: 5 automata (2 x 3 x 4 x 2 x 7) total of 336 states, 16 global reachable states.

Markov Chain vs SAN Form-based Documents Editor MC: 417 states and 2593 transitions SAN: 3 automata (2 x 2 x 3 x 10) total of 417 states, 720 global reachable states.

Markov Chain vs SAN n Generation time (simple counter navigation)

Markov Chain vs SAN n Generation time (calendar manager)

Markov Chain vs SAN n Generation time (docs editor)

Markov Chain vs SAN n Quality of test suite

Markov Chain vs SAN n Quality of test suite

Markov Chain vs SAN n Quality of test suite

Markov Chain vs SAN n Quality of test suite

Markov-based GUI Testing n n Event flow graph Have an usage model Retrieve sequences of events Given a start and final state, one could use the properties of markov chains to generate tests.