Subject Name Software Testing Subject Code 10 CS

Subject Name: Software Testing Subject Code: 10 CS 842 Prepared By: Tamilarasi. R Department: CSE Date 6/16/2021

Unit –V System Testing , Interaction Testing 6/16/2021

Topics • Threads • Basic concepts for requirements specification, Finding threads • Structural strategies and functional strategies for thread testing • SATM test threads • System testing guidelines • ASF (Atomic System Functions) testing example. Context of interaction • A taxonomy of interactions, Interaction, composition, and determinism 6/16/2021 • Client/Server Testing, .

Threads • A thread is a sequence of activities that are observable. • There are many levels of granularity for a thread: 1. 2. 3. • – – A sequence of instructions as in a DU-path for unit test A flow of control among a set of related modules as in a MM-path in an integration path. A sequence of “atomic” functions that contributes to the accomplishment of a major functionality as described in a requirements specification. In system testing, we are interested in the 6/16/2021 a) high-level (cross-functional) threads and b) multiple threads.

1. Examples of threads from the ATM Problem Entry of a digit: (port entry of a numerical digit keystroke and a port out of a digit echo on the screen. – 2. Entry of a PIN number: (this entails a set of activities from screen request of a PIN with interleaved sequence of digit entry until completion - - - possibly including cancellation and up to three re-tries). – 3. This is a sequence of atomic system functions that also qualifies as a thread, probably fit for integration testing or unit testing levels. A simple ATM transaction: (this includes a set of functions such as 1) Card entry, 2) PIN entry, 3) choice of transaction type (deposit, withdraw, account detail processing, etc. ) – 4. This is a minimal “atomic system function” and qualifies as a thread of interest at the unit test level. This is a sequence of atomic system functions that is a thread accomplishes a major function, fit for integration testing or system testing levels. Multiple ATM transactions: (this includes a complete user session where several different transactions- - - query, withdraw, deposit of different accounts - - - may transpire. ) – 6/16/2021 This is a sequence of different/multiple threads and is definitely in the domain of system testing.

Some definitions related to thread • An atomic system function (ASF) is an action that is observable at the system level in terms of port entry and port output. • An ASF graph is a directed graph of a ASF represented system where the nodes are the ASF’s and the edges represent the sequential flow. • A source ASF is an ASF that is a source node in an ASF graph. • A sink ASF is an ASF that is a sink node in an ASF graph. • A system thread is a path from a source ASF to a sink ASF in the ASF graph of the system. • The thread graph of a system is a directed graph where the – a) nodes are threads of the system and – b) the edges represent the sequential execution flow from one thread to another. 6/16/2021

Requirements Specification for System Testing • Requirements specification served as a tool (input) for, mainly black-box testing, but also white-box testing even at the unit testing level; it is even more important as a source of developing test cases for system testing. • There are many ways to express requirements and many ways to view the specification. As a whole the requirements specification may be viewed a set of inter-related elements (“basis” for requirements): – – – Data (information and information structure) Actions (functional tasks) Devices (for input, output, and storage) Events (triggers and combination of action and data) Threads (business flow or process flow, involving events) Only thing missing is the non-functional attribute such as quality, performance, etc.

Multiple Ways of Viewing Specification “Basis” elements Used mostly for early spec and development of system devices Contextual model Data Actions Devices Events Structural model Used mostly for development Behavior model Often used for requirements specifications Threads There are many tools for expressing these models: (e. g. ) - state transition diagram ( or Finite State Machine) - E-R diagram - Petri net

Using Finite State Machine or State Transition Diagram State Welcome Card entry (state A) wrong card Display screen 1; Eject card Correct card Display screen 2 Pin Entry (state B) Successful Pin Process Input Next State A correct card B screen 2 A bad card A screen 1; card B good pin # C screen 5 B failed pin # A screen 4 Failed Pin Process Display screen 4 Tabular form Display screen 5 Choosing Transaction (state C) Output Graphical form

Finite State Machine at a “deeper” level for PIN processing Welcome Card entry (state A) Incorrect Pin/ cancel wrong card Display screen 1; Eject card Display screen 4, 1 Correct card Display screen 2 1 st Pin Entry (state B) Successful Pin Process Display screen 5 Choosing Transaction (state C) Incorrect Pin/ cancel 2 nd Pin Entry Display screen 3, 2 (state D) Incorrect Pin/ cancel Display screen 3, 2 Correct Pin Display screen 5 3 rd Pin Entry (state E)

Threads with Finite State Machine • If we can model the requirements specifications with the finite state machine model, then: – A set of transitions (with inputs and outputs) can be a source of threads for system test because it covers: • • Events Action Data Thread ( state and input) (state transition and output) (inputs and outputs) (series of state transitions) • The drawbacks are: – that not all specifications “easily” fit this model – that the determination of what depth level should the finite state machine be for system test; very low a level is more appropriate for unit or integration test

Test Coverage Metrics with Finite State Machine • Every node (state) is covered. – This is a very minimal coverage, much like every statement is covered (recall this from structural path testing) – In our 1 st state transition chart - - - this is equivalent to covering states A, B, and C with correct card and successful PIN processing. • Every node and edge (transition) is covered – This is a more complete coverage if the finite state machine is carried to deep enough level.

Strategies for Thread testing Finite State Machine represented systems • Event based (state-input based): – Each input event occurs – A “typical” or “common” sequence of input events occur – Each input event occurs in every “relevant” data context – For a given context, all “inappropriate” input events occur – For a given context, all possible (both appropriate & inappropriate) inputs events occur • Event based (state-output based): – Each output event occurs for every “cause” • Port-based (both input and output): – Cover all input and out ports

Strategies for thread Testing Data Represented Systems • Some software system requirements are better represented with E-R diagram. • For E-R modeling used for specifications, the threads may be identified via: – Looking at the cardinality of every relationship • • One-one One-many Many-one Many - many – Looking at the participation factor of every relationship – Looking at functional dependencies among relationships • e. g. loaning a book that’s not available

Using Operational Profile when interested in “Effectiveness” • The effectiveness of testing is often measured in terms of problems found versus effort spent or – (# of defects found) / person hours spent – (# of defects found) / number of test cases • Historically, most of the failures also tend to fall in small parts of the system (80% failures occur in 20% of the system - - - e. g. the most heavily traversed threads). – Thus if we can collect the operational profiles of the users, we can identify the most heavily used threads. – This strategy of system testing with operational profile can improve our test efficiency.

User-Operational Profile Example (naive user versus experienced user) (20/20) naïve users (20/80) default f 1 f 4 (5/10) Start (80 users) (10/60) experienced users (60/80) f 1 (5/10) (5/5) f 4 Choice of (10/60) f 1, f 2, f 3 (5/5) (40/40) f 7 f 2 f 3 term f 6 (40/40) (40/60) Note that 2 usage cases take up 75% of operational profile ! term prob =. 25 √ (3/10) (7/10) f 4 (3/3) term prob =. 063 term prob =. 50√√ term (7/7) f 5 prob =. 063 prob =. 037 prob =. 087 term

Software changes and Regression Test • In the life span of a software, we can anticipate changes that result in multiple cycles of fixes and release. Thus it needs to be retested to ensure that worked before still works (did not get regressed). • A common strategy is to re-run the complete system test over as the regression test • A more economical strategy for regression test is to develop (1) new threads for the new areas and (2) only include those old threads that are affected by the new release. (Yes, there may be an element of risk)

INTERACTION TESTING • • It is a relationship Interacts With among Data Events Threads Actions Ports The relationship is reflexive It is binary relation between Data & events Data & threads Events & threads 6/16/2021

Properties of threads and processors • Textbook has two meanings for event Causes confusion, ambiguity, wordy explanations Use two words Use event for instant Use state or activity for duration Occurs between two e • Properties of threads and processors • Threads have duration • They are activities • At one time a processor can execute only one thread Events. • A processor is in a state of executing a thread �Timesharing, multiprocessing interleaves thread execution �Processor changes state for each thread �Here thread durations overlap in time 6/16/2021

• On one processor events can be simultaneous within the minimum resolution of time-grain markers. • BUT reality (hardware) puts an order on those events – puts them in a sequence. - As far as we can tell it is a random choice - At another occurrence the events may be ordered in a different sequence - That is an essential difficulty of interaction testing 6/16/2021

• On different processors, events can occur simultaneously • Common events by definition must occur at the same time • Consider a two people colliding – the collision is a common event to the two people (processors) • Synchronous communication for processors start and end with common events 6/16/2021

• For a single processor • Input and output events occur during thread execution • From the perspective of a thread they cannot occur simultaneously, because they occur at instructions and instructions are executed sequentially • From the perspective of devices port events can be simultaneous • For each port events occur in time sequence 6/16/2021

• Threads occur only within one processor • Do not cross processor boundaries • Have trans-processor quiescence when threads reach processor boundaries • Analogous to crossing unit boundaries in integration testing 6/16/2021

• What we want is sane behaviour • This results from considering events to be in a linear sequence • For example synchronous communications takes into account message transmission time Break the communication into events such as Sender starts sending • Receiver starts receiving • Sender ends sending 6/16/2021

Taxonomy of interactions • Static interactions in a single processor system Static interactions in multiprocessor system Dynamic interactions in a single processor system • Dynamic interactions in multiprocessor system 6/16/2021

Given two propositions P and Q They are contraries if both cannot be true Sub-contraries if both cannot be false Contradictories if exactly one is true R is a subaltern of P if the truth of P guarantees the truth of R – i. e. P → R • Rules in a decision table, if correct, are contradictories • • • 6/16/2021

Static interactions in a single processor • Analogous to combinatorial circuits • Model with decision tables and unmarked event-driven Petri nets • Telephone system example • Call display and unlisted numbers are contraries • Both cannot be satisfied • Both could be waived 6/16/2021

Data-data connectedness – Logical relationships • • 0 -connected Logically independent 2 -connected Sub-alternation 3 -connected – bidirectional Contraries Contradictories Sub-contraries 6/16/2021

EXAMPLES • 3 -connected data-data • When data are deeply related, as in repetition and semaphores • 1 -connected data-event • Context-sensitive port input events 6/16/2021

Dynamic, single processor interactions Six potential interaction pairs Combination pairs of Data Events Threads Each interaction can exhibit 4 different graph connectedness attributes • Result is 24 sub-categories for these interactions IAT– 31 • • • 6/16/2021

Thread –thread interaction • Each thread can be represented by an EDPN • The symbolic names of the places and transitions correspond to those in the EDPN for the system • Synonyms in thread nets need to be resolved when they interact 6/16/2021

Dynamic Multiprocessor Interactions • Problem here is threads and events occur in parallel • We have concurrent behaviour with a collection of communicating sequential processors (CSP) • Have non-deterministic behaviour • To fully understand need to learn the mathematics of CSP • Without that can only work through an example 6/16/2021

Determinism • A system is deterministic if, given its inputs, we can always predict its outputs • A system is deterministic if it always produces the same outputs for a given set of inputs • (For a non-deterministic system it may be difficult to demonstrate different output • Process P chooses non-deterministically at every step whether to engage in event 6/16/2021

• a or b Process Q chooses non-deterministically once whether to engage only with event a or only with event b • P = (a → P) (b → P) Q = (a → Qa) (b → Qb) Qa = (a → Qa) Qb = (b → Qb) • P is deterministic ↔ ∀s : traces (P) • X ∈ refusals (P / s) ↔ X ∩ (P / s)1 = {} P 1 = { e * 〈 e 〉 ∈ traces (P) } �A system is deterministic if at every step the system never refuses to engage in any external event appropriate at that step 6/16/2021

• P is deterministic ↔ ∀s : traces (P) • X ∈ refusals (P / s) ↔ X ∩ (P / s)1 = {} P 1 = { e * 〈 e 〉 ∈ traces (P) } • P 1 definition is the set of events in which P may engage on the first step • P / s is the process after P has engaged in all of the events in the trace s • A trace is a record of the external events in which a process has engaged • A refusal is a set of events in which a process refuses to engage 6/16/2021

Client Server Complexities • Base system has program components Database, application, presentation (logical output) Have a centralized, fat server and distinction • Entire system includes above items plus Network • GUI May have homogeneous or heterogeneous processors 6/16/2021

Client Server Testing • Extend notion of threads beyond an EDPN • CS transaction • A sequence of threads across EDPN boundaries • Client processor --> network --> application ->DBMS back again • Much of the system is stable – e. g. DBMS, existing application Should testing be needed Use functional testing – no source text 6/16/2021