Path selection criteria Tor Stlhane Wande Daramola Why
- Slides: 41
Path selection criteria Tor Stålhane & ‘Wande Daramola
Why path selection criteria Doing white box testing (Control flow testing, data flow testing, coverage testing) using the test-all-paths strategy can be a tedious and expensive affair. The strategies discussed here alternative ways to reduce the number of paths to be tested. As with all white box tests, it should only be used for small chunks of code – less than say 200 lines.
Data flow testing -1 • Data flow testing is a powerful tool to detect improper use of data values due to coding errors. main() { int x; if (x==42){. . . } }
Data flow testing -2 • Variables that contain data values have a defined life cycle. They are created, they are used, and they are killed (destroyed) - Scope { // begin outer block int x; // x is defined as an integer within this outer block …; // x can be accessed here { // begin inner block int y; // y is defined within this inner block. . . ; // both x and y can be accessed here } // y is automatically destroyed at the end of this block. . . ; …; // x can still be accessed, but y is gone } // x is automatically destroyed
Static data flow testing • Variables can be used – in computation – in conditionals • Possibilities for the first occurrence of a variable through a program path – ~d the variable does not exist, then it is defined (d) – ~u the variable does not exist, then it is used (u) – ~k the variable does not exist, then it is killed or destroyed (k)
define, use, kill (duk) – 1 We define three usages of a variable: • d – define the variable • u – use the variable • k – kill the variable. A large part of those who use this approach will only use define and use – du. Based on the usages we can define a set of patterns potential problems.
duk – 2 We have the following nine patterns: • • • dd: define and then define again – error dk: define and then kill – error ku: kill and then used – error kk: kill and then kill again – error du: define and then use – OK kd: kill and then redefine – OK ud: use and then redefine – OK uk: use and then kill – OK uu: use and then use – OK
Example: Static data flow testing For each variable within the module we will examine define-use-kill patterns along the control flow paths
Example cont’d: Consider variable x as we traverse the left and then the right path ~define correct, the normal case define-define suspicious, perhaps a programming error define-use correct, the normal case ddu du
duk examples (x) – 1 Define x Use x ddu Define x Use x du
Example Cont’d: Consider variable y ~use use-define-use use-kill major blunder acceptable correct, the normal case acceptable uduk udk
duk examples (y)- 2 Use y Define y Use y Kill y uduk udk
Example Cont’d: Consider variable z ~kill programming error kill-use major blunder use-use correct, the normal case use-define acceptable kill-kill-define probably a programming error acceptable define-use correct, the normal case kuuud kkduud
duk examples (z) - 3 Kill z Define z Use z Use z Define z kuuud kkduud
Dynamic data flow testing Test strategy – 1 Based on the three usages we can define a total of seven testing strategies. We will have a quick look at each • All definitions (AD): test cases cover each definition of each variable for at least one use of the variable. • All predicate-uses (APU): there is at least one path of each definition to p-use of the variable
Test strategy – 2 • All computational uses (ACU): there is at least one path of each variable to each cuse of the variable • All p-use/some c-use (APU+C): there is at least one path of each variable to each cuse of the variable. If there any variable definitions that are not covered then cover a c-use
Test strategy – 3 • All c-uses/some p-uses (ACU+P): there is at least one path of each variable to each c-use of the variable. If there any variable definitions that are not covered then cover a p-use. • All uses (AU): there is at least one path of each variable to each c-use and each puse of the variable.
Test strategy – 4 • All du paths (ADUP): test cases cover every simple sub-path from each variable definition to every p-use and c-use of that variable. Note that the “kill” usage is not included in any of the test strategies.
Application of test strategies – 1 All definitions All c-use Define x Kill z c-use x Define z Define y p-use z Define x p-use y Kill z Define x c-use z All p-use Define x c-use z Kill z c-use x Define z Define y p-use z c-use z Kill y Define z
Application of test strategies – 2 ACU APU+C Define x p-use y Kill z Define x c-use z Kill z c-use x Define z Define y p-use z c-use z Kill y Define z
Relationship between strategies All paths All du-paths All uses All p/some c All c/some p All p-uses All c-uses All defs Branch The higher up in the hierarchy, the better is the test strategy Statement
Acknowledgement The material on the duk patterns and testing strategies are taken from a presentation made by L. Williams at the North Carolina State University. Available at: http: //agile. csc. ncsu. edu/testing/Data. Flow. Testing. pdf Further Reading: An Introduction to data flow testing – Janvi Badlaney et al. , 2006 Available at: ftp: //ftp. ncsu. edu/pub/tech/2006/TR-2006 -22. pdf
Use of coverage measures Tor Stålhane
Model – 1 We will use the following notation: • c: a coverage measure • r(c): reliability • 1 – r(c): failure rate • r(c) = 1 – k*exp(-b*c) Thus, we also have that ln[1 – r(c)] = ln(k) – b*c
Model – 2 The equation ln[1 – r(c)] = ln(k) – b*c is of the same type as Y = α*X + β. We can thus use linear regression to estimate the parameters k and b by doing as follows: 1. Use linear regression to estimate α and β 2. We then have – k = exp(α) – b=-β
Coverage measures considered We have studied the following coverage measures: • Statement coverage: percentage of statements executed. • Branch coverage: percentage of branches executed • LCSAJ Linear Code Sequence And Jump
Statement coverage
Graph summary
Equation summary Statements: -ln(F) = 6. 5 + 6. 4 Cstatement, R 2(adj) = 85. 3 Branches: -ln(F) = 7. 5 + 6. 2 Cbranches, R 2(adj) = 82. 6 LCSAJ -ln(F) = 6. 5 + 6. 4 CLCSAJ, R 2(adj) = 77. 8
Usage patterns – 1 Not all parts of the code are used equally often. When it comes to reliability, we will get the greatest effect if we have a high coverage for the code that is used most often. This also explains why companies or user groups disagrees so much when discussing the reliability of a software product.
Usage patterns – 2 input domain X X X Input space A X X Corrected X
Usage patterns – 3 As long as we do not change our input space – usage pattern – we will experience no further errors. New user groups with new ways to use the system will experience new errors.
Usage patterns – 4 input domain Input space B X X X Input space A
Extended model – 1 We will use the following notation: • c: coverage measure • r(c): reliability • 1 – r(c): failure rate • r(c) = 1 – k*exp(-a*p*c) • p: the strength of the relationship between c and r. p will depend the coupling between coverage and faults. • a: scaling constant
Extended model – 2 Residual unreliability R(C) 1 Large p Small p 1 -k 0. 0 1. 0 C
Extended model - comments The following relation holds: ln[1 – r(c)] = ln(k) – a*p*c • Strong coupling between coverage and faults will increase the effect of test coverage on the reliability. • Weak coupling will create a residual gap for reliability that cannot be fixed by more testing, only by increasing the coupling factor p – thus changing the usage pattern.
Bishop’s coverage model – 1 Bishop’s model for predicting remaining errors is different from the models we have looked at earlier. It has a • Simpler relationship between number of remaining errors and coverage • More complex relationship between number of tests and achieved coverage
Bishop’s coverage model – 2 We will use f = P(executed code fails). Thus, the number of observed errors will depend on three factors • Whether the code – Is executed – C – Fails during execution – f • Coupling between coverage and faults - p N 0 – N(n) = F(f, C(n, p)) C(n) = 1 – 1/(1 + knp)
Bishop’s coverage model – 3 Based on the assumptions and expression previously presented , we find that If we use the expression on the previous slide to eliminate C(n) we get
A limit result It is possible to show that the following relation holds under a rather wide set of conditions: The initial number of defects – N 0 – must be estimated e. g. based on experience from earlier projects as number of defects per KLOC.
An example from telecom
- Andreas carlsson bye bye bye
- Unity iot
- Database selection criteria
- Ointment preparation by fusion method
- Criteria for selection of plc includes
- Crm decision matrix
- George makdisi md
- Modification
- Criteria for natural selection
- Requirements for natural selection
- Software evaluation criteria
- Indirect cost of expatriate failure
- Sv-hris
- What is project portfolio matrix
- Gradconnect
- Jiyo implemented a strategy
- Erp selection methods and criteria
- Motherboard selection criteria
- Path selection in software testing
- Dont ask why why why
- Balancing selection vs stabilizing selection
- Artificial selection vs natural selection
- K selected
- Natural selection vs artificial selection
- Artificial selection vs natural selection
- Example of sexual selection
- What is exponential growth in ecology
- Natural selection vs artificial selection
- Two way selection and multiway selection
- Multiway selection
- Mass selection and pure line selection
- Fakta om oden
- Roger dingledine nick mathewson
- Tor acara
- Kömür elmas
- Váltivarúak
- Traumdeutung tor
- Brandenburger tor alexanderplatz
- Fazy reedukacji posturalnej
- Tor morfologiczny
- Tor parda kasalliklari
- Roger dingledine nick mathewson