UNIT I SOFTWARE MEASUREMENTS Reference Books 1 Software

UNIT – I SOFTWARE MEASUREMENTS

§Reference Books 1) Software Metrics – A Rigorous & Practical Approach, 2 E – By: - Norman E Fenton & Shari Lawrence Pfleeger – Publication : - Thomson Learning. – (Chapter 1, 2, 3, 7, 8, 9, 10, 12 ) – Syllabus covered I, III Units 2) Software Quality By : - Garry Marliss and Ben-Menachem – Publication : - Thomson Learning. – (Chapter, 7, 8, 9, 19 ) – Syllabus covered IV Unit 3) Software Testing, 2 E By: - Ron Patton – Publication : - Pearson Education. – Syllabus covered V Unit

Learning outcomes 1 Able to Define Software Mesurement. 2 Able to Explain What is it and why do it? 3 Able to understand different representational theory and model. 4 Able to classify and determine what to measure 5 Able to apply framework 6 Benefits

1. Measurement in everyday life • Without measurement technology can’t function • Examples of Measurement – Economics: To determine price and pay increases – Radar System: To detect aircraft – Medical System: To diagnose disease – Weather forecasting system: Weather prediction – Shop (bill), Cloth Purchasing (height & size), Journey (distance, speed) etc. are the examples which requires measurement

Measurement helps to • Understand our world • Interact with the surroundings • Improve our lives.

1. 1. 1 What is Measurement ? • Measurement is the process by which numbers or symbols are assigned to attributes of entities in the real world, so as to describe them according to the clearly defined rules. • An Entity : - an object (person or object) an event (Journey or the testing process) § Attribute: - It is the feature or property of an entity. e. g. area or color of the room, cost of the journey,

1. 1. 2 Making Things Measurable • Measurement can be called as quantification • There are two types of quantification – Direct and Indirect quantification – Measurement is direct quantification • E. g. Height of a tree , length of software – Calculations are indirect quantification • E. g. Area of room = Length * Breadth

2. Measurement in software engineering • Software engineering describes the collection of techniques that apply an engineering approach to the construction and support of software products. 1. Managing 5. Modeling 2. Costing 6. Analyzing 3. Planning 7. Implementing 4. Designing 8. Testing and maintaining § As software engineering focuses on implementing the software in controlled and scientific way. To do this, all the above activities must be understood then we can control them and further we can improve.

1. 2. 1 Neglect of Measurement in Software Engg Ø It is difficult to imagine electrical, mechanical and civil engg without a central role of measurement. Neither effective nor practical ØFor most development projects: 1. We fail to set measurable targets for our software products. 2. We fail to understand quantify the cost of software products. 3. We do not quantify or predict the quality of the product we produce. 4. We can’t find out the improvements in our product development.

1. 2. 2 Objectives for Software Measurement • Measurement is needed at least for assessing the status of your projects, products and resources. • You cannot control what you cannot measures. v. Manager: 1. What does each process cost? – Measure time and effort 2. How productive is staff? – time it takes 3. How good is the code being developed? – carefully recording faults, failure & changes as they occur, we can measure quality. 4. Will user be satisfied? - functionality 5. How can we improve? – time for development, effect on quality & productivity. v Engineer: 1. Are requirement testable ? – analyze satisfaction must be expressed in a measurable, Objective way. 2. How we found all fault? – whether inspection testing effective. 3. Have we met our product or process goal? - measure characteristic 4. What will happen in future? – measure attributes of existing product

1. 2. 3 Measurement for understanding, control & improvements § Measurement is imp for three basic activities: - Understand- What is happening during development and maintenance. - Control: What is happening on our projects. - Improve: process or products.

§ 1. 3 The Scope of Software Metrics : Quantitative measure of degree to which a system, component or process possesses a given attribute For ex. No. of errors found person hours expended 1. Cost and Effort Estimation : a. Boehm’s COCOMO model b. Putnam’s SLIM Model c. Albrecht’s function model.

2. Productivity Model and Measures :

3. Data Collection: • Quality of any measurement product is clearly depend on careful data collection. • It is require to be done in consistent and complete. • It must be planned and executed in careful and sensitive manner • It is easier to say but difficult to done specially in case of large projects.

4. Quality Models and Measures: Boehm’s Quality Model, MC Call Quality model. Criteria are easier to understand & measure than factors

5. Reliability Models: • It is most important factor of quality model • It indicates probability of failure free operation • Models available are a) Jelinski-Moranda model b) Little Wood model c) Little wood Verall Model d) NHPP Model

§ 1. 3 The Scope of Software Metrics : (Contd. . ) 6. Performance evaluation and models - Aspects of quality - Response time and completion rates - Focuses on the internal workings of system including efficiency and complexity of algorithms.

7. Structural and Complexity Metrics: • Structural attributes are used to predict which parts of the s/w system are likely to be less reliable, more difficult to test or required more maintenance • We measure structural attributes of representation of s/w which are available without need for execution. • Structural Measures are available

§ 1. 3 The Scope of Software Metrics : (Contd. . ) 8. Management by Metrics: • Measurement is most important aspect of s/w project management • Measurement based Charts and graphs help the developer to track the status of project. • Many organizations define a set of measurements & reporting methods so that project can be compared & contrasted.

9. Evaluation of methods and tools: • Organization uses new methods & tools to make product better. • Organization performs experiments, case studies & administrator surveys to check whether a method or tool is likely to make a positive difference in their particular situations.

10. Capability Maturity Assessment: • In the 1980 s, the US (SEI) proposed a capability maturity model to measure contractors ability to develop the quality software for US government. • With this model contractor need to answered 100 questions • 5 Level Framework

1. 4 REPRESENTATIONAL THEOTY OF MEASUREMENT 1. 4. 1 Empirical relations It has 3 types: 1. Unary Empirical Relation 2. Binary Empirical Relation 3. Ternary Empirical Relation

Unary Relation

Binary Relation

Ternary Relation

REPRESENTATIONAL THEOTY OF MEASUREMENT (Contd …. . ) üIt may be binary, but need not be binary üIt may also unary such as “is tall ” üBehavior of measure in no. system to be same as corresponding element in real world üMapping to preserve relation i. e. rule of representation condition • We normally understand things by comparing them instead of assigning them numbers. • Avinash is tall - ‘is tall’ is the unary relation • Avinash is taller than Sushant. – Taller than is the binary relation

REPRESENTATIONAL THEOTY OF MEASUREMENT (Contd …. . ) 1. 4. 2 The Rules of the Mapping • The real world is the domain of mapping and mathematical world is the range. • When we map the attributes from real world to to a mathematical system, we have many choices for the mapping and the range. We can use real number, integers or even a set of nonnumeric symbol. – E. g. To measure person height, it may be in inches, centimeter or in feet. we have different mapping depending upon the conventions we adopt. - E. g. we need to specify may or may not allow shoes to be worn or people standing or sitting.

REPRESENTATIONAL THEOTY OF MEASUREMENT (Contd …. . ) 1. 4. 3 The representation condition of measurement • The representation condition states that a measurement mapping M must map the entities in to numbers and empirical relations into the numerical relations in such a way that the empirical relations are preserve and are preserved by the numerical relations.

REPRESENTATIONAL THEOTY OF MEASUREMENT (Contd …. . ) • A is taller than B if M(A) > M(B). • This statement implies that – When ever A is taller than B then M(A) must be bigger number that M(B). • We can map A to higher number than B only if A is taller than B.

1. 5 Measurement and Models § Model is an abstraction of reality allowing us to strip (left) away from details and view a particular concept or entity from a particular perspective. § Models come in many different forms: as equation, mappings, diagram etc. § These shows how component parts relate to one another, so that we can examine and understand these relationships and make judgments about them.

1. 5. 1 Defining attributes When measuring, there is always danger that we focus too much on the formal mathematical system and not enough on the empirical one. We hurry to create mapping & then manipulate numbers, without carefully examining relationship among entities & their attributes in real world. Following example shows how we can generate numbers without considering real meaning.

1. 5 Measurement and Models (Contd. . ) 1. 5. 2 Direct and indirect measurement § Direct measurement of an attribute of an entity involves no other attribute or entity. Eg. Length of physical object § Indirect measurement : Eg. Density of physical object = mass/volume § Direct Measures in SE : i) Length of source code, ii) Duration of testing process iii) Number of Defects discovered iv) Time a programmer spends § Indirect Measure in SE:

REPRESENTATIONAL THEOTY OF MEASUREMENT (Contd …. . ) q. Measurement for prediction üMeasure to assess what exist now or what happened in past üTo predict how much development will cost or how much time or effort will require üTo allocate appropriate resources to project üPrediction system consist of a mathematical model together with a set of Prediction procedure for determining unknown parameter & interpreting result

1. 5 Measurement and Models (Contd. . ) 1. 5. 3 Measurement for prediction : § In many circumstances, we would like to predict an attribute of some entity that does not exist. § For eg: control software for an aircraft, power plant or x-ray machine. § Simply waiting for the project to end and then measuring cost and schedule attributes, are clearly not acceptable. § The distinction between measurement for assessment and prediction is not always clear-cut. For eg. Globe as model and prediction procedure how to use a model. § We can replace “gambling” with “software prediction”. § Quality of prediction is based on several other assumptions, including the notion that the future will be like the past.

1. 6 MEASUREMENT SCALE & SCALE TYPE § Measurement mapping, M, together with the empirical and numerical relation system is called as a measurement scale. i. Nominal Scale: ü Characteristics: üConsists only different classes; there is no notion of ordering among the classes. üDistinct numbering or symbolic representation of the classes is an acceptable measure. The classes are not ordered; even if the classes are numbered from 1 to n for identification.

1. 6 MEASUREMENT SCALE & SCALE TYPE ii. Ordinal Scale: ü It provides more information than nominal scale & also preserves the ordering among the classes. üCharacteristics: üEmpirical relation system consists of classes that are ordered with respect to the attribute. üAny mapping that preserve the ordering is acceptable. üThe numbers represent ranking only, other arithmetic operation have no meaning.

1. 6 MEASUREMENT SCALE & SCALE TYPE III. Interval Scale ü Interval Scale are numeric scale in which we know not only the order but also the exact differences between values. Eg. Celsius and temp. üCharacteristics: üIt preserves the ordering. Also preserves the differences but not ration. üAddition and subtraction acceptable on the interval scale, but not multiplication and division. üTrue Zero is not available Example of Interval Scale

1. 6 MEASUREMENT SCALE & SCALE TYPE IV. Ratio Scale: ü Some times we need more analysis than either nominal, ordinal or interval scale. üCharacteristics: üIt is measurement mapping that preserves ordering, the size of intervals between entities and ratio between entities. üThere is zero element, representing total lack (absence) of the attribute. üThe measurement mapping must start at zero and increase at equal intervals, known as units. üAll arithmetic can be meaningfully applied to the classes in the range of the mapping. üExample: Temperature on Kelvin scale

1. 6 MEASUREMENT SCALE & SCALE TYPE V. Absolute Scale: ü Only one way measurement can be made, so M & M’ must be equal. ü Properties: ü The measurement for an absolute scale is made simply by counting the number of elements in the entity set. ü The attribute always takes the form “number of occurrences of x in the entity” ü There is only one possible measurement mapping, namely the actual count. ü All arithmetic analysis of the resulting count is meaningful. Eg: failures observed during integration testing – counting no of failure observed.

Summary In summary, nominal variables are used to “name, ” or label a series of values. Ordinal scales provide good information about the order of choices, such as in a customer satisfaction survey. Interval scales give us the order of values + the ability to quantify the difference between each one. Finally, Ratio scales give us the ultimate–order, interval values, plus the ability to calculate ratios since a “true zero” can be defined.

1. 6 MEASUREMENT SCALE & SCALE TYPE

1. 7 MEANINGFULLNESS IN MEASUREMENT § Understanding scale type enables us to determine when statements about measurement make sense. § Scale type tell about limitation on kind of mathematical manipulation that can be performed § Statement’s meaningfulness involving measurement is quite distinct from the notion of the statement’s truth. eg: The president of the United States is 125 years old.

1. 7 MEANINGFULLNESS IN MEASUREMENT Eg 1: “Person A twice as tall as Person B” - Statement implies ratio scale. § Statement is meaningful because no matter which measure of height we use (inch, feet), truth or falsity of statement remain constant M(fred)= 2 M (jane) Eg 2: The temperature in Shirpur today is twice that in Dhule. - Statement implies ratio scale but is not meaningful, because we measure temp in two scale – Celsius and Fahrenheit.

1. 7 MEANINGFULLNESS IN MEASUREMENT (Contd. . ) Eg 3: Difference in temperature between Shirpur & Dhule today is twice what it was yesterday. - For E. g. Yesterdays temp were 35 C in Shirpur & 25 C in Dhule (difference of 10), While today it is 40 C in Shirpur & 20 C in Dhule(difference of 20), . If we transform this temp to Fahrenheit scale, then Yesterdays temp were 95 F in Shirpur & 77 F in Dhule (difference of 18), ; whereas Todays temp is 104 F in Shirpur & 68 F in Dhule (difference of 36).

1. 7 MEANINGFULLNESS IN MEASUREMENT (Contd. . ) Eg 4: Failure x is twice as critical as failure y. - This Statement is not meaningful, if we have only ordinal scale for failure criticality.

1. 7 MEANINGFULLNESS IN MEASUREMENT (Contd. . ) 1. 7. 1 Statistical operation on measures üScale type affect type of operation & analysis on data üStatistical analysis uses arithmetic operation +, -, /, * ü 2 basic measure to capture information 1) Measure of central tendency 2) Measure of dispersion 1. Central tendency: - called average, tells about middle of set 2. Dispersion: - about how far data point stray from middle

1. 7 MEANINGFULLNESS IN MEASUREMENT (Contd. . ) 1. 7. 2 Objectives and subjective measures § Keep measurement objective and different people should get the same measures, this consistency of measurement is very important. § Although no measurement is truly objective there is always some degree of subjectivity about entity and attributes. § It is important to recognize that subjectivity measurements can be useful. As long as we understand their impression. § Subjective measures shows us that we may have problems with our interface requirements.

1. 7 MEANINGFULLNESS IN MEASUREMENT (Contd. . ) 1. 7. 2 Objectives and subjective measures § Objective measurement is based on how well people perform a task, irrespective of what they experience while performing the task. Subjective measurement on the other hand refers to measures that have to do with what people say they actually experience. § Subjective measures are very good at allowing a supervisor to exercise judgment about an employee's performance in complicated systems.

1. 7 MEANINGFULLNESS IN MEASUREMENT (Contd. . ) 1. 7. 2 Objectives and subjective measures § Objective measures are those that involves an impartial measurement, that is, without bias or prejudice. Subjective measures are influenced by the observers personal judgment of how the skill was performed. These measures are often criticized & analyzed as they are open to interpretation and opinion.

1. 7 MEANINGFULLNESS IN MEASUREMENT (Contd. . ) 1. 7. 3 Measurement in extended number systems § In many situation we cannot measure an attribute directly. Instead, we must measure it in terms of the more easily understood component attributes of which it is composed.

1. 7 MEANINGFULLNESS IN MEASUREMENT (Contd. . ) § A strict weak order has 2 -property üIt is asymmetric & transitive üAsymmetric means , if pair(x, y) is in relation then (y, x) is not in relation üRelation is negative transitive if (x, y) is in relation then for every z, either (x, z) or (z, y) is in relation

1. 7 MEANINGFULLNESS IN MEASUREMENT (Contd. . ) 1. 7. 4 Indirect measurement and meaningfulness § When we are measuring a complex attribute in terms of simpler sub-attributes, we are measuring indirectly.

1. 8 Classifying Software Measures In software there are three such classes: i. Process are collection of software related activities. ii. Products are any artifacts, deliverables or documents that result from a process activity. iii. Resources are entities required by a process activity. Within each class of entity, we distinguish between internal and external attributes § Internal attributes: can be measured by examining the product, process or resources on its own, separate from its behavior. For. Eg. Size, Structure, Complexity etc. § External attributes: Here the behavior of the process, product or resource is important rather than the entity itself. § For Eg. Quality, Productivity, Reliability, Usability etc.

1. 8 Classifying Software Measures (Contd. . )

1. 8 Classifying Software Measures (Contd. . ) 1. 8. 1 Processes: § We want to know how long it takes for a process to complete, cost, effective or efficient and compare with other processes. § These measure include § The duration , effort associated and no. of incidents of a specified type arising during the process or one of its activities. § Cost is not the only process measure we can examine. Controllability, observability and stability are also important in managing large project.

1. 8 Classifying Software Measures (Contd. . ) 1. 8. 2 Products: a. External product attributes: ü Since an external product attribute depends on both product behavior and environment, each attribute measure should take these characteristics into account. ü Usability, integrity, efficiency, testability, reusability, portability and interoperability are other external attributes that we can measure. b. Internal product attributes: ü Internal product attributes are sometimes easy to measure. Eg: Determine size of product by measuring the number of pages. ü other internal product difficult to measure, because opinions differ

1. 8 Classifying Software Measures (Contd. . ) 1. 8. 2 Products: c. The importance of internal attributes: ügood internal structure leads to good external quality D. Internal attributes and quality control and assurance: üA major reason that developers want to use internal attributes to predict external ones is the need to monitor and control the products during development.

1. 8 Classifying Software Measures (Contd. . ) 1. 8. 2 Products: D. Validating composite measures ü “Quality” is frequently used by software engineers to describe an internal attribute of design or code. However, “quality” is multidimensional; it does not reflect a single aspect of particular product. ü For eg: economists want to control the economy and make a country more productive.

1. 8 Classifying Software Measures (Contd. . ) 1. 8. 3 Resources: ü We measure resources to determine their magnitude, their cost and their quality. ü Poor quality software, too few people or people with wrong skills. ü Cost is often measure all types of resources, so that managers can see how the cost of the inputs affects the cost of outputs– Eg. Large investment in CASE tools yielding benefits in terms of more productive staff. ü Productivity is always important, and managers are enthusiastic not only to measure it but also to understand how to improve it. üBusiness or markets are judged by comparing what goes in with what comes out. üThere are many other staff attributes that we can measure, whose values may have an influence on the process or product. üWe can also classify and analyze tools and methods.

1. 9 APPLYING THE FRAMEWORK I. Cost & effort estimation II. Productivity model & measure III. Data collection IV. Quality model & measure V. Reliability model: VI. Performance evaluation & model: VII. Structural complexity & metrics: VIII. Management by metric: IX. evaluation of effort & tools X. Capability maturity assessment: -

1. 10 S/W MEASUREMENT VALIDATION • Validation approach depend on distinguishing measurement from prediction • 2 -types of measuring 1. Measures or Measurement System: Used to assess existing entity by numerically characterizing one or more of its attributes 2. Prediction system: - used to predict some attribute of future entity involving a mathematical model with associated prediction system 1. 10. 1 Validating Prediction System ü It is process of establishing accuracy of prediction system by empirical mean i. e. by comparing model performance with known data in given environment. ü It involves experimentation & hypothesis testing rather than mathematical proof.

1. 10 S/W MEASUREMENT VALIDATION (Contd. . ) § Deterministic Prediction System: - always get the same output for given i/p § Stochastic Prediction System: -o/p for given i/p will vary probabilistically üPrediction system for cost estimation, effort estimation, schedule estimation & reliability is very stochastic 1. 10. 2 Validating Measure: - üIt is the process of ensuring that the measure is proper numerical characterization of the claimed attribute by showing that representation condition satisfied

Determining What to Measure § GQM Approach: § It provides framework for involving 3 steps:

Determining What to Measure

Measurement and Process Improvement:

§ It defines 5 levels of process maturity: § Level 1 measurement provides baseline for comparison to improve process & product § Level 2 measurement focus on project management § Level 3 measures the intermediate & final product produced during development § Level 4 captures characteristics of development process to control activities of individual process § Level 5 measurement allows to provide feedback for dynamically changing the processes during development.

§ Level 1 (ad hoc): § Characterized by ad hoc approach to s/w development process § i/p- not well defined, o/p- expected and transitions from i/p to o/p is undefined & uncontrollable § At this level it is difficult to write down or describe the overall process

§ Level 2 (Repeatable): § Identifies i/p and o/p of the process, constraints & resources used to produce final product § Process is repeatable § Proper i/p produces proper o/p but there is no visibility to show o/p is produce

§ Level 3 (Defined): § Provide visibility to o/p produce § Intermediate activities are defined and their i/p and o/p are understood well

§ Level 4 (Managed): § It adds management oversights to defined process § We can use feedback for early project activities for current as well as later project activities.

§ Level 5 (optimizing): § Measures from activities are used to improve the process by removing and adding the process activities and changing the process structure dynamically in response to measurement feedback.