BASICS OF MEASUREMENT MEASUREMENT IN EVERYDAY LIFE MEASUREMENT

BASICS OF MEASUREMENT

MEASUREMENT IN EVERYDAY LIFE

MEASUREMENT IN SOFTWARE ENGINEERING • Neglect of measurement in software engineering

OBJECTIVES FOR SOFTWARE MEASUREMENT

OBJECTIVES FOR SOFTWARE MEASUREMENT How measurement is useful for Managers • Below are examples of the kinds of information needed to understand control a software development project by Managers • What does each process cost • We can measure the time and effort involved in the various processes that comprise software production • We can identify the cost of requirements, specification, design , coding and testing • How productive is the staff? • We can measure the time it takes for staff to specify the system, design it, code it and test plans • We can determine how productive the staff is at each activity • How good is the code being developed? • By carefully recording faults , failures and changes as they occur, we can measure software quality, that enable us to compare different products

HOW MEASUREMENT IS USEFUL FOR MANAGERS • Will the user be satisfied with the product? • We can measure functionality by determining if all the requirements requested have actually been implemented properly • Customer will be happy with both functionality and performance • How can we improve? • We can measure the time it takes to perform each major development activity and calculate its effort on quality and productivity • Then we can weigh the costs and benefits of each practice to determine if the benefit is worth the cost • We can try several variations of a practice and measure the results to decide which is best

OBJECTIVES FOR SOFTWARE MEASUREMENT How measurement is useful for Engineers? • Below are examples of the kinds of information needed to understand control a software development project by Engineers • Are the requirements testable? • We can analyze each requirements to determine if its satisfaction is expressed in a measurable way • Eg: if a requirement states that a system must be reliable then the requirement can be replaced by one that states that the mean time to failure must be greater than 15 elapsed hours of CPU time • Have we found all the faults? • We can measure the number of faults in the specification, design, code and test plan • Using this information to the models of expected detection rates can decide whether inspections and testing have been effective and whether a product can be released

HOW MEASUREMENT IS USEFUL FOR ENGINEERS? • Have we met our product or process goals? • We can measure characteristics of the products and processes that tell us whether we have met standards, satisfied requirement, or met a process goal • What will happen in the future? • We can measure attributes of existing products and current processes to make predictions about future ones • Eg: predictions about future maintenance problems can be made from measures of structural properties of the design documents

SCOPE OF SOFTWARE METRICS

SCOPE OF SOFTWARE METRICS • Cost and effort estimation • Motivation – to predict the costs early in the development life cycle • Numerous empirical cost models have been developed • Eg: Boehm’s COCOMO model, Albrecht’s function points model • Productivity models and measure • There is a need to define measure and models for assessing staff productivity during different software processes

SCOPE OF SOFTWARE METRICS • Productivity models and measure • The below productivity model illustrate various components that contribute to overall productivity. It shows productivity as a function of value and cost • Many mangers make decisions based on the rate at which lines of code are being written person month of effort

SCOPE OF SOFTWARE METRICS • Data Collection • The quality of any measurement program is dependent on careful data collection • Metric data collection must be planned and executed in careful and sensitive manner • The data collected can be distilled into simple charts and graphs the show mangers the progress and problem of development

SCOPE OF SOFTWARE METRICS • Quality models and measures • Without an accompanying assessment of product quality, speed of the production is meaningless • This has made software engineers to develop models of quality.

SCOPE OF SOFTWARE METRICS • The diagram of consists of quality factors of product such as reliability, usability etc. Each quality factor is composed of lower level criteria such as structuredness and traceability • The criteria are easier to understand measure than the factors • The actual measures are proposed for the criteria • Reliability models • Most quality models include reliability as a component factor • But the need to predict and measure reliability itself has led to a separate specialization in reliability modelling and prediction

SCOPE OF SOFTWARE METRICS • Performance evaluation and models • Performance is another aspect of quality • Performance evaluation includes externally observable system performance characteristics such as response times and completion rates • Structural and complexity metrics • Desirable quality attributes like reliability and maintainability cannot be measured until some operational version of the code is available • It is needed to predict which part of the software system are likely to be less reliable, more difficult to test or require more maintenance than others even before the system is complete • Possibility of doing this by measuring structural attributes of representations of the software which are available in advance

SCOPE OF SOFTWARE METRICS • Management by metrics • Customers and developers rely on measurement based charts and graphs to help them decide if the project is on track • Many companies and organizations define a standard set of measurements and reporting methods, so that projects can be compared and contrasted • Evaluation of methods and tools • Many articles and books describe new methods and tools that may make your organization of project more productive and your products better and cheaper • But it is difficult to separate the claims from the reality • Capability maturity assessment • Capability maturity model to measure a contractor’s ability to develop quality software for the US government • This model is assessed many different attributes of development, including use of tools, standard practices and more

REPRESENTATIONAL THEORY OF MEASUREMENT

REPRESENTATIONAL THEORY OF MEASUREMENT • Empirical Relation

REPRESENTATIONAL THEORY OF MEASUREMENT Rules of Mapping

REPRESENTATIONAL THEORY OF MEASUREMENT The representation condition

REPRESENTATIONAL THEORY OF MEASUREMENT

MEASUREMENT AND MODELS • Model is an abstraction of reality, allowing us to strip away detail and view an entity or concept from particular perspective • Eg: Cost models permit us to examine only those project aspects that contribute to the project’s final cost • Models comes in different form • Equations • Mapping • Diagram • Representation condition requires every measure to be associate with a model of how the measure maps the entities and attributes in the real world to the elements of a numerical system • These models are essential in understanding not only how the measure is derived, but also how to interpret the behavior of the numerical elements when we return to the real world

MEASUREMENT AND MODELS • Defining attributes • There is always danger that we focus too much on the formal, mathematical system, not enough on the empirical one • We rush to create mappings and then manipulate numbers, without giving careful thought to the relationships among entities and their attributes in the real world

Defining Attribute • Wrong to define the intelligence of dogs in this way • Needed is a comprehensive set of characteristics of intelligence, appropriate to the entity

MEASUREMENT AND MODELS • Direct and Indirect measurement • Direct Measurement of an attribute of an entity involves no other attribute or entity • Eg : Length of the code • Duration of testing process • Number of defects • Time a programmer spends on a project

MEASUREMENT AND MODELS Indirect Measurement

MEASUREMENT AND MODELS • Measurement for Prediction • Case 1 • When we talk about measuring something, we usually mean that we wish to assess some entity that already exists • This measurement for assessment is very helpful in understanding what exists now or what has happened in the past • Case 2 • Many circumstances we would like to predict an attribute of some entity that does not exist • Eg: Building a software system that must be highly reliable (control software for aircraft, xray machine)

MEASUREMENT AND MODELS • Reliability is defined in terms of operational performance. Something we cant measure before the product is finished • To provide reliability indicators before the system is complete, we can build a model of the factors that affect reliability, and then predict the likely reliability based on our understanding of the system while it is still under development • Predication system • It consists of a mathematical model together with a set of prediction procedures for determining unknown parameters and interpreting results

MEASUREMENT SCALES • Nominal • Ordinal • Interval • Ratio • Absolute

WHY IS LEVEL OF MEASUREMENT IMPORTANT? • Helps you decide what statistical analysis is appropriate on the values that were assigned • Helps you decide how to interpret the data from that variable

EXAMPLE

MEASUREMENT SCALES - NOMINAL

NOMINAL EXAMPLE

• Diagnostic categories • Sex of the participant • Classification based on discrete characteristics (e. g. , hair color) • An arbitrary identification, including identification numbers that are arbitrary • Menu items selected • Any yes/no distinctions • Group affiliation (e. g. , Republican, Democrat, Boy Scout, etc. ) • Most forms of classification (species of animals or type of tree) • The town people live in • Location of damage in the brain • A person's name

MEASUREMENT SCALES -ORDINAL When attributes can be rank-ordered Distances between attributes do not have any meaning • For example, code Educational Attainment as 0=SSLC; 1=HSC; 2=Some College degree; 4=post college

ORDINAL EXAMPLE

How happy are you with the customer service? ” 1 - Very Unhappy 2 - Unhappy 3 - Neutral 4 - Unhappy 5 - Very Unhappy

INTERVAL MEASUREMENT • When distance between attributes has meaning, for example, temperature (in Fahrenheit) -- distance from 3040 is same as distance from 70 -80 • Note that ratios don’t make any sense -- 80 degrees is not twice as hot as 40 degrees (although the attribute values are).

RATIO MEASUREMENT • Has an absolute zero that is meaningful • It is an interval scale with the additional property that its zero position indicates the absence of the quantity being measured

RATIO MEASUREMENT • Has an absolute zero that is meaningful • Can construct a meaningful ratio (fraction), for example, number of clients in past six months • It is meaningful to say that “. . . we had twice as many clients in this period as we did in the previous six months.

EXAMPLE • time to complete a task • number of objects detected • number of responses given in a specified time period • number of errors made in a specified time period • weight of an object • size of an object • proportion of responses in a specified category

THE HIERARCHY OF LEVELS Nominal

THE HIERARCHY OF LEVELS Nominal Attributes are only named; weakest

THE HIERARCHY OF LEVELS Ordinal Nominal Attributes are only named; weakest

THE HIERARCHY OF LEVELS Ordinal Nominal Attributes can be ordered Attributes are only named; weakest

THE HIERARCHY OF LEVELS Interval Ordinal Nominal Attributes can be ordered Attributes are only named; weakest

THE HIERARCHY OF LEVELS Interval Ordinal Nominal Distance is meaningful Attributes can be ordered Attributes are only named; weakest

THE HIERARCHY OF LEVELS Ratio Interval Ordinal Nominal Distance is meaningful Attributes can be ordered Attributes are only named; weakest

THE HIERARCHY OF LEVELS Ratio Interval Ordinal Nominal Absolute zero Distance is meaningful Attributes can be ordered Attributes are only named; weakest

Example l l l Identify the following as nominal level, ordinal level, interval level, or ratio level data. Nominal 1. Flavors of frozen yogurt ________ Ratio 2. Amount of money in savings accounts________ 3. Students classified by their reading ability: Above average (50 -70), Below Interval average (<30), Normal (30 -50) ________ Ordinal 4. Letter grades on an English essay ________ Nominal 5. Religions ________ Ratio 6. Commuting times to work ______ Ratio 7. Ages (in years) of art students ________ 8. Ice cream flavor preference ________ Ordinal Interval 9. Years of important historical events ________ Ordinal 10. Instructors classified as: Easy, Difficult or Impossible ________

MEANINGFULNESS IN MEASUREMENT • After measurement is made , can we reduce meaningful statements about the entities being measured

CLASSIFYING SOFTWARE MEASURE • Software measurement activity is identifying the entities and attributes to measure • There are three entities in software • Processes • They are collection of software – related activities • Products • They are any artifacts, deliverable or documents that result from a process activity • Resources • They are entities required by a process activity

ENTITIES ATTRIBUTES • Internal Attributes of an entity • They can be measured only based on the entity • They can be measured directly • External Attributes of an entity • They can be measured only with respect to how the entity relate to its environment • They can not be measured directly

PRODUCT ENTITIES AND ATTRIBUTES

SAMPLE PRODUCT MEASURE

PROCESS ENTITIES AND ATTRIBUTES • Requirements • Time and effort • Cost effectiveness • Design • Time, effort, number of changes • Cost effectiveness

SAMPLE PROCESS MEASURE

RESOURCES ENTITIES AND ATTRIBUTES • Personnel • Age, cost • Experience , productivity • Hardware • Memory, speed, cost • Reliability, usability

SAMPLE RESOURCE MEASURE

DETERMINING WHAT TO MEASURE • A particular measurement is useful only if it helps you to understand the underlying process or one of its resultant products • Improvement of the process or products can occur only when the project has clearly defined goals for process and products • Goal – Question – Metric approach to process and metrics has proven to be a effective approach to selecting and implementing metrics

THE GOAL-QUESTION-METRIC PARADIGM • Many metric program begin by measuring what is convenient or easy to measure, rather than by measuring what is needed • A measurement program can be more successful if it is designed with the goals of the project in mind • The GQM approach provides a framework involving three steps

GQM

GQM

GQM

TEMPLATES FOR GOAL DEFINITION

SOFTWARE MEASUREMENT VALIDATION • Measures or measurement systems are used to assess an existing entity by numerically characterizing one or more of its attributes • Prediction systems are used to predict some attribute of a future entity, involving a mathematical model with associated prediction procedure

SOFTWARE MEASUREMENT – VALIDATING MEASURE • The measure is valid if it accurately characterizes the attribute it claims to measure P 1 and P 2 are program. P 2 length is combined length of P 1 and P 2

SOFTWARE MEASUREMENT – VALIDATING PREDICTION SYSTEM Validation of prediction system involves experimentation and hypothesis testing