The SQUID method Stefano De Panfilis Engineering Ingegneria

The SQUID method Stefano De Panfilis Engineering Ingegneria Informatica S. p. A. Department “Research and Development Laboratory” “European Projects” Manager SQUAD Workshop 5 June, 2001 - Budapest

SQUID Approach SQUID o Provides support for product quality specification, control and evaluation o Measurement-based o Two types of measure • Internal measures relating to software and its development process • External measures relating to operational behaviour and the support process o Problem • Software engineering research can’t show a functional relationship between internal measures and external measures o SQUID View • Controlling internal quality is likely to improve product quality • Collecting and analysing data will confirm whether any relationships exist in your own organisation SQUAD Workshop - 5 June 2001 - Budapest 2

Four steps to quality SQUID o Specify product quality requirements objectively: • Target values of measurable properties o Assess feasibility of quality requirements before starting • Address feasibility problems as necessary o Monitor progress towards achieving quality requirements • Set Targets for measures arising during product development • Monitor and control these measures o Evaluate final product quality by comparing target values with actuals SQUAD Workshop - 5 June 2001 - Budapest 3

SQUID Quality Process SQUAD Workshop - 5 June 2001 - Budapest 4

SQUID Approach SQUID o SQUID Approach requires • Integration of different types of measure (internal & external) • Integration of different models (quality & development models) o Problem • There are many different quality models – Different quality models lead to different external measures • Different organisations will have use different development models – Different development methods lead to different internal measures o SQUID toolset supports configuration • Definition of organisation specific models and measures • Analysis methods independent of specific models and measures SQUAD Workshop - 5 June 2001 - Budapest 5

Models Integration SQUID Product View ? Quality View Data View SQUAD Workshop - 5 June 2001 - Budapest 6

Measurement SQUID Example: “Braking distance of a car” Attribute: Braking distance Evaluation Object: Car SQUAD Workshop - 5 June 2001 - Budapest 7

Measurement SQUID “Braking distance (in meters) from the speed of 80 km/hour on a plane dry surface road with less than 2 m/sec. head wind and 200 kg total weight of passengers and luggage of a car. ” Attribute: Braking distance (in meters) from the speed of 80 km/hour on a plane dry surface road with less than 2 m/sec. head wind and 200 kg total weight of passengers and luggage. Evaluation Object: Car SQUAD Workshop - 5 June 2001 - Budapest 8

Measurement SQUID “ Braking distance (in meters) from the speed of 80 km/hour on a plane dry surface road with less than 2 m/sec. head wind and 200 kg total weight of passengers and luggage of a car. ” Attribute: Braking efficiency Evaluation Object: Car Unit: Meters Counting Rule: Drive with 80 km/hour on a plane dry surface road with less than 2 m/sec head wind. Carry 200 kg as the total weight of passengers and luggage. Brake as sharply as possible when passing a line on the road. Measure the distance from the line to the stopped evaluation object. SQUAD Workshop - 5 June 2001 - Budapest 9

Measurement SQUID “Braking time of a car” “Braking heat of a car” Attribute: Braking efficiency Evaluation Object: Car Unit: Seconds Counting Rule: Drive with 80 km/hour on a plane dry surface road with less than 2 m/sec head wind. Carry 200 kg as the total weight of passengers and luggage. Brake as sharply as possible when passing a line on the road. Measure the time from passing the line and until the evaluation object stops. Attribute: Braking efficiency Evaluation Object: Car Unit: Degrees centigrade Counting Rule: Drive with 80 km/hour on a plane dry surface road with less than 2 m/sec head wind. Carry 200 kg as the total weight of passengers and luggage. Brake as sharply as possible when passing a line on the road. Wait 30 sec. after the evaluation object has stopped and then measure the temperature of the brakes. SQUAD Workshop - 5 June 2001 - Budapest 10

Measurement SQUID Measurement definitions: Attribute: A measurable property of an object. Evaluation Object: The object that possesses the property to be measured. Unit: The way to quantify the measurement. Counting Rule: Conditions and procedures under which the measurement is obtained. SQUAD Workshop - 5 June 2001 - Budapest 11

Measurement SQUID SQUAD Workshop - 5 June 2001 - Budapest 12

Integration SQUID Unit / Scale Quality View Product View Evaluation Object Measurement Attribute Counting Rule Value Data View SQUAD Workshop - 5 June 2001 - Budapest 13

SQUID Quality Process SQUAD Workshop - 5 June 2001 - Budapest 14

Quality Specification SQUID Quality Requirements Unit / Scale Product View Evaluation Object Quality View Measurement Attribute Counting Rule Value Data View SQUAD Workshop - 5 June 2001 - Budapest 15

SQUID Feasibility Analysis o Have we any examples which match against the current project? o Use Similarity Algorithm to sift the known projects on available measures SQUAD Workshop - 5 June 2001 - Budapest 16

Quality Planning SQUID o After quality specification agreed o Development process must be defined • Assumed to be one of several company standard methods o Set targets for internal measures • SQUID assumes that controlling internal measures will deliver required quality • To monitor progress internal measures should be associated with different development activities and intermediate deliverables • Target values can be based on past performance: – What internal values where observed on project that achieved similar requirements SQUAD Workshop - 5 June 2001 - Budapest 17

SQUID Quality Planning: Setting Targets o Use previous project data • Actual Values of similar past projects • Similar past projects: – same development process – similar targets o Different target setting strategies • Average of actuals for all similar projects • Actual values for single most similar project • Expert opinion adjustments – Allowing for process improvement actions SQUAD Workshop - 5 June 2001 - Budapest 18

Quality Control SQUID o Compares actual values with target values during software production • To assess current status o Compares estimates of future quality measures with target values • To re-assess feasibility o Identifies “anomalies” • Unusual components (modules, documents, programs) • Possible problem components o Problems with Software • Many causes --Many corrective actions • Targets themselves may be inappropriate SQUAD Workshop - 5 June 2001 - Budapest 19

Analysis tools SQUID o Provide regression facilities o Provide facilities to focus the data • Distributions – Boxplots – Histograms • Relationships – Scatterplots o Different from a statistical package, these understand the data and which analyses are legitimate SQUAD Workshop - 5 June 2001 - Budapest 20

Anomaly Detection SQUID o Monitoring against targets • Checks overall project progress o Still need to avoid “Hot Spots” • Analysis shows small percentage of extreme modules cause most of the problems and delays • Traditional quality control based on normal distribution won’t work, although general principles apply. SQUAD Workshop - 5 June 2001 - Budapest 21

Univariate Anomaly Detection SQUID o Boxplots look for outliers beyond the upper and lower fourths Lower Tail Lower Fourth Median Upper Fourth Upper Tail Outliers X Anomaly detection needs different X Cut off point Anomalies X SQUAD Workshop - 5 June 2001 - Budapest X X 22

SQUID Bivariate Anomaly Detection o Based on a density detection algorithm o Equivalent to visual application of a grid to a scatterplot SQUAD Workshop - 5 June 2001 - Budapest 23

Anomaly Detection SQUID o A single Univariate or Bivariate analysis is not sufficient for software o Some items are only detected by a combination of analyses o Groups allow us to look at each stage for linked problems • High Level Design • Coding • Etc SQUAD Workshop - 5 June 2001 - Budapest 24

SQUID Quality Evaluation o At end of product development o Compares quality targets with actuals o Accumulate results into overall assessment SQUAD Workshop - 5 June 2001 - Budapest 25

SQUID Tooset architecture SQUID Configuration Quality Specification Quality Evaluation Data Base Quality Control Quality Planning Data Collection SQUAD Workshop - 5 June 2001 - Budapest 26

SQUID Overview