Information Software Systems SOFTWARE DEVELOPMENT PROCESS METHODOLOGY Ludmila

Information Software Systems SOFTWARE DEVELOPMENT PROCESS METHODOLOGY Ludmila Matveeva ISS SQE Leader Information Software Systems 18 May 2007 ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007

Information Software Systems TOPICS COVERED q ISS Process Brief Overview q Software (SW) Process Improvement q SW Process Quality Models and Standards q CMMI Overview q The Benefits from SW Process Deployment q How to Get Things Started ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 2

Information Software Systems ABOUT US q ISS – Information Software Systems Ltd. , was founded in 2000 as a company for Motorola outsourced engineering projects in Ukraine. q ISS partners with different Motorola and Freescale (former Motorola Semiconductor sector) affiliates, and works on various projects for these affiliates. q In July 2003 ISS became the first software company in Ukraine formally assessed for CMM, joining the ranks of 1, 500 companies from 60 countries that meet this highest Quality Standard. ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 3

Information Software Systems ISS SOFTWARE DEVELOPMENT OVERVIEW 2007 CMMI Appraisal SCAMPI Class C 2006 Tailoring CMMI process to the Customer’s Agile process 2005 Turned to CMMI 2004 Implementation of CMM Level 4 and 5 practices 2003 Institutionalization of of ISS SW SW Process ISS CMM level 33 2002 ISS SW Process Establishment 2001 Planning foundations and basic metrics gathering foundations ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 4

Information Software Systems ISS SW PROCESS BEST PRACTICES q ISS Software Engineering Handbook q SEPG and regular Process Improvement q Tailoring of the project’ Process according the project needs q Periodic Process Gap analysis and self-assessments q Everyone participates in Process activities q Use of quantitative methods n ISS concept of process management is founded on the principles of statistical process control 1. 2. 3. 4. 5. q Establishing admissible levels of processes variability Controlled process Stable process Predictable results Prepare achievable plans, meeting cost and scheduling commitments & Deliver required product functionality and quality Defect prevention practices ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 5

Information Software Systems ISS SW PROCESS BEST PRACTICES (cont. ) q q q Quality and Test Groups are independent Defect Tracking System established. DTS is the tool for tracking of different process and project activities Audit Program n n q Metrics Program n n q q Quality Assurance activities planned on organizational and project levels, audits conducted on regular basis Audits results stored in Audit Database Projects and process metrics collected analysed Historical Database maintained Quality engineer participates in: project reviews, postmortems, and causal analysis Checklists used across all process areas ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 6

Information Software Systems ISS SW PROCESS BEST PRACTICES (cont. ) q Training Program Developed n n q Training activities planned on organizational and project levels records stored in Training Database Champion identified Standards and Procedures Defined Training Program Implemented n n n Local training regularly conducted by local instructors. Training materials reviewed Training feedback collected analyzed Permanent Process training and consultations are provided by Motorola specialists ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 7

Information Software Systems SOFTWARE PROCESS DEFINITION. PROCESS IMPROVEMENT p A Definition of SW Process is the basis of industrial software production p SW Process Improvement foundation – quality of product is governed by quality of process; quality cannot be added later to the product. n Most of the SW products are extremely complex systems. To build quality into such products, engineering discipline is required p Understanding existing processes and introducing process changes to improve product quality, reduce costs or accelerate schedules. n Surviving in the competitive business world that is constantly changing requires constant adaptation and learning ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 8

Information Software Systems PROCESS IMPROVEMENT q The process improvement cycle involves process measurement, process analysis and process change q Process measurement p p Attributes of the current process are measured to quantify its status and performance. These are a baseline for assessing improvements and information for decision making. Process measurement should be used to answer specific process questions, based on organizational improvement goals (Goal-Question-Metric Paradigm) q Process analysis p The current process is assessed and bottlenecks and weaknesses are identified § Study existing processes to understand their activities and the relationships between parts of the process and to compare them with other processes p Process modeling is the documentation of a process which records the tasks, the roles and the entities used § Produce an abstract model of the process and analyze it to discover process problems § This involves discussing process activities with stakeholders and discovering problems and possible process changes q Process change p p p Changes to the process that have been identified during the analysis are introduced: plan, justify, implement, and train Changes should be driven by measurable goals The impact and benefits gained should be assessed and compared to the costs of changes made to the processes ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 9

Information Software Systems A PROCESS MODEL q What is a Process Model? n n A model is a structured collection of elements that describes characteristics of effective process A model is used § to help set process improvement objectives and priorities § as a guide for improvement of project and organizational processes § with an assessment method to diagnose the state of improvement efforts q Why is a Model Important? n A model provides a starting place, a framework for prioritizing actions, a common language and a shared vision, the benefit of a community’s prior experiences § It is always best to start process analysis with an existing model. People then may extend and change this q What are the risks of model-based improvement? n n n Models are not comprehensive Interpretation and tailoring must be aligned to business objectives Judgment is necessary to use models correctly and with insight Potential for dysfunctional behavior to get a maturity level rating “All models are wrong; some are useful” ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 10

Information Software Systems SW PROCESS QUALITY MODELS AND STANDARDS p Many possible choices for improvement framework… n SEI’s Capability Maturity Models n ISO 9001 (Quality Management System) n ISO/IEC 15504 (SPICE) n …or ISO 12207 (Software Life Cycle Processes), Bootstrap (Europe), domain-specific model Trillium (Bell): Telecom domain… n … or improvement “philosophies”/approaches such as Six. Sigma, GQM (Goal-Question-Metrics), IDEAL model, SPC (Statistical Process Control), PDCA (Plan-Do-Check. Act) cycle… ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 11

Information Software Systems SW PROCESS IMPROVEMENT APPROACHES. Six. Sigma q q Six Sigma originally developed by Motorola to systematically improve processes by eliminating defects Bill Smith 1986 Six. Sigma is: n Quality management philosophy p n Reduce variation in your business and make customer-focused, data driven decisions Quality improvement methodology p Methodology and tools to improve business processes § § p Pareto charts Histograms Cause-and-effect diagrams (Ishikawa) Run charts and Control charts, etc. Uses SPC methods, for ex. : § DMAIC – Define, Measure, Analyze, Improve, Control n Measurement of product and process quality p 3. 4 defects per million opportunities ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 12

Information Software Systems SW PROCESS IMPROVEMENT APPROACHES. GQM Paradigm q GQM paradigm aims to provide information needed for understanding, guiding, and changing the software processes of a software development project Basili & Weiss 1984 q Goals n What is the organization trying to achieve? The objective of process improvement is to satisfy these goals q Questions n Questions about areas of uncertainty related to the goals. You need process knowledge to derive these q Metrics n Measurements to be collected to answer the questions q GQM translates Goals into Measures by means of Questions and Indicators = GQ(I)M q GQ(I)M operates with: § § § G - Entities (Objects) QI - Attributes (Characteristics of Objects) M - Measures (Values of Attributes), Measurements (Rules to assign Values of Attributes) ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 13

Information Software Systems SW PROCESS IMPROVEMENT APPROACHES. IDEAL model q Two-dimensional view of the IDEAL model ISS Mc. Feeley 1996 Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 14

Information Software Systems SW PROCESS IMPROVEMENT APPROACHES. SPC q SPC (Statistical Process Control) measurement process Florac & Carleton 1999 ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 15

Information Software Systems SW PROCESS IMPROVEMENT APPROACHES. PDCA cycle p p PDCA (Plan-Do-Check-Act) cycle W. Edwards Deming 1986 Control Circle within the Total Quality Control (TQC) improvement model Ishikawa 1985 ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 16

Information Software Systems SW PROCESS QUALITY MODELS AND STANDARDS (cont. ) q SW Process best practices n Assessment-based approaches § SPICE § CMM(I) § Bootstrap: the European counterpart to CMM n SW Process standards § ISO 9000 series. ISO 9001(2000) provides a model for quality assurance in design, development, production, installation, and servicing. ISO 9000 -3(1997) provides guidelines for the application of ISO 9001 in SW development. § SWEBOK (international project promoted by IEEE Computer Society and the ACM) ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 17

Information Software Systems SW PROCESS QUALITY MODELS AND STANDARDS (cont. ) q Other Process frameworks… n n RUP (Rational Unified Process) – SW engineering process based on the best practices in modern SW development with an interactive knowledge base integrated with tools Agile methods: Agile Modeling, e. Xtreme Programming, Scrum, Feature Driven Development, Crystal methods… § § q Individuals and Interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan …and product quality standards n n ISO 9126 – standard for SW product quality IEEE Std. 1061: 1998 – standard for SW quality metrics methodology ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 18

Information Software Systems SW PROCESS QUALITY MODELS AND STANDARDS (cont. ) p Process used should depend on the type of product being developed n n p There is no uniformly applicable process which should be standardized within an organization n n p For large systems or long life-time products, management is usually the principal problem so a strictly managed process is needed; For smaller systems or short life-time products, more informality is possible High costs may be incurred if you force an inappropriate process on a development team; Inappropriate methods can also increase costs and lead to reduced quality A fundamental factor for successes is the capability to focus on the right processes in improvement lest resources be wasted for nothing ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 19

Information Software Systems SEI CMM(I) HISTORY p The CMMI framework is the current stage of work on process assessment and improvement that started at the Software Engineering Institute (SEI) in the 1980 s. p Software Engineering Institute (SEI) USA n n p Established in 1984 Sponsored by Do. D (Department of Defense) Under Carnegie Mellon University The SEI’s mission is to promote software technology transfer particularly to US defense contractors SEI has had a profound influence on process improvement n n Capability Maturity Model introduced in the early 1990 s. Revised maturity framework (CMMI) introduced in 2001. ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 20

Information Software Systems FOUNDATIONS OF CMMI Process ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 21

Information Software Systems FOUNDATIONS OF CMMI. WHAT IS CMMI? p A framework of the key process elements for system development n n p An integrated view of process improvement across multiple disciplines n n n p Structured collection of processes proven through experience The CMMI tells you what to do but not how to do it Set process improvement goals and priorities Provides guidance for quality processes Provides a yardstick for assessing current practices Based on concepts and approached pioneered by Crosby, Deming, Juran, Humphrey, et al. ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 22

Information Software Systems FOUNDATIONS OF CMMI (cont. ) p Two approaches to process improvement n Process capability – pertains to an individual process § Knowing the process capabilities of a collection of processes has implications for organizational maturity n Organizational maturity – pertains to a set of processes § Begins at a particular level of organizational maturity has process capability implications for multiple processes p Two representations exist: staged and continuous p CMMI support each approach with a representation n n p Process capability approach continuous representation Organizational maturity approach staged representation The type of representation to use is based on the purpose of the improvement task ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 23

Information Software Systems FOUNDATIONS OF CMMI (cont. ) Initial ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 24

Information Software Systems FOUNDATIONS OF CMMI (cont. ) ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 25

Information Software Systems THE CMMI MODEL COMPONENTS p Process areas n p Goals are descriptions of desirable organizational states. Each process area has associated goals. Practices n p 24 process areas that are relevant to process capability and improvement are identified. These are organized into 4 groups. Practices are ways of achieving a goal - however, they are advisory and other approaches to achieve the goal may be used. Process improvement in the CMMI model is based on reaching a set of goals related to good software engineering practice ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 26

Information Software Systems THE CMMI MODEL COMPONENTS (cont. ) Process areas ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 27

Information Software Systems CMMI APPRAISALS q Assessment requirements for CMMI define: § § § § q Responsibilities of the Sponsor and Leader Assessment Method Documentation Planning and Preparing for Assessment Data collection Data Consolidation and Validation Rating Reporting results SCAMPI (Standard CMMI-based Assessment Method for Process Improvement) principles include: § § § § Senior management sponsorship Focus on organization’s business goals Use of documented assessment method Use of a process reference model (a CMMI model) Confidentiality of interviewees Collaborative team approach Focus on follow-up improvement actions ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 28

Information Software Systems CMMI APPRAISALS (cont. ) q 3 SCAMPI classes: § class C – approach § class B –deployment § class A – institutionalization q Appraisal Team qualification ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 29

Information Software Systems THE BENEFITS OF PROCESS DEPLOYMENT p Typical benefits of software process improvement (SPI) include: n Higher quality (fewer defects) n Lower maintenance (less rework) n Higher productivity (lower development cost) n Faster cycle times (quick time-to-market) n Greater value (more product features) and variety (more product variations) n Higher customer satisfaction (more contracts) ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 30

Information Software Systems THE BENEFITS OF PROCESS DEPLOYMENT (cont. ) p SW Process Improvement results from Motorola GSG (now – Motorola Software Group): ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 31

Information Software Systems THE BENEFITS OF PROCESS DEPLOYMENT. ROI p ROI (Return On Investment) is the quantification of the benefits received or financial return of any given investment p ROI numbers ease decision making n How should managers estimate the costs and benefits of different software process improvement (SPI) methods in order to select SPI method with the greatest ROI from the alternatives? ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 32

Information Software Systems THE BENEFITS OF PROCESS DEPLOYMENT. ROI p Model Phillips J. L. (1997). Return on investment in training and performance improvement programs. Houston, TX: Gulf Publishing Company ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 33

Information Software Systems THE BENEFITS OF PROCESS DEPLOYMENT. ROI p Motorola reported total ROI 677% when raising their capability from SW-CMM level 2 to 5. n p The calculation is based on required SPI investment rework originating from defects, and differences in defects rates between the SW-CMM level 2 and level 5 projects Boeing Space Transportation Systems reported total ROI 775% measured after CMM and inspections implementation ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 34

Information Software Systems THE BENEFITS OF PROCESS DEPLOYMENT. Cost of SW Quality (COQ)/Cost of Poor Quality (COPQ) p COQ = (A + T + DP) + COPQ n n A-auditing efforts; T-testing efforts; DP-defect prevention activity efforts; COPQ- time for defects fixing COQ introduced by J. M. Juran in 1951 as a means of providing justification to management for investments in process improvements EXAMPLE p p Efforts spent for quality reduce COQ COPQ is increased exponentially during project lifecycle ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 35

Information Software Systems THE BENEFITS OF PROCESS DEPLOYMENT. Cost of SW Quality (COQ)/Cost of Poor Quality (COPQ) p Knox’s theoretical model for cost of software quality Stephen T. Knox 1993 ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 36

Information Software Systems THE BENEFITS OF PROCESS DEPLOYMENT (cont. ) p p Our Sigma (DKNCSS) value: from 4. 68 before CMM level 3 formal assessment to 5. 16 after it Our SEA: ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 37

Information Software Systems HOW TO GET THINGS STARTED p Basic steps to improve process capability: n Understand the current status of development process and practices (software process assessment) n Develop a vision of the desired process (improvement objectives) n Establish a list of required process improvement actions in order of priority (high level roadmap) n Produce a plan to accomplish the required actions (action plan) n Commit the resources to execute the plan (improvement schedule) n Start over at step 1(continuous loop) ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 38

Information Software Systems HOW TO GET THINGS STARTED (cont. ) p IDEAL (SPI Model) ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 39

Information Software Systems HOW TO GET THINGS STARTED (cont. ) p Quality Control (QC) vs Quality Assurance (QA) n p QA analyst identifies improvement opportunities n p Any initial process description or procedure must be seen as only the starting point for the real process engineering work. Only monitoring a process as it is used to perform real work can we see where its inefficiencies lie QA ensures that people are doing their work consistently n p Searches for patterns of inefficiencies, defects, or other problems QA engineers effective processes n p QC (for ex. , testing) is about detecting defects; the QA is about avoiding them This part of QA is the closest to QC, but it is still different in its focus. QC evaluates the product of actions; QA evaluates the actions themselves QA is an investment in improving our organization n n Start by using your data to find the most troublesome problems, either problems that keep repeating themselves, or those that are most costly QC indicates the level of quality that we produced on a project. When we supplement QC with QA, we improve the level of quality we will produce on all future projects ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 40

Information Software Systems The End Thank You For Attention ISS Modern SW Development Practices: Processes and Technologies (c) 2007 Information Software Systems 18 May 2007 41