EVMS Planning and Execution for Agile Projects Jim

EVMS Planning and Execution for Agile Projects Jim Duffy Raytheon Integrated Defense Systems 10/14/2015 Copyright © 2015, Raytheon Company. All Rights Reserved.

Presenter – Jim Duffy Jim is a Senior Manager at Raytheon, Integrated Defense Systems (IDS) in Sudbury, MA. He has 18 years software project management and EVMS experience. Since 2012, Jim has led the deployment of Agile methods on several large-scale projects, with special emphasis on designing and implementing the EVM methodology for Agile projects that has become the IDS standard. Jim created and teaches the IDS EVM for Agile Projects training course. Jim is currently the Agile Implementation Lead and Software Development CAM on a large-scale project with over 18 scrum teams. In addition, Jim works closely with DCMA on EVM topics, presenting EVM for Agile Projects to the Office of the Secretary Defense, Performance Assessment and Root Cause Analysis (OSD – PARCA) in Feb 2015. Finally, Jim is a member of the NDIA Agile Working Group. Jim received his Bachelors Degree in Computer Engineering from the University of Notre Dame. He holds certifications in Project Management (PMP), Raytheon Earned Value, Scrum Master and Product Owner. Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 2

Abstract This presentation will explore the natural synergy between the Agile development focus on incrementally delivering value and the earned value management (EVM) requirement to measure earned value and cost against a defined and configuration managed EVM baseline. Using practical examples and lessons learned from large-scale agile implementations, we will discuss the key elements of EVM planning and then move into EVM execution, showing how earned value is collected and reported. Agility is demonstrated through the disciplined use of the agile planning levels, allowing teams to adapt to changing circumstances on a sprint-by-sprint basis while implementing a fully-compliant EVM system. Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 3

Agenda § Agile Planning § § § § Agile Planning and Planning Levels Traditional / Agile Planning Comparison Agile Planning and the Product Backlog Budget Determination Release Planning and Rolling Wave Planning Inchstone Definition using Stories Agile Planning Summary § Agile Execution § Calculating Earned Value § Early or Late Story Completion § Adding or Removing Stories § § Lessons Learned Summary Agile Planning Checklist References and Contacts Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 4

EVMS Implementation for Agile Projects Planning Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 5

Agile Planning § Consistent with existing EVMS policies, procedures and guidelines § Value Driven: the value of the Capabilities/Features to the customer drives the order of work § Capability/Feature-Based rather than Activity-Based § Feature examples: shoot a missile, track a target, establish comm circuit § Activity examples: write requirements, update design, write code, run unit tests § Agile Planning Terms: § Capability: customer-required ability of the system that provides value to the warfighter § Feature: part of a capability which can be completed within an incremental release (e. g. 3 – 6 months) § Story: part of a Feature which can be complete within one iteration (2 -4 wks) Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 6

Agile Planning Levels Agile Planning levels related to Agile EVMS 1: Increasing Precision Planning Level Planning Frequency Planning Horizon Planning Precision Planning Artifact Product Planning Project startup; Project updates throughout Duration the project Capabilities Product Backlog; Product Roadmap Release Planning Each incremental Incremental release/rolling wave Release Features/ Stories Release Plan Iteration (Sprint) Planning Each iteration or sprint Weeks Stories/ Tasks Sprint Backlog Daily Planning Daily Day Tasks Updated Sprint Backlog A fifth planning level, Vision Planning, which occurs prior to Product Planning, is not included in this training. See reference 1 for more information on Vision Planning. 1 Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 7

Agile Planning: Product Planning System Product Backlog • Prioritized List of Features of value to the customer • Comprises the total technical scope of the delivered system • Includes uncorrected defects (technical debt) • Managed by Product Owner • Constitutes Product Owner’s direction to the scrum team • Serves as basis for System-Level Integration and Acceptance Testing • Is a list of Features, not development tasks SOW Defect Database (STRs / SCRs) System Product Backlog 1. Single-Object Tracking 2. Multi-Object Tracking 3. Missile I Engagement 4. Missile II Engagement Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 8

Agile Planning: System/Team Product Backlog & Product Roadmap System Product Backlog Single-Object Tracking (SOT) Multi-Object Tracking (MOT) Missile I Engagement Missile II Engagement Display SW Backlog SOT Displays Missile II Displays Comms Each System Backlog Feature is decomposed into SW Mission SW SW-specific Features using system-level analysis Backlog (e. g. threads or integration test cases) 1 SOT Radar Each SW Product Backlog consists of a prioritized list Comms Control of SW-specific Features. MOT Radar Comms Control 3 Missile I Radar SW Product Roadmaps Comms Control • Assign SW Backlog Features to Releases Missile II Radar • Use Feature estimates to balance effort across Comms Control releases 2 System Product Roadmap Assign System Backlog Features to Releases System Product Roadmap Release 1 Release 2 Single-Object Missile I Tracking Engagement Multi-Object Missile II Tracking Engagement 4 System Capabilities achieved through completion of the SWspecific Features comprising each System Capability Display SW Product Roadmap Comms 1 SW Product Roadmap Release 2 Mission A – F 1 1 SW Product A – Roadmap F 4 2 Release A Release –– F 2 F 1 1 AA – F 5 Release A – F 4 2 – F 2 Control A – F 5 SOTARadar Missile I Radar Control MOT Radar Control Missile II Radar Control Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 9

Agile Planning: Product Planning Budget Determination Technical Scope (by Release) Mission SW Product Roadmap Release 1 Release 2 SOT Radar Control Missile I Radar Control MOT Radar Control Missile II Radar Control Development Budget (by Release) Lines of Code Productivity Hours Rate BAC Release 1 12, 000 2. 00 6000 $23 $137 K Release 2 15, 000 2. 25 6667 $24 $160 K Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 10

Agile Planning: Release Planning Release Plan System Product Roadmap Release 1 Release 2 Single-Object Missile I Tracking Engagement Multi-Object Missile II Tracking Engagement Display Team Comms Release. Team 1 Mission A – F 1 Team Release 1 A Release –– F 2 F 1 1 A – F 2 Control SOTARadar MOT Radar Control System-Level Integration drives feature priority Display Team – Release 1 Plan Comms Team Release Plan 5 Sprint 6 Sprint 1 Sprint 2 Sprint 3 –Sprint 4 1 Sprint Mission Release 1 Sprint Plan Sprint 2 Sprint C - S 1 1 Sprint C - S 5 C -Team S 9 3 –CSprint - S 134 C - S 165 CSprint - S 206 Sprint S 1 1 CC S 5 2 CC S 9 3 CCSprint S 13 4 CCSprint S 16 5 CCSprint S 20 6 CC - -S 2 - -S 6 - -S 4 - -S 17 - -S 21 SOT S 2 Initiation S 6 CC- -S 8 S 4 closeout S 14 CCMOT S 17 Maintenance S 21 CC- -S 3 CC- -S 7 CC- -S 15 - -S 18 CC- -S 22 MOT S 3 Maintenance S 7 CC- S 12 -MOT S 8 Initiation C - S 15 CC- -S 19 S 18 Closeout C - S 22 CCSOT - -S 4 CC- -S 8 CStory - S 4 1 CStory - S 8 5 C - S 12 C - S 19 Story 8 Story 10 Story 2 Story 6 Story 12 Story 13 Story 7 Story 9 Story 11 Story 4 Confirm release scope features planned for the release at the system and team level The number of sprints is based on sprint length and release duration Establish milestone scope decompose release-level features to scope that can be completed within milestone duration. Balance milestone scope, duration and budget. Define & estimate user stories to the extent needed to define milestone scope. Near-team Stories are sized to complete in one sprint. Release Planning Provides Milestone Scope Definition for Rolling Wave Planning Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 11

Agile Planning: Rolling Wave Planning Release Planning Rolling Wave Planning • Conducted prior to rolling wave planning • Use milestones weight with percent complete (MWPC) earned value technique (EVT) • Establish Release Scope: features planned for the release Mission SW Release 1 Release Scope Single-Object Track Radar Control Multi-Object Track Radar Control • Establish Milestone Scope: features planned for each milestone. • Work Package period of performance aligns with sprint boundaries (beginning of 1 st sprint; end of last sprint). • Milestone scope is the features planned for the milestone • Work Package scope is the features planned for the release • Work packages and milestone budget determined from feature scope. Mission SW Release 1 Milestones Milestone 1 Single-Object Track Initiation Single-Object Track Maintenance Milestone 2 Single-Object Track Closeout Multi-Object Track Initiation Milestone 3 Multi-Object Track Maintenance Multi-Object Track Closeout Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 12

Agile Planning: Inchstone Definition § Inchstones are used MWPC EV reporting. § Raytheon EVMS Manual 10255 Milestone Weights with Percent Complete: … values for percent complete of the milestone is based on CAM assessment of work accomplished or subordinate detail. CAM controlled planning of the milestone into “inchstones” with defined values is one common method used for work-in-process assessments. Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 13

Agile Planning: Inchstones Definition Use of Story points weights is the “defined value” used to calculate milestone percent complete • Stories are used as inchstones for earned value reporting to reinforce value of working software over completion of engineering tasks • Inchstones are not IMS tasks and therefore have no budget, scope or schedule assigned to them • Story point weightings are used to calculate the earned value contribution of each completed story Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 14

Agile Planning: Summary EV Baseline (subject to BCR) • Work Package and milestone scope baselined at rolling wave; revised only with BCR • Work Package scope equals the features planned for the release • Milestone scope equals features planned for the milestone Supporting Data for Milestones (not subject to BCR) modified at the discretion of the team to achieve milestone scope • Stories are used as inchstones for earned value reporting to reinforce value of working software over completion of engineering tasks • Inchstones are not IMS tasks and therefore do not require budget, scope or schedule assigned to them • Story point weightings are used to calculate the earned value contribution of each completed story Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 15

EVMS Implementation for Agile Projects Execution Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 16

Agile Execution: Calculating Earned Value Status at end of Sprint 2: Half of story weight is earned when the story is started and half when the story is finished. Legend: NS = not started; S = started; F = finished Total story points earned is used to calculate milestone percent complete used for earned value reporting Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 17

Agile Execution: Starting Stories Early Status at end of Sprint 2: SOT Initiation and Maintenance Milestone Complete Second milestone starts early. Stories associated with SOT Closeout; MOT Initiation started early but remained with that milestone IMS update shows early finish and early start NOTE: If a started story implements scope outside the WP, you need to open a new WP Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 18

Agile Execution: Finishing Stories Late Status at end of Sprint 2: SOT Initiation and Maintenance milestone incomplete Incomplete stories are planned for a subsequent sprint but remain with the SOT Initiation and Maintenance milestone. IMS update shows late finish Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 19

Agile Execution: Adding a Story The team determined that “correlate pulse pair detections” story was needed to complete the milestone so they added the story and its associated story point estimate to the Product Backlog and EV reporting sheet. Addition of the story correctly resulted in a decrease in milestone % complete, since the remaining effort was increased. was 39 was 87% Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 20

Agile Execution: Removing a Story The team determined that “update 9 -state KF” story was not needed to complete the milestone so they removed the story and its associated story point estimate from the Product Backlog and EV reporting sheet. Removal of the story correctly resulted in a increase in milestone % complete, since the remaining effort was decreased. was 39 was 87% Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 21

Lessons Learned § EVM on Agile projects is compliant and compatible with existing EVMS requirements and guidelines § Key challenge: how nimble are your EVMS and Contract change processes? § Take advantage of industry advances in EVM for Agile projects (e. g. NDIA) § Engage with your DCMA representatives on your plans to “go agile” “Agility is the ability to balance flexibility and stability” (Highsmith, 2002) Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 22

Summary § Agile Planning and Execution compliant with established EVMS policies, procedures and guidelines § Agile promotes a strong, continuous planning discipline with the right level of precision for each planning time-horizon under consideration. § Agile Planning is Feature-Based rather than Activity-Based to reinforce the importance of focusing on customer value. § Product Planning establishes the initial product backlog and product roadmap and provides details needed to complete the initial IMS baseline and establish the PMB § Release Planning establishes milestone scope sufficient to perform Rolling Wave Planning § Work package scope is features planned for the release § Milestone scope is features planned for the milestone Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 23

Summary § Stories comprise the inchstones for EV reporting § Inchstones are not IMS tasks and therefore have no budget, scope or schedule § When starting stories early or finishing stories late, the sprint in which a story is completed may change but the story remains associated with the milestone for which it is required § Adding or removing stories requires no changes to the EV baseline so long as the milestone scope is not impacted Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 24

Agile Planning Checklist § Product Planning − Product Backlog consists of Features of value to the customer § Release Planning: − WP and milestone scope established from the Product Roadmap and expressed as features of the system recognized as capabilities of value to the customer − Stories identified to the extent needed to establish appropriate scope for each milestone − Story points per Sprint (velocity) consistent with team composition and historical performance § IMS Planning − − Budget established at WP level using product size estimates (e. g. SLOC) and productivity Work package scope is features planned for the release Milestone scope is features planned for the milestone Work packages and milestone names are the release and milestone features Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 25

References / Contact Info References 1. Software Project Manager’s Bridge to Agility, Sliger and Broderick, 2008 2. Agile EVM – Earned Value Management in Scrum Projects, Sulaiman et al. , 2006 3. Agile Estimating and Planning, Mike Cohn, 2006 4. Manual 10255, Raytheon Company Earned Value Management Systems Process Description Contact § Jim Duffy: James_J_Duffy@raytheon. com Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 26

BACKUP Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 27

Traditional / Agile Planning: A Comparison Fixed fix scope, estimate time and resources fix date and resources, estimate scope Variable Duration and resources are a function of Scope is a function of duration and resources Manage Cost through regular Scope vs Cost analysis Source: “The Challenges of Being Agile in Do. D”, Defense AT&L, Jan – Feb 2013 Copyright © 2015, Raytheon Company. All Rights Reserved. Slide 28