EVMS Grab Bag 5 EACETC Estimate Uncertainty Monte

EVMS Grab Bag #5 EAC/ETC Estimate Uncertainty Monte Carlo Kevin Long Project Controls Manager EVMS Grab Bag #4 • Tucson • 1 -11 -2018 1

EVMS Grab Bag #5 - Outline • Estimate at Completion (EAC) 2018 development – Definitions – Estimate to Complete (ETC) Roles and responsibilities • CAMs • Subsystem Managers • Project Controls – Timeline • Updated Monte Carlo Analysis for Cost/Schedule – Estimate Uncertainty – Risk Register Refresh – Timeline EVMS Grab Bag #4 • Tucson • 1 -11 -2018 2

ETC and Estimate At Complete (EAC) EVMS Grab Bag #4 • Tucson • 1 -11 -2018 3

ETC and Estimate At Complete (EAC) Definition • Estimate At Complete (EAC) = Actual Cost of Work Performed (ACWP) + Estimate To Complete (ETC) – EAC = ACWP + ETC – EAC = actuals + remaining work • Your ETC is a forecast of what you as CAM believe it will take you to deliver your remaining scope of work in your control accounts • While earned value performance reporting gives us a view of the past (% complete to date, SPI, CPI), the ETC allows us to take a close look at what we have left to complete, or the balance of the remaining work – Work in process - If a work package is 40% complete, are you on track to complete the remaining 60%? – Future work - If you haven’t started the work package or planning package, based on what you know today, is the work planned and costed appropriately? • What work is remaining and how will you accomplish it and at what cost? EVMS Grab Bag #4 • Tucson • 1 -11 -2018 4

ETC Role & Responsibilities for CAMs [1] • CAMs must consider the financial state of the project – We have not retired risks or adjusted the cost exposure at the initially projected rate, now our contingency position is no longer appropriate for the remaining risks. – CAMs must: • Limit their scope to what is necessary for delivery of the baseline and • Accurately estimate the remaining work • Ensure data in the Risk register is accurate and up to date • Identify the remaining work, April 1 st forward – What needs to be purchased – Who will do the work – Determine execution means and methods • Identify changes to risks and opportunities – Propose risk register updates • Define any uncertainties and be reality based EVMS Grab Bag #4 • Tucson • 1 -11 -2018 5

ETC Role & Responsibilities for CAMs [2] • Prepare a presentation with the following 5 slides – Control accounts for which work has not started – Control accounts for which work has started • Remaining planning packages where work has not started – CAM’s assessment of accuracy of remaining work – CAM’s assessment if estimated effort is appropriate – Time by which work will be planned out • Remaining and Open work packages – – CAM’s assessment of accuracy of remaining work CAM’s assessment if estimated effort is appropriate Time by which work will start and complete Number and types of FTEs needed to complete work – Discussion of open risks that are not retired and list of steps to be taken to retire the risk – CAM’s staffing profile and plan to transition staff – Identification of scope that can be cut • Presentations and meetings with Project Manager Will be held March 28 th-29 th EVMS Grab Bag #4 • Tucson • 1 -11 -2018 6

Example Presentation • Control accounts for which work has not started – None. All Control Accounts assigned to CAM Name here are closed or in progress EVMS Grab Bag #4 • Tucson • 1 -11 -2018 7

Example Presentation • Control accounts for which work has started – Remaining planning packages where work has not started – – I am confident the accuracy of remaining work scope is clearly defined in this planning package Remaining work in the Project schedule to be detail planned is clearly defined and time phased and will be planned through the rolling wave process. – Open work packages – – – Remaining work to complete – Look at e. CAM (BAC – BCWP) Number and types of FTEs needed to complete work (ETC) Time by which work will be complete (Early, on schedule, Late) EVMS Grab Bag #4 • Tucson • 1 -11 -2018 8

Example Presentation • Discussion of open risks that are not retired and list of steps to be taken to retire the risk EVMS Grab Bag #4 • Tucson • 1 -11 -2018 9

Example Presentation • CAM’s staffing profile and plan to transition staff – Breakdown of Resources by, FY and if the transition to Operations EVMS Grab Bag #4 • Tucson • 1 -11 -2018 10

Example Presentation • Identification of scope that can be cut – Is there scope that can be cut? What are impacts to procjet deliverables? Cost Savings? EVMS Grab Bag #4 • Tucson • 1 -11 -2018 11

ETC Role & Responsibilities for Subsystem Managers • Meet with each CAM, review and discuss ETC before the Project Manager review – Ask appropriate questions to assess strength of estimate • Attend ETC meetings with Project Manager – CAMs present prepared slides • Update Risk register • Document explanations for significant increases/decreases from EAC 2017 EVMS Grab Bag #4 • Tucson • 1 -11 -2018 12

Recap EAC 2017 • To Date 19 M of the 27 M VAC has been added to the PMB through the change control process. EVMS Grab Bag #4 • Tucson • 1 -11 -2018 13

ETC Role & Responsibilities for Project Controls • Collect ETC input from CAMs and enter into P 6 – Labor resources by name – Procurement estimates and timing – Activity start and duration – Activity relationships (predecessors and successors) – Milestones • Assess impact to project schedule – Float values for major systems – Critical path and near critical path items • Review P 6 results with CAM • Determine EAC 2018 – Integrate P 6 into Cobra • Calculate cost of ETC • Add ETC to month-end March actual cost (ACWP) – Communicate EAC 2018 results by control account EVMS Grab Bag #4 • Tucson • 1 -11 -2018 14

EAC 2018 Development Timeline Action (What) • CAMs identify remaining scope, issues, risks and opportunities • Review with Subsystem managers • CAMs prepare slides • CAMs meet with Project Manager and Subsystem Manager • CAMs meet with Project Controls (PC) • PC enters ETC data into P 6 • PC integrates into Cobra • PC communicates EAC 2018 results Date (by When) 15 Mar – 27 Mar 28 Mar – 29 Mar 08 Apr – 18 Apr • Executive Management Team reviews • CAMs make adjustments as needed • PC integrates into Cobra 18 Apr – 21 Apr • PC freezes EAC 2018 • PC publishes EAC 2018 in e. CAM • PC publishes month-end Mar performance in e. CAM 23 Apr – 30 Apr EVMS Grab Bag #4 • Tucson • 1 -11 -2018 15

LSST Change Request (LCR) Considerations • A Project Change Request (LCR) is a request to make a formal change to the baseline – Once a LCR is approved and implemented, the baseline plan you are measured against is changed • An Estimate To Complete (ETC) is a forecast of what it will cost you to complete your scope assigned to you in your control accounts – A forecast or ETC does not change the baseline – A forecast or ETC is not a request for additional funds or resources • Approved and implemented LCRs will already be incorporated into both the Baseline and Forecast schedules as of month-end March, 2018 – Look at your future activities in the Forecast Schedule to develop your ETC • Unapproved or Pending LCRs have not been incorporated into the Baseline Schedule nor the Forecast Schedule – Account for pending PCRs as manual modifications to the Forecast Schedule as you develop your ETC EVMS Grab Bag #4 • Tucson • 1 -11 -2018 16

Updated Monte Carlo Analysis for Cost/Schedule EVMS Grab Bag #4 • Tucson • 1 -11 -2018 17

Estimate Uncertainty • The risk management tool uses a three point probability distribution for the Monte Carlo simulation. – the best case, most likely, and worst case data for cost and schedule • The Baseline cost and schedule data provided by the subject matter experienced estimator is defined as the "most likely" for both cost and schedule duration. • To ensure a methodical and uniform approach to developing "best" and "worst" case values, LSST uses well defined uncertainty factors and Risk multipliers to derive default values. • The uncertainty DOES NOT include items that are in the risk register and vice versa. That would be double counting. EVMS Grab Bag #4 • Tucson • 1 -11 -2018 18

Factors and Multipliers Factor Technical Uncertainty Cost Uncertainty Schedule Uncertainty 1 Existing design and off-the-shelf hardware Off the shelf or catalog item Not used 2 Minor modifications to an existing design Vendor quote from established drawings No schedule impact on any other item 3 Extensive modifications to an existing design Vendor quote with some design sketches Not used 4 New design within established product line In-house estimate for item within current production line Delays completion of non-critical path subsystem item 6 New design different from established product line. Existing technology In-house estimate for item with minimal company experience but related to existing capabilities Not used 8 New design. Requires some R&D In-house estimate for item with minimal company development but does not advance the stateexperience and minimal in-house capability of-the-art Delays completion of critical path subsystem item 10 New design. Development of new technology Top down estimate from analogous programs which advances the state-of-the-art Not used 15 New design way beyond the current state-ofthe-art Not used Engineering judgment • Each uncertainty factor is multiplied by a risk-factor multiplier between 1– 4 percent. Technical Risk Cost Risk Condition Design or manufacturing concerns only Design and manufacturing concerns Material cost or labor rate concern Material and labor rate concern Schedule Risk EVMS Grab Bag #4 • Tucson • 1 -11 -2018 Risk-Factor Multiplier 2% 4% 1% 2% 1% 19

Uncertainty factors and multipliers to develop the Best, Most Likely, and Worst Case probability distribution Parameter Abbreviation Definition Technical Uncertainty Factor TU Valued 1 -15 based on the technical readiness of the element from existing and off-theshelf to beyond the state-of-the-art. Cost Uncertainty Factor CU Valued 1 -15 based on the basis of estimate from an existing catalog item, or some related previous experience to an engineering judgement. Schedule Uncertainty Factor SU Valued 2, 4, and 8 based on the schedule impact from no impact to other scheduled activities to Critical Path impact. Technical Risk Multiplier Cost Risk Multiplier Schedule Risk Multiplier TM CM SM Value is based on Design and/or Manufacturing concerns. Value is based on Material cost and/or Labor rate concerns. Value is based on schedule concern. 3 -Point Distribution Most Likely Cost Best Case Cost Worst Case Cost Most Likely Schedule Best Case Schedule Worst Case Schedule Value Formula This is 100% of the Baseline Activity Cost. 100% -[(TU +CU)*CM] * Baseline Cost 100% + [(TU*TM)+(CU*CM)+(SU*SM)] * Baseline Cost 100% of the Baseline Activity Duration 100% -[(TU +SU)*SM] * Baseline Duration 100% + [(TU*TM)+ (SU*SM)] * Baseline Duration • https: //confluence. lsstcorp. org/display/PC/EAC+2018 will be populated with excel templates to collect uncertainty factors and multipliers EVMS Grab Bag #4 • Tucson • 1 -11 -2018 20

Cost Contingency Estimation through Monte Carlo • The objective is to calculate contingency cost estimate and risk adjusted schedule using Monte Carlo analysis. • Contingency calculation is based on an estimate uncertainty analysis + risks identified in the risk register and other project wide risk factors. • Primavera Risk Manager integrates directly with the LSST project schedule and holds the risk factor matrix. EVMS Grab Bag #4 • Tucson • 1 -11 -2018 21

Risk register refresh • Update the risk register • https: //confluence. lsstcorp. org/display/PC/EAC+2018 populate the Risk. Factors field for each activity. • There can be multiple risks assignet to an activity, separate with a comma and include all applicable risks – Example: 95, 96, 97, 98, 99, 101, 103, 104, 105, 106, 108, 109 EVMS Grab Bag #4 • Tucson • 1 -11 -2018 22

Risk Updates and Monte Carlo Timeline Action (What) • Validate Risk Register entries (PM/CAMs) • Verify uncertainty factors on the basis of risk update • Validate Risk factor assignments • Post updates to Confluence EAC 2018 Page Date (by When) 15 Mar – 15 Apr • Incorporate changes (PC) • Run Primavera Risk Manager (PC) 15 Apr – 20 Apr • Produce final results (PC) • Develop briefing package(PM/PC) 22 Apr – 27 Apr EVMS Grab Bag #4 • Tucson • 1 -11 -2018 23

Cost EVMS Grab Bag #4 • Tucson • 1 -11 -2018 24

Schedule EVMS Grab Bag #4 • Tucson • 1 -11 -2018 25