AMERICAS ARMY THE STRENGTH OF THE NATION CBA
AMERICA’S ARMY: THE STRENGTH OF THE NATION CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Cost-Benefit Analysis (CBA) Four-Day Training Briefing Step 7: Compare Alternatives Step 8: Report Results and Recommendations Version 3. 0 (Draft) Last Updated: 10 November 2011 Visit our CBA Website for more information regarding locations, signing up, upcoming training sessions, and more https: //cpp. army. mil 1
Step 7: Compare Alternatives 1. Define the Problem/Opportunity and Objective 2. Define the Scope; Formulate Facts and Assumptions UNCLASSIFIED 3. Define Alternatives 4. Develop Cost Estimate for each Alternative 5. Identify Quantifiable and Non- Quantifiable Benefits CBA 4 -DAY TRAINING SLIDES 6. Define Alternative Selection Criteria 7. Compare Alternatives 7 a. Compare alternatives using the selection criteria to identify the preferred alternative. 7 b. Identify the billpayer if there is a bill associated with the recommended alternative. 7 c. Identify the positive and negative impacts of the 2 nd and 3 rd order effects. What must be done to manage the negative impacts? 7 d. Determine the robustness of the conclusions. – If anything changes (i. e. , assumptions, costs, benefits) would the recommendation change? 7 e. Identify the high-risk aspects of the recommended alternative and define appropriate risk mitigation measures. 8. Report Results and Recommendations 2
Step 7: Compare Alternatives CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Compare alternatives by: • • Concepts Comparisons of costs with benefits Decision support tools and methods (bringing the CBA together) Sensitivity analysis and Risk assessment 3
CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Concepts for Comparing Costs with Benefits • The CBA process is essentially comparing the costs with the benefits between all alternatives • The preferred alternative provides the greatest amount of value • Risk and second- and third-order effects should be included in the comparison of alternatives Value = Benefit – Costs 4
Aid for Completing - Step 7 a Costs Benefits UNCLASSIFIED Unequal Equal CBA 4 -DAY TRAINING SLIDES Unequal Equal Most likely CBA Scenario Selection Criteria Alternative that provides greatest benefits for given cost Subjective reasoning and a fortiori analysis Alternatives ranked in order (based on benefit/cost ratio, net present value, or other relevant criterion) Least costly alternative CBA may not be appropriate 5
CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Overly Complex Final Recommendation Production Risk of shutdown Local Taxes Competition for Employees Cost Alternative 1 20 widgets/yr 5% 8% 30 competitors $25 Alternative 2 15 widgets/yr 8% 6% 20 competitors $15 Alternative 3 10 widgets/yr 12% 7% 10 competitors $10 6
Comparative Methods CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Method Description When Used Net Present Value (NPV) Converts future cash flows into present equivalent values and then adds them together When alternatives have the same economic life Benefit/cost ratio (BCR) Compares present value (PV) of benefits with present value of costs When competing alternatives have unequal costs and unequal benefits Break-even Point Identifies point at which cumulative cost of two alternatives equal the cumulative benefits When projects are high-risk, to show when investment costs need to be recovered quickly Subjective reasoning Applies professional judgment as a complement to, or to the exclusion of, quantitative data. When professional judgment is considered to be more important Point System Applies objective values to subjective criteria. When decision makers wish to narrow the list of alternative solutions to the few that are most suitable Decision Matrix Allows for multiple criteria to be used to compare alternatives It is a very flexible tool that can be used under many circumstances. It can even account for other decision support methods described in this table. A Fortiori Analysis Determines whether a strongly favored alternative is still the best choice even when assumptions are formulated that put that alternative at a disadvantage. When generally accepted intuitive judgment strongly favors one alternative 7
Best Practices: • Compare the incremental costs with the incremental benefits CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED • Compare the one or two most relevant benefits with the costs • Find the “knee in the curve” or optimal value solution 8
Common Mistakes • “Over-averaging” for sake of simplification: – Example: CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED • These are examples of § “Fort Hood data is representative of all bases. ” flawed assumptions that should have been Assuming away cost: rejected in step 2. – Examples: § § “Year-end funds will be available, ” when that’s not the case “Higher headquarters will pay for it. ” “Other organizations will pay for it. ” “Military personnel are free. ” • Assuming away the problem: – Example: § “Unused office space is available. ” § “Chief of Staff said we need this. ” § Adding a layer of oversight will increase process efficiency 9
CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Common Mistakes: Over Simplification Cost ($M) Benefit Status Quo 30 Good Added capability (Alternative 1) 45 Very Good Greatly added capability (Alternative 2) 55 Excellent Which alternative has the best value? 10
The Cost of a Green Chiclet UNCLASSIFIED • “Green (fill in blank, i. e. , funding, status…) for $55 M is the preferred alternative” is the value statement. CBA 4 -DAY TRAINING SLIDES This should help in making a decision. Cost ($M) Benefit Alternative 1 55 Good Alternative 2 45 Average Alternative 3 30 Critical 11
Effects of Over simplification CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Metric Benefit Score/Rank Cost Alternative 1 20 widgets/yr Good 3 $25 Alternative 2 15 widgets/yr Average 2 $15 Alternative 3 10 widgets/yr Bad 1 $10 • What happens when using a per cost or per benefit metric? CBI BCI Alternative 1 8. 3 0. 12 Alternative 2 7. 5 0. 13 Alternative 3 10 0. 10 Best Case: COA 2 is best? 12
CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED “Cost Free” Analysis • In some cases, new programs or requirements will be supported entirely by resources that are already funded. This could be described as “cost free” in the sense that there no additional costs in terms of the budget. • Nonetheless, a “cost free” analysis must still consider cost in terms of opportunity cost: what the already-funded resource could have been used for if it had not been applied to the new requirement. 13
“ Cost Free” Analysis CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED • Example: the Organization of the Joint Chiefs of Staff is planning a holiday party for itself. The Chairman of the JCS must assign three current employees to serve on the Holiday Party Planning Committee, which requires a commitment of 16 hours of work per member. CJCS is presented with two courses of action: – COA 1: Appoint the Chief of Naval Operations, the Chief of Staff, Army, and the Chief of Staff, Air Force, to the Holiday Party Planning Committee. – COA 2: Appoint Intern #1, Intern #2, and Intern #3— three GS-05 recent college graduates—to the Holiday Party Planning Committee. • The additional budgetary cost of this new Holiday party requirement is equal for both COAs: it is “cost free” because the salaries of current employees will be paid regardless of whether the Holiday party gets planned. Determine which COA has the lower opportunity cost. 14
Common Mistakes: Assuming Away the Problem CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED All COAs proposed within a CBA must be solutions to the problem/opportunity statement. Assumptions should not solve the problem. 15
Non-Quantitative Methods Subjective Reasoning • Uses informal criteria to rank alternatives CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED A Fortiori Analysis • Used when intuitive judgment strongly favors an alternative – Assumptions are the basis in choosing which alternatives are preferred 16
Decision Matrix: CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED • A tool that compares benefits with costs to produce a single value score One of the best ways to elucidate the resource-informed decision to senior leadership is to include a Decision Matrix in the Decision Brief. 17
Decision Matrix Merits CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Pros • Easy to use • Normalizes costs and benefits • Is a familiar tool • Flexible Cons • Error prone • Highly judgmental • Loss of information via normalization • Results not definitive • Scoring is subjective 18
Example - Decision Matrix Cost $M in BY-2011 COA-2 COA-3 $20 $16 $12 Cost = $ quantifiable cost – $ quantifiable benefit or saving Rating or Ranking Decision Matrix CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Benefit Criteria Weight COA-1 COA-2 COA-3 Lethality 30% 9 6 2 Safety 45% 4 6 6 Survivability 25% 6 5 3 Score : 6. 0 5. 8 4. 1 Rating: 1 (worst) to 9 (best) Benefit = $ non-quantifiable benefit and $ non-quantifiable risk Cost per Benefit COA-1 highest benefit COA-1 COA-2 COA-3 $3. 33 $2. 78 $2. 96 COA-2 best value COA-3 lowest cost 19
Decision Matrix Note: If calculating Cost-Benefit index, the benefits criteria for each COA should be rated, not ranked. CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Otherwise, the Cost-Benefit index calculated would be meaningless. 20
Questions for Steps 7 c and 7 d 1. How sensitive is the recommendation to possible changes in costs, benefits, assumptions, etc. ? • If the recommendation is highly sensitive, can deeper analysis be done? CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED 2. Which elements of the CBA require sensitivity analysis? • Only test elements with significant uncertainty or risk. – Elements may include assumptions, constraints, costs, benefits, weighting of selection criteria, etc. 3. Address sensitivity from one or both perspectives: • What is the impact of a change of such and such magnitude? • How large a change can occur before the recommendation changes? 4. Have all probable risks and their respective impacts been identified? • • Can the risks decrease future benefits? Can the risks increase future costs? 21
Risk Assessment and Mitigation CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED The goal of risk assessment is to answer questions like: • • • What risks may occur? What is the likelihood that risks will occur? Are the source of these risks internal or external? What causes these risks? What are the consequences if the risks go unresolved? – What assets, operations, activities, functions, etc. will be affected as a result? • How much risk is tolerable? • What should be done to anticipate and limit risks? Always measure the risk by the potential adverse impacts on alternatives. 22
CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Types of Risks Business/Programmatic Risk: • Affect the budget and viability of a program Operational Risk: • Affect the ability to perform a mission Process Risk: • Associated with failing to meet standards and performance benchmarks in a newly established process Technical Risk: • Associated with failing to develop or implement technology Schedule Risk: • Associated with allocating time to perform and manage tasks Organizational Risk: • Associated with management changes All risks need to be reflected in Costs and/or Benefits of COAs Risks must factor into the decision-making process 23
Ways to Measure/Address Risk CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED • Methods of measuring and/or addressing risk in a CBA include, but are not limited to: – Effective Mean – Cost of Risk Mitigation – Sensitivity Analysis 24
Effective Mean (Expected Value) CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED • The effective mean, or expected value, of a measurable quantity is the sum of all possible outcomes multiplied by their corresponding probabilities. • Example: if the cost of a new ground combat vehicle is judged to be $1. 4 M with 50% probability, $2 M with 25% probability, and $1. 2 M with 25% probability, then the effective mean (expected cost) is $1. 4 M x 0. 5 + $2 M x 0. 25 + $1. 2 M x 0. 25 = $1. 5 M. • Example: if inter-theater transit time for a sustainment brigade is projected to be 5 days with 90% probability and 4 days with 10% probability, then the expected transit time is 5 x 0. 9 + 4 x 0. 1 = 4. 9 days. 25
Cost of Risk Mitigation CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED If a cost can be associated with reducing risk, then risk can be measured by that monetary value. Example: If for $22 K extra, the risk of a schedule over -run can be reduced from 15% to 3%, then $22 K can be a measure for the difference in risk. 26
Sensitivity Analysis CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Sensitivity Analysis: • Done mainly on the most important and least certain selection criteria, assumptions, and cost estimates • Addresses the impacts of risks comprehensively • Identifies the impact on a recommendation when elements of the analysis change, such as: – – – – Assumptions Costs Benefits Constraints Scope Weighing of selection criteria Risk probabilities 27
Suggested Steps to Conduct Sensitivity Analysis 1. Divide analysis into two groups of factors: CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED – Those outside an agency’s control (i. e. , assumptions) – Those agencies that have a degree of influence or control 2. Choose several elements (costs, assumptions, benefits, etc. ) that have greatest impact and are most likely to change – Vary each element over a reasonable set of values while holding the other variables constant relative to each other 3. Determine the impact of these changes on: – Total cost – Total benefit – Ranking of alternatives 28
CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Example: Sensitivity Analysis • Crystal Ball software may be used to perform sensitivity analyses. • In the example to the right, “Cost of Revenues %” has the greatest effect on Net Present Value, while “Year 4 Units Sold” is the least sensitive. 29
CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Exercise: Sensitivity Analysis COA-1 COA-2 • Perform a $20 $16 sensitivity analysis Cost $M in BY-2011 on the weight of Cost = $ quantifiable cost – $ quantifiable benefit or saving Lethality in this Rating or Ranking Decision Matrix Decision matrix. COA-1 COA-2 Test for weights of Benefit Criteria Weight – – 16% 44% 58% 86% Determine if the recommended COA changes. COA-3 $12 COA-3 Lethality 30% 9 6 2 Safety 45% 4 6 6 Survivability 25% 6 5 3 Score : 6. 0 5. 8 4. 1 Benefit = $ non-quantifiable benefit and $ non-quantifiable risk Cost per Benefit COA-1 highest benefit COA-1 COA-2 COA-3 $3. 33 $2. 78 $2. 96 COA-2 best value COA-3 lowest cost 30
Paying for Alternatives Identify billpayers to fund alternatives! CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Defense funding is a zero-sum game: every dollar awarded to one program comes at the expense of another competing program that will not be funded. • The CBA team rarely will have the authority to identify a bill-payer. This is a task for the organization’s comptroller/resource manager (the G-3 for most organizations) • Don’t offer a “gold watch” as a bill-payer … a program that is so important to the Army that it’s certain higher headquarters will reject it. • Alternatives will compete with other developments for funding if no billpayers are identified Note: This does not guarantee funding. 31
Quick Review CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED • Comparing the costs with benefits of each alternative is a required part of the CBA methodology. • Sensitivity analysis is a technique that analyzes whether changes in assumptions, quantitative values, or priorities will affect the recommendation. • Billpayers are the sources that will fund the costs of alternatives. 32
CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Review Exercise: Henry Ford • In a rented garage at 58 Bagley Street in Detroit, Henry Ford completed his first gas-powered car on the morning of June 4, 1896. He had spent $250 FY 1896 dollars on equipment to build the car, and $100 on labor. The garage cost $15 per month to rent. After the car was assembled, it was clear that it would not be able to fit through the door of the rented garage. The car was to be used as a model for the two-year long test-driving phase of his R&D process, with the final goal of arriving at a model for mass production and sale. 33
Review Exercise: Henry Ford CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED • Identify selection criteria and compare each of the following Courses of Action: – COA 1 (Status Quo): Leave the car in the garage, do nothing else. – COA 2: Disassemble the car and reassemble it outside the garage. – COA 3: Obtain permission of garage landlord to widen the garage door. 34
Step 8: Report Results and Recommendations 1. Define the Problem/Opportunity and Objective 2. Define the Scope; Formulate Facts and Assumptions UNCLASSIFIED 3. Define Alternatives CBA Decision Package: • • • Results and Recommendations: • • Summarizes the analysis Makes conclusive statements about each of the alternatives – 4. Develop Cost Estimate for each Alternative • 5. Identify Quantifiable and Non- Quantifiable Benefits Preferred format is a narrative or Power. Point Accompany with a decision briefing Write it in such a way that a layman can understand the topic • Results address how the alternatives were ranked using the criteria developed in Step 6 Document all data and information used in Steps 1 -8 – CBA 4 -DAY TRAINING SLIDES 6. Define Alternative Selection Criteria 7. Compare Alternatives 8. Report Results and Recommendations The recommended alternative should follow Provide supporting information so reviewers can understand how Steps 1 -8 were developed Questions for the Reviewer: - Does the package contain all key elements that are accompanied with supporting documentation? - Does the recommended alternative address the problem, and is it consistent with the assumptions and constraints? - Does the analysis explain how the recommended alternative is the best one to satisfy the selection criteria? 35
Key Elements for Documentation The CBA’s quality and comprehensiveness may be enhanced by the following supplementary content: CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED • Glossary – Defines unfamiliar abbreviations, acronyms, and terms used in the CBA • Timeline – Displays key dates and actions associated with the CBA in terms of its development and/or implementation • Coordination Sheet – Identifies who will need/have to review the CBA 36
Briefing the Results of the CBA Briefing Format: CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED • This is a suggested but not required format. – This is just a suggestion. The actual briefing for a CBA (or any briefing, for that matter) should be determined based on the briefer’s style, the known preferences of the individual being briefed, and the subject matter. – An example has been provided in the CBA guide • It is not a substitute for a well documented CBA in narrative form. – The CBA coordinator, the decision makers, and stakeholders should collaborate to determine the content and format of the CBA decision brief as they will have differing preferences. 37
CBA Workflow Automation Tool CBA Registration, Development and Submission Start (Capability Gap) UNCLASSIFIED • Proponent organizations: • Register CBA online • Develop and submit CBAs online with value proposition linked to command’s strategic priorities • Proponent leadership endorses submission online Requirements Approval CBA Validation • Requirement approval forums use CBA to: • Make resource -informed decisions • Validate and approve requirements • Organization validates the CBA methodology • Accuracy and completeness Resource Approval • Resource approval forums use CBAs to make resourceinformed decisions • Identify tradeoffs and billpayers • Decide funding approval decision End CBA 4 -DAY TRAINING SLIDES https: //cpp. army. mil/ Register and submit your CBA online Draft Submitted Lead Analyst Review CBARB Review Final Review Completed 38
CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED https: //cpp. army. mil/ 39
CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED CBA Workflow Tool 40
Quick Review CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED • Document the CBA narrative and share the results with stakeholders for approval. • CBA templates are optional tools that can be used to brief the results of the CBA. • Include all documentation and calculations in back up. • Ensure that the CBA clearly describes the value proposition, and highlight how the benefits outweigh the cost, risks, and trade offs. 41
CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED FOUR DAYS FOUR 42
Mini-case Exercise #7 or “can you hear me now? ” or “what’s the expected value? ” CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED • • U. S. Army Special Operations Command maintains a specialized communications link between the Commander, USASOC, the Chief of Staff and Vice Chief of Staff, Army, and Team Leaders in the field during highly sensitive counter-terrorism missions. The link provides video and voice transmissions that are typically delayed by 30 seconds. Currently, $2 M per month is spent on operating and maintaining the communications link: mostly consisting of payments to the National Security Agency for “satellite bandwidth. ” The NSA has recently launched into orbit a new satellite that promises shorter transmission delays: typically 10 seconds, but with a 20% risk of 20 second delays. The satellite takes advantage of recently developed technology: costs to the Army for the first month are set at $10 M, and are projected to decrease based on an 85% learning curve ($10 M for the first month, 0. 85 x $10 M per month for the next two months, 0. 85 x $10 M per month for the next four months, 0. 85^3 x $10 M per month for the next eight months, 0. 85^4 x $10 M per month for the next sixteen months, etc. ), though there is a 40% risk that costs will only decrease according to a 90% learning curve. Because time delay is the overwhelming factor in communications for counterterrorism operations, Army planners have assigned a 100% benefit weight to signal delay time. The benefit score will be determined by the function , where t is the expected time delay in seconds. 43
Mini-case Exercise #8 CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED or “to insource or not to insource, that is the question” • Currently, a contractor provides 12 translators to a SIGINT battalion, each contracted for 1885 hours annually, on a government contract costing $1. 2 M per year. The battalion commander would like to evaluate the option of insourcing the labor to Army civilians. The total burdened cost per civilian is $85, 209 and the estimated annual productivity is estimated at 1740 hours each. Determine whether insourcing would be more cost effective than the status quo. 44
Mini-case Exercise #9 CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED or “I’ll gladly pay you Tuesday for air conditioning today” • OA 22 has invited a contracting team to the Pentagon to inspect its centralized HVAC system. Following the inspection, the contractor reported minor damage to the HVAC system, and offers to replace it. Despite the damage, the HVAC system continues to function at the same capacity, although with risk of damage to the building. You assess that in any one year, there is a 15% chance of damage being done. In the event that there is damage, you assess that the average amount of damage will be $200 K if occurring within the first year, $300 K if occurring during the second year, $400 K if occurring during the third year, etc. (increasing by $100 K each year). If you decide to replace the HVAC system, the contractor has offered a payment program of $256 K for the first year of ownership, $128 K for the second year, $64 K for the third year, etc. (reduce by 50% each year). The expected life cycle of this HVAC system is 6 years. Alternatively, you may buy the same HVAC system from a separate contractor at a cost of $100 K for the first year, and $90 K for every subsequent year. Under this special program, the contractor will replace the HVAC system at the end of its life cycle without any additional material cost (operations & maintenance, replacement costs not included). All costs are given in constant FY 12 dollars. Perform all steps of the CBA process to support a course of action. 45
CBA 4 -DAY TRAINING SLIDES UNCLASSIFIED Mini-case Exercise #10 • The Captain of the Pequod has ordered a costbenefit analysis to determine the optimal number of rowboats to deploy to pursue a whale sighting. As defined by the Commander, the selection criteria are manpower, stealth, and ease of sustainment, with weights of 0. 5, 0. 3, and 0. 2, respectively. You have been told as a rule of thumb that the total mission cost can be estimated at $2200 for each rowboat involved in the hunt. Each rowboat can transport 15 fishermen. 46
- Slides: 46