Estimation Quiz For each question fill in the

Estimation Quiz For each question, fill in the upper and lower bounds that, in your opinion, give you a 90% chance of including the correct value. Be careful not to make your ranges either too wide or too narrow. Make them wide enough so that, in your best judgment, the ranges give you a 90% chance of including the correct answer. Please do not research any of the answers – the quiz is intended to assess your estimation skills, not your research skills.

There's no point in being exact about something if you don't even know what you're talking about. John von Neumann 12/7/2020 copyright Martine Devos 3

Garbage in garbage out l l Stories are different from “requirements” Often it is useful to spend some time checking quality of the stories Card Conversation Confirmation (Test) There's no point in being exact about something if you don't even know what you're talking about. —John von Neumann 4

Killer: We already have all the requirements Relating stories to requirements and legacy backlog Market User assessment wanted behavior TEST CARD Customer profiles Conversation Technical realization User stories requirement specs requirement 1 F 1. 2 STORY requirement 2 F 2. 1 F 2. 2 12/7/2020 F 2. 2. 1 5 copyright Martine Devos …

12/7/2020 copyright Martine Devos 6

Improving estimates – How to? l Spend more time estimating ? l l Diminishing returns Using correction factors ? l 12/7/2020 Skills, collocation… increases the subjectivity and diminishes accuracy copyright Martine Devos 7

Cone does not improve itself l Estimates improve l l l When we collect data Reflect on estimates Remove variability Making decisions l Keeping team stable… l l If we do not cone can very clouded and even get wider 12/7/2020 copyright Martine Devos Strong stimulas for Product Owner and Managers to make decisions 8

When to re-estimate? l l l estimate actual Chart pie 3 At least 6 Text 3 Chart line 5 ? Charts are so much more difficult They finish only 1 and 6. Line Chart 3 ? No re-estimating? Re-estimating the finished story? Re-estimating relative sizing Sum 3 Recommend 5 When to re-estimate: When relative size changes Use as a learning experience 5 9

Over or Under-estimate ? 12/7/2020 copyright Martine Devos 10

FAQ: Relating task estimates to velocity l l NO Teams go astray when they start believing there is a strong relationship between story points or ideal days and exact number of hours. 11

Bugs – How to plan? Agile teams have the goal to fix bugs IN the iteration they are discovered l Estimates should include that work l Automated tests help us getting better at that l l Defects found later need to be treated the same as user stories – put on backlog, prioritized, acceptance tested… 12/7/2020 copyright Martine Devos 12

Maintenance, current engineering, code reviews…what is impact? Estimates do not change l Commitments however need to be made with maintenance work in mind l Reduce the availability or make allowance (as a task) for the group l l Do not forget meetings (like code review)… that are needed Add estimate for the group 12/7/2020 copyright Martine Devos 13

Estimating large backlog items l l Some features are too large to reliably estimate until they are broken down That work is not done until later in the release A rough order of magnitude can be achieved using 3 -value estimation Each feature is given 3 estimates l l 12/7/2020 Best case, nominal case, worst case Best and worst case should reflect the risk and uncertainty in requirements Should be in story points IS NOT a Bell Curve copyright Martine Devos 14

What if agile planning is not acceptable? Why? l Project planned far in advance l Must meet a firm deadline and include reasonable set of features l Project is contracted l Requirements are understood only at a very high level l Organization is uncomfortable allowing too much schedule flexibility 15

Put 5 car lengths between 2 cars -- We work with Buffers l Agile normally Feature Buffer l Must l have, should have, could have Sometimes need for Schedule Buffer l Best Case, Most Likely Case Often combining both To be uncertain is to be uncomfortable, but to be certain is to be ridiculous. (chinese proverb) l 16

Beware of: “Schedule compression impossibility zone”

Estimation and Planning Focus on Agile Beyond Agile 12/7/2020 copyright Martine Devos 19

References Excellent book: Steve Mc Connell, Software Estimating l Intro to agile estimation and planning: Mike Cohn, Agile Estimation and Planning l 12/7/2020 copyright Martine Devos 20

Estimation has a bad reputation l “Estimation is the most optimistic prediction that has a non-zero probability of coming true” l Accepting this definition leads toward a method called “what is the earliest date by which you cannot prove you will not be finished”- estimating l Tom 12/7/2020 De. Marco (cited by Mc Connell) copyright Martine Devos 21

Urban legends l l Complicated formulas are more accurate than simple estimates We need to correct estimates with factors for team collocation, experience… reality is opposite – the more chances for subjectivity to creep l “They will not let us…” management does not want correct estimates, they just want a date… Many so called estimation problems arise from misunderstanding what an “estimate’ is and blurring other similar-but-not-identical concepts with estimation. 12/7/2020 copyright Martine Devos 22

What is an “estimate”? 1. 2. 3. A tentative evaluation or rough calculation. A preliminary calculation of the cost of a project A judgment based upon one’s impressions; opinion (source: American Heritage Dictionary, Second College Edition, 1985) Does this sound like what we are asked for (asking for) an estimate? When executives ask for an estimate, they’re often asking for a commitment or for a plan to meet a target. 12/7/2020 copyright Martine Devos 23

Strictly speaking the dictionary is correct, but in software (and other projects) estimation interplays with targets, commitments and control. l Need to make clear distinctions l l In our vocabulary l And in our conversations 12/7/2020 copyright Martine Devos 24

Targets – (very legitimate!) We need version 2. 1 to be ready to demonstrate at the tradeshow in May l We need that release stabilized before the holidays l We must limit the cost to $2 million. That is the maximum budget for the release. l 12/7/2020 copyright Martine Devos 25

A typical conversation l Executive: How long do you think this project will take? We need to have this ready in 3 months for a trade show. I cannot give you any more team members, so you will have to work it out with your current staff. Here is the list of features we will need. 12/7/2020 l l Later, the project lead: We have estimated it will take 5 months. Executive: Five months? Did not you hear me? I said we needed it ready in 3 months for the tradeshow. copyright Martine Devos 26

A better conversation l l Executive: How long do you think this project will take? We need to have this ready in 3 months for a trade show. I cannot give you any more team members, so you will have to work it out with your current staff. Here is the list of features we will need. Project lead: Let me make sure I understand what you are asking for. Is it more important for us to deliver 100% of the feature, or is it more important to have something ready for the tradeshow? 12/7/2020 l l Executive: We have to have something ready for the tradeshow. We would like 100% of those features if possible. Project lead: I want to make sure I follow through on your priorities as best as I can. If it turns out that we cannot deliver 100% of the features by the tradeshow, should we be ready to ship what we have or should we plan to slip the date beyond the tradeshow? Executive: We have to have something for the tradeshow, so we have to ship something even if it is not 100% what we want. Project lead: OK, we will come up with a plan for delivering as many features as we can in the next 3 months. copyright Martine Devos 27

What if there is Mismatch between goal and estimates? l l Tell a lie to keep our job? Negotiate the estimate? If the mismatch is allowed to persist the options will be far fewer There are NO white lies. l l NO ! The detection of a mismatch between the project goal and the project estimate should be interpreted as useful, early-in-the-project risk information. 28

10 key characteristics of software executives Execs will always ask for what they want l Execs will always probe to get what they want if they do not get it initially l Execs will tend to probe until they discover your point of discomfort l Execs will not always know what is possible, but they will know what would be good for the business if it were possible l Execs will be assertive. That is how they got to be executives in the first place l Execs want you to operate with companies best interest in mind l Execs will want to explore lots of variations to maximize business value l Execs know things about the business, the market and the company that we do not know and they may prioritize goals differently than we would l Execs will always want visibility and commitment early (which would indeed give high business value if we could) Mc Connell l 12/7/2020 copyright Martine Devos 29

Vocabulary A Target = a statement of desirable (often much needed) business objective l A Commitment = a promise to deliver defined functionality at a specific level of quality by a certain date. Commitment can be the same as an estimate, it can be more aggressive or more conservative. It does NOT have to be the same as the estimate. l An Estimate should be treated as an unbiased, analytical process – the goal is accuracy, not to seek a specific result l Planning should be treated as a biased, goal-seeking process – the goal is to seek a particular result Combining estimation and planning ends up in a poor estimate and a poor plan or both l l We refine estimates over time and we improve the plan 12/7/2020 copyright Martine Dweevos 30

Consequently –> Good Estimates help project control l Sometimes people treat software estimation as a pure prediction activity. In reality, once we make an estimate and, on the basis of that estimate a commitment, we control the project to meet the targets. A good estimate is an estimate that provides clear enough view of the project to allow PM to make good decisions to hit the targets. 12/7/2020 copyright Martine Devos 31

AGILE PLANNING 12/7/2020 copyright Martine Devos 32

Planning depends on accurate estimates Creating a detailed schedule l Identifying the project’s critical path l Creating a complete work breakdown structure l Prioritizing functionality for delivery l Breaking a project into iterations l 12/7/2020 copyright Martine Devos 33

Project Management Variables Feature Breakdown Structure, rather than WBS Focus on Value Produced, rather than hours burned (Velocity ) turning Product Backlog into increments of functionality Product Backlog Time Other variables: Quality, Value Copyright 2003 -2005, ADM, All Rights Reserved v 6. 0 34

Core to agile is feedback Reduce the Black-Out Periods Done is Binary - done/not done 12/7/2020 copyright Martine Devos 36

Focus of Scrum Planning l Levels of Planning l Strategy l Portfolio l Product l. Release l. Sprint l Day Agile focus is on planning between product (considered +/- defined) and iteration. Daily planning is team’s concern Mike Cohn calls it the Planning Onion

Variation in Backlog Decomposition Longer term plan can tolerate more uncertainty Short term plan needs more detail 1 2 3 4 5 Release 1 6 7 Release 2

Planning – Release Sprint l l Release plan/feature backlog l Desired features are defined at a high level and prioritized in a release plan Iteration plan/task backlog l High level features from the release plan are elaborated into tasks and prioritized by the team 12/7/2020 copyright Martine Devos 39

Stories Refine the Project Backlog items come in many shapes and sizes to Tra n ac sform tio na bac ble kl sto og rie s Stories are small, and only come in a few sizes

In an iteration: from story to task Story a 5 Story b 6 Story c 4 Code UI 12 Story d 3 5 Story e 5 Code database Story f 5 Script tests 12 Story g 3 … 6 Story h 3 Story i 5 Story j 5 Story k 3 Story l 8 Story m 2 …. …. …

Sprint Backlog – visibility within the team Shows at any time real progress – done Tasks need to be small enough – about 1 ideal day of work. Copyright 2003 -2005, ADM, All Rights Reserved v 6. 0 42

Sprint Planning and estimates at sprint planning l l l Decompose and Recompose l Story task Story l Estimate tasks and compare with original estimate l Tasks (functional) not Activities (like coding, testing…) l Negotiate commitment if needed Should be a group effort l Since tasks are not allocated to individuals at time of planning l Hearing someone’s reason for estimate helps understand stories l Pride of doing the work within estimate by the person who picks it up later Estimate in hours for corporate metrics and allocation l Do not use too soon for conclusion – no micro management (scrum master…) l Is a tool for team usage l External pressure may/will lead to padding 12/7/2020 copyright Martine Devos 43

Scrum and PM tools l l A Gantt chart for scrum is simple Each sprint is a roll-up Product backlog items are the tasks AND each task lasts the full sprint 30 days Task 1 Task 2

Estimation and Planning Release Planning l l Create a plan with the estimates and velocity Shows feasibility and indicates what will be worked on each iteration Sprint 1 Sprint 2 Following Sprints

ESTIMATION

Reducing Encertainty l Why do we plan? Why do we estimate? high low high What? high low How? Waterfall Approach How? Agile Approach

Agile and Risk The software process – a process of steady refinement. l A good estimate is an estimate that provides clear enough view of the project to allow PM to make good decisions to hit the targets l The further into the project you are, the finer grained the decomposition can be. l 12/7/2020 copyright Martine Devos 48

Estimates and Probability Single point estimates assume 100% probability that the actual outcome is equal the planned outcome 12/7/2020 copyright Martine Devos 49

Accuracy and (Unwarranted) Precision Shortest possible schedules are achieved by creating the most accurate possible not the most precise estimates If you want to achieve maximum development speed avoid false precision An estimate of 20 -50 days may be accurate but not precise An estimate of 35 days for the same situation creates a false feeling of precision 373. 2 days + or – 2 months 3741 hours for a feature Estimates in hours at Release planning (NOT enough detail available!) = unwarranted precision Is Pi = 3. 32367 better than Pi is 3? 12/7/2020 copyright Martine Devos 50

pi = 3 Accurate to 1 significant digit, but not precise pi = 3. 37882 Precise to 6 significant digits, but accurate only to 1 significant digit pi = 3. 14159 Both accurate and precise, to 6 significant digits My height = 2 meters Accurate to 1 significant digit, but not very precise Airline flight times Precise to the minute, but not very accurate "This project will take 395. 7 days, ± 2 months“ Highly precise, but not accurate to the precision stated "This project will take 4 years”’ Not very precise, but could be accurate "This project will require 7, 214 staff hours“ Highly precise, but probably not accurate to the precision stated 12/7/2020 51

Cone of uncertainty Vision estimate -75% - + 300% Exploratory estimate -50% - + 100% Budget estimate -20% - + 25% Commitment estimate -10% -+ 10% 12/7/2020 copyright Martine Devos 52

Correspondence with 3 Levels of Precision in agile l Stories in story points l Car rental (or Ideal days) system l Tasks in hours l (estimate) l Looking back at daily scrum: I worked for X hours on the UI and I will be able to finish before the end of the day (actual time) l l l As a user, I want to… As a renter, I want to… Write test fixture Code UI… Release 12/7/2020 copyright Martine Devos 53

FAQCan we estimate before we do analysis? l l l If the acceptance tests are not written yet, can we give an estimate? Yes – we can give an estimate. AND it will not be perfect. This is not different from before – management needed estimates at the gates, to make budgets… We can give a range, can give a relative value (story points) with level of confidence THE BENEFIT of agile is that we learn faster and have the opportunity to improve our estimates faster 12/7/2020 copyright Martine Devos 54

IEEE/ACM on Using ranges when estimating We do not take away the uncertainty by presenting the estimate without uncertainty. l Ignoring it is a serious fault on the engineer’s side 3. 09 IEEE-CS/ACM Software engineers code of ethics (cited by Mc Connell): Ensure realistic quantitative estimates of cost, scheduling, personnel, quality and outcomes of any project on which they work or propose to work and provide an uncertainty assessment of these estimates l 12/7/2020 copyright Martine Devos 55

It is better to be roughly right than precisely wrong. John Maynard Keynes 56

Cone does not improve itself l Estimates improve l l l When we collect data Reflect on estimates Remove variability Making decisions l Keeping team stable… l l If we do not cone can very clouded and even get wider 12/7/2020 copyright Martine Devos Strong stimulas for Product Owner and Managers to make decisions 57

TECHNIQUES 12/7/2020 copyright Martine Devos 58

Turning size into duration The estimation of duration begins with size (and complexity) l When we can count, we count l Estimate Size Derive Duration Schedule 59

Story points – Dimensionless units l Estimate in dimensionless but proportional units l. A “ 6” takes twice as long as a “ 3” Customer uses this information to juggle priorities l Relationship between these units and real time is known as “velocity” l l Measured l at the end of each iteration Spikes may be used to learn enough to estimate 12/7/2020 copyright Martine Devos 60

Velocity l l To understand how estimation in unit-less points works we need the concept of velocity. Velocity is a measure for the teams rate of progress. l l It is calculated by summing up the number of story points assigned to each user story that the team completed during each iteration. Our best guess is that the team will complete the same amount of work next iteration (“yesterday’s weather”) 61

Floorplan l l You are hired to paint this house, materials will be there… You only have the floorplan, did not see the actual house. You estimate the two bedrooms to be each 5 points… How long will it take to paint this house. 62

Learning from the Burndown Current date Start date Actual stories Planned Target date

When to re-estimate? l l l estimate actual Chart pie 3 At least 6 Text 3 Chart line 5 ? Charts are so much more difficult They finish only 1 and 6. Line Chart 3 ? No re-estimating? Re-estimating the finished story? Re-estimating relative sizing Sum 3 Recommend 5 When to re-estimate: When relative size changes Use as a learning experience 5 65

Tool: Planning Poker Group technique l Precision decreases as story size increases, therefore constrain estimates to pre-defined values (e. g. 1, 2, 3, 5, 8, 21 or ½, 1, 2, 3, 5, 8, 13, 20 (40, 80 if you really need to) l 12/7/2020 copyright Martine Devos 66

Over or Under-estimate ? 12/7/2020 copyright Martine Devos 67

Wideband Delphi Estimation Iterative approach for improved estimates l Every developer gives his estimate l Discuss differences, especially extremes l Re-estimate until convergence l 12/7/2020 copyright Martine Devos 68

Advantages Relative Size Wideband Delphi l Studies that show we are l Improvements of 2/3 better at relative and when applied to estimating terrible early l Some developers find it requirements up to much easier to say “this is 8/10 improvements like that” l Enforces separation of l Cuts error by 40% estimates of size from estimates of duration l Need to re-estimate only when the relative 12/7/2020 copyright Martine Devos changes 69

Improving estimates – How to? l Spend more time estimating ? l l Diminishing returns Using correction factors ? l 12/7/2020 Skills, collocation… increases the subjectivity and diminishes accuracy copyright Martine Devos 70

Cone does not improve itself l Estimates improve l l l When we collect data Reflect on estimates Remove variability Making decisions l Keeping team stable… l l If we do not cone can very clouded and even get wider 12/7/2020 copyright Martine Devos Strong stimulas for Product Owner and Managers to make decisions 71

Triangulate estimates and baseline Compare with stories from the past – same team l AND Keep track of actual values as source of triangulation for the future Think about some other stories what we have estimated so far, consider which stories seemed smaller and what value; consider stories that seemed bigger, what value. Now what is a close triangulation of an estimate for this story based on these other stories l 12/7/2020 copyright Martine Devos 72

Planning support tools? l l l Story-cards Wall, white board… Later used to show to and track progress at all time 12/7/2020 copyright Martine Devos 73

Tool Support There are planning but the should not dictate the process Copyright 2004, ADM, All Rights Reserved 75

PRESENTING ESTIMATES TO OTHER STAKEHOLDERS

Poorly estimated projects present ONE single point early estimate Initial concept 10 Approved product definition 12 Requirements complete 13 User interface design 14 First interim release 16 • Customer will consider the project to have slipped the first time the estimate changes. • What really happened is that the first time you had a very high degree of inaccuracy

Start with a range indicating the uncertainty Rather than losing the customer’s confidence by taking one schedule slip after another, you build confidence by meeting the expectations and by tightening the ranges estimate Initial concept 3 -24 Approved product definition 5 -20 Requirements complete 9 -20 User interface design 12 -18 First interim release 15 -18

Risk quantification in estimates? Estimate 6 months (+3 -2) l +1 for late delivery of graphics formatting system l + 1 for new dev tools not working as well as expected l + 0. 5 for staff sickness l +0. 5 for underestimating by new members l l - 1 for less delay in hiring new developer - 1 for new dev tools working better than expected 12/7/2020 copyright Martine Devos 79

Use the appropriate units and express uncertainty l Try to use estimates in units that are consistent with the estimate’s underlying accuracy. l l We can deliver in the 3 th quarter The project will require 10 staff years Rather than l l l We will deliver May 21 st This project will require 15388 hours I am 90 percent sure we will complete this function… 80

Project Estimate Include assumptions The base schedule estimate is 6 calender months, which we believe to be accurate to within 25%. This estimate can be used as basis for the product budget, but nor for external commitments. The estimate is based on following assumptions. 1. The three key technical leaders will be assigned 100% to the project 2. No more than 5 new reports are required. 3. The project will be able to reuse at least 80% of the database code… …… 81

There's no point in being exact about something if you don't even know what you're talking about. John von Neumann 12/7/2020 copyright Martine Devos 83

Garbage in garbage out l l Stories are different from “requirements” Often it is useful to spend some time checking quality of the stories Card Conversation Confirmation (Test) There's no point in being exact about something if you don't even know what you're talking about. —John von Neumann 84

Killer: We already have all the requirements Relating stories to requirements and legacy backlog Market User assessment wanted behavior TEST CARD Customer profiles Conversation Technical realization User stories requirement specs requirement 1 F 1. 2 STORY requirement 2 F 2. 1 F 2. 2 12/7/2020 F 2. 2. 1 85 copyright Martine Devos …

12/7/2020 copyright Martine Devos 86

Improving estimates – How to? l Spend more time estimating ? l l Diminishing returns Using correction factors ? l 12/7/2020 Skills, collocation… increases the subjectivity and diminishes accuracy copyright Martine Devos 87

Cone does not improve itself l Estimates improve l l l When we collect data Reflect on estimates Remove variability Making decisions l Keeping team stable… l l If we do not cone can very clouded and even get wider 12/7/2020 copyright Martine Devos Strong stimulas for Product Owner and Managers to make decisions 88

When to re-estimate? l l l estimate actual Chart pie 3 At least 6 Text 3 Chart line 5 ? Charts are so much more difficult They finish only 1 and 6. Line Chart 3 ? No re-estimating? Re-estimating the finished story? Re-estimating relative sizing Sum 3 Recommend 5 When to re-estimate: When relative size changes Use as a learning experience 5 89

Over or Under-estimate ? 12/7/2020 copyright Martine Devos 90

FAQ: Relating task estimates to velocity l l NO Teams go astray when they start believing there is a strong relationship between story points or ideal days and exact number of hours. 91

Bugs – How to plan? Agile teams have the goal to fix bugs IN the iteration they are discovered l Estimates should include that work l Automated tests help us getting better at that l l Defects found later need to be treated the same as user stories – put on backlog, prioritized, acceptance tested… 12/7/2020 copyright Martine Devos 92

Maintenance, current engineering, code reviews…what is impact? Estimates do not change l Commitments however need to be made with maintenance work in mind l Reduce the availability or make allowance (as a task) for the group l l Do not forget meetings (like code review)… that are needed Add estimate for the group 12/7/2020 copyright Martine Devos 93

Estimating large backlog items l l Some features are too large to reliably estimate until they are broken down That work is not done until later in the release A rough order of magnitude can be achieved using 3 -value estimation Each feature is given 3 estimates l l 12/7/2020 Best case, nominal case, worst case Best and worst case should reflect the risk and uncertainty in requirements Should be in story points IS NOT a Bell Curve copyright Martine Devos 94

What if agile planning is not acceptable? Why? l Project planned far in advance l Must meet a firm deadline and include reasonable set of features l Project is contracted l Requirements are understood only at a very high level l Organization is uncomfortable allowing too much schedule flexibility 95

Put 5 car lengths between 2 cars -- We work with Buffers l Agile normally Feature Buffer l Must l have, should have, could have Sometimes need for Schedule Buffer l Best Case, Most Likely Case Often combining both To be uncertain is to be uncomfortable, but to be certain is to be ridiculous. (chinese proverb) l 96

Beware of: “Schedule compression impossibility zone”