ARC 214 Microsoft Solutions Development Centre Development approaches

ARC 214 Microsoft Solutions Development Centre Development approaches Using metrics to manage the software development process Presented by Vajira Weerasekera Principle Consultant

Agenda 1. What we do in the Solution Development Centre 2. How we approach software development 3. Why we use metrics to manage our delivery processes 4. Some real world examples of the metrics we use

Agenda 1. What we do in the Solution Development Centre 2. How we approach software development 3. Why we use metrics to manage our delivery processes 4. Some real world examples of the metrics we use

What is the Solution Development Centre? What is it? ► A fully equipped project delivery environment hosted within Microsoft ► Focused on complex & leading edge software development ► Uses teams made up of customer, partner and Microsoft people Why does the SDC exist? To help customers deliver ‘lighthouse’ projects that exemplify what can be done with technology and good development practices

Webjet Project Profile A snapshot of a recent large development project completed in the SDC The Solution • Online e. Commerce Solution in the travel industry, • Integrated with J 2 EE systems in the US and Qantas locally, • High availability challenge – ‘always up’ mission critical application The Development • $2 m development effort over 12 months, • Team of 13 people (3 Microsoft, 8 partner and 2 customer), • Build on. NET using Windows 2003 Server, The Results • Delivered on budget and ahead of schedule, • Turned over $1 m in business in the first 22 days, • Critical acclaim in the local and international IT press,

What is the Solution Development Centre?

Agenda 1. What we do in the Solution Development Centre 2. How we approach software development 3. Why we use metrics to manage our delivery processes 4. Some real world examples of the metrics we use

Our approach for managing projects What framework should we use ? Pick one development framework [and make sure everyone is using it ] We use Microsoft Solutions Framework as the foundation We use Australian Institute of Project Management (AIPM) Certifications on top of MSF Who is our project/program manager ? Some one who has delivered projects successfully using the same development approach Finish the project every month [daily build, test, deployment & monthly releases] For long running (6+ months) projects we construct our plans on a monthly cycle Customer is not just some one you consult for requirements [product manager] Customer needs to be in the delivery team on a regular basis Who is testing the testers ? Develop -> Testing your testing Use automation (builds, deployments, testing) & close the loop with code coverage type techniques Don’t fly blind Use project metrics to know where you are

Agenda 1. What we do in the Solution Development Centre 2. How we approach software development 3. Why we use metrics to manage our delivery processes 4. Some real world examples of the metrics we use

The challenges of project delivery ► There are many types of obvious project failure û Budget overruns û Late delivery û Non delivery û Not fit for purpose ► And some less immediately obvious types of failure û High operational costs û Burnt out delivery teams û Poor solution designs that age quickly û Inability to innovate or improve

The challenges of project delivery Some of our lessons ► We have learnt that 1. Project failures are rarely due to the technologies in use 2. Failures do not happen suddenly – they happen gradually [day by day] 3. With hindsight we can often spot signs of failure 4. Signs of failure are often observable in measurable processes 5. If we measure the right processes and watch these metrics We can see and fix problems before they become failures Metrics are about spotting signs of failure in advance by equipping ourselves with the right information now

Agenda 1. What we do in the Solution Development Centre 2. How we approach software development 3. Why we use metrics to manage our delivery processes 4. Some real world examples of the metrics we use

How we collect and use metrics We focus heavily on quality metrics, which involves: ► Building, deploying and testing every project every day ► Using automation tools to drive this process ► Generating and publishing result metrics every night ► ‘Course correcting’ the delivery process with these metrics on a daily basis ► Keeping the project schedule up to date - so that we know what trends to look out for

Types of metrics we care about We make extensive use of time phased project metrics ► Active Issues by Functional Area “What are the trouble areas in the architecture” ► Active issues by person “Workload spread and management” ► Severity mix of issues “Overall health of the architecture” ► Find and fix rates “Can we manage a predictable delivery” ► Active issues threshold “Can the team control the workload ? ” Let’s look at some real world examples …

Understanding where the product is Active issues by functional area New Testing Dimensions New functionality Code Complete Trend Scope Probing Release Trend

Getting a daily read on quality Keep an eye on the pass rates, use this to drive the test team as well

Regular statistics for the team This is the team position right now Who is the team most dependant on right now? Who is closing the most issues? Bouncing issue over time

Daily regression testing statistics This allows the team to handle defects in a controlled manner

Getting a daily read on coverage & cost Automation vs. manual The gap between these two lines is high cost – as it demands manual labour This area ‘under the curve’ is low cost once the Automated test infrastructure is in place

Predicting closure performance Find and fix rate (can we manage a predictable delivery ? )

Active Issues By Severity Overall health of the solution ‘Cleaning Up’ behaviour (bad) Functional testing engaged Stable project issue trend Stable critical issue trend

Where the issues are Workload spread and management

How long does a build and deployment take? How quickly can we deploy a release/hot fix

Who’s the most active “defect finder”

In closing… ► Daily development metrics are essential to our business ► Without it we would be ‘flying blind’ ► You need to know what trends to look for ! ► Software development is too expensive to not do this ► It is an investment in technology, process and people ► It works

Thank You Vajira Weerasekera vajiraw@microsoft. com 041 4405 996 Web resources • SDC site http: //www. microsoft. com/australia/business/services/sdc. aspx • Visual Studio Team Systems site: http: //msdn. microsoft. com/vs 2005 • MSF Site http: //www. microsoft. com/msf • Case Study http: //www. webjet. com. au/Homepage/Webjet_Microsoft. html

Your Feedback is Important! We invite you to participate in our online evaluation on Comm. Net, accessible Friday only If you choose to complete the evaluation online, there is no need to complete the paper evaluation

© 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.