Service Metrics Measure improve measure improve Richard Mironov

Service Metrics § § § Decide what’s important Quantify it Measure compulsively Root cause

What’s Important to Users? § Each service has a few fundamental attributes § Postpone

Specific and Narrow § Easy to paint universal goals § “All functions must be

Take the Customer’s View § Metrics must be external, as customers experience them §

Quantify and Measure § Ongoing analytical effort § Operations and finance have best talent

1. Customer-Reported Outages § When customers call in, you have already failed 7 www.

2. Design Metric and Test Tool § Goal: working 99. 5%, no transactions lost

3. Read the Charts 9 www. mironov. com rich@mironov. com 650. 315. 7394

4. Isolate and Fix One Problem § Something weird happens every fourth day… …a

5. Repeat § Prominent dips are power outages 11 www. mironov. com rich@mironov. com

6. Turn the Numbers Sideways § Overloaded when most users connect: 9 AM-10 AM

Measure Compulsively § Many causes of problems § Some outside your control § Improves

Service Level Agreements § SLAs are owned by Marketing § Designed to win and

Love Your Customers Word-of-mouth is the only route to service success, so. . .

Summary § Decide what’s important § Define and measure it § Find and fix

Slides: 16

Download presentation

Service Metrics § § § Decide what’s important Quantify it Measure compulsively Root cause analysis Love your customers 2 www. mironov. com rich@mironov. com 650. 315. 7394

What’s Important to Users? § Each service has a few fundamental attributes § Postpone less important metrics Examples: § Email: server up, addresses work, mail saved § Next tier: delivery time, disk limits, virus scans § Newswire filtering: automatic, easy, broad search base, inexpensive § Next tier: uptime, speed, delivery method 3 www. mironov. com rich@mironov. com 650. 315. 7394

Specific and Narrow § Easy to paint universal goals § “All functions must be up 99. 995%” § Instead, focus on critical service elements first § Example: Uptime for corporate directory § Sub-second password look-up needed 24 x 7 § OK to wait 5 minutes for password updates § New account creation needed only during normal work hours § Backup must be done within 8 hours, by midnight § Remote disaster recovery can use last Friday’s data 4 www. mironov. com rich@mironov. com 650. 315. 7394

Take the Customer’s View § Metrics must be external, as customers experience them § Example: ISP dial-in success rate § What: fraction of attempted calls that get connected § Metric: % success by hour, day, phone number § Tool: “war room” dialers that ring each number 20 x daily § Will also spot outside problems like telco outages § Best tools capture user results directly 5 www. mironov. com rich@mironov. com 650. 315. 7394

Quantify and Measure § Ongoing analytical effort § Operations and finance have best talent pools § Daily review with Operations VP § Charts, trends, incident reports § What happened? How can we automate prevention? Where can we simplify? § Weekly summary to Executive Staff Let’s follow an extended example. . . 6 www. mironov. com rich@mironov. com 650. 315. 7394

1. Customer-Reported Outages § When customers call in, you have already failed 7 www. mironov. com rich@mironov. com 650. 315. 7394

2. Design Metric and Test Tool § Goal: working 99. 5%, no transactions lost § Hours of operation: 8 AM ET to 6 PM PT § Tool: external “robot” submits request every 90 seconds § Notices simple failures… § T-1 down, power out, CPU down, disk full, account closed § Does not test everything… § Value of service, browser versions, data speeds, pricing, network security, tech support helpfulness 8 www. mironov. com rich@mironov. com 650. 315. 7394

3. Read the Charts 9 www. mironov. com rich@mironov. com 650. 315. 7394

4. Isolate and Fix One Problem § Something weird happens every fourth day… …a critical system is being rebooted 10 www. mironov. com rich@mironov. com 650. 315. 7394

5. Repeat § Prominent dips are power outages 11 www. mironov. com rich@mironov. com 650. 315. 7394

6. Turn the Numbers Sideways § Overloaded when most users connect: 9 AM-10 AM and 5 PM 12 www. mironov. com rich@mironov. com 650. 315. 7394

Measure Compulsively § Many causes of problems § Some outside your control § Improves your vendor selection § Opportunity for creative problem-solving § Root cause analysis replaces the “blame game” § Celebrate each time a milestone is reached § Entire organization begins to think about customers § Expand metrics over time 13 www. mironov. com rich@mironov. com 650. 315. 7394

Service Level Agreements § SLAs are owned by Marketing § Designed to win and keep customers § Competitive weapon, not neutral or objective § Loosely related to internal metrics § Keep customer SLA data separate from operational data § What is the penalty for missing SLA targets? § Financial risk balanced against marketplace risk 14 www. mironov. com rich@mironov. com 650. 315. 7394

Love Your Customers Word-of-mouth is the only route to service success, so. . . § § § Complaints are an opportunity Customer Service knows more than you do Pay novices to make mistakes Humility is a virtue First step is the hardest 15 www. mironov. com rich@mironov. com 650. 315. 7394

Summary § Decide what’s important § Define and measure it § Find and fix the biggest problem § Repeat § Love your customers § Be a star 16 www. mironov. com rich@mironov. com 650. 315. 7394