ITU Workshop on Quality of Service and Quality

ITU Workshop on “Quality of Service and Quality of Experience of Multimedia Services in Emerging Networks” (Istanbul, Turkey, 9 -11 February 2015) Monitoring End-User Qo. S for Data Services Alberto Pérez Business Development Consultant, Omnitele alberto. perez@omnitele. com

agenda ©Omnitele Ltd. 2015 2 1. Introduction − About Omnitele 2. Customer Experience Challenge KPIs that capture the End-User experience 3. Methodologies for Monitoring End-User Qo. S Different approaches – one goal 4. Take-outs Summary

Maximised Experience, Minimised Cost International And Independent We provide consulting and expert services for telecom operators and regulators in network strategy, design and quality assurance. Our mission is to maximise mobile subscriber quality of experience and minimise operator network expenditures. Our headquarters is located in Helsinki, Finland. We have local presence in the Netherlands and the Hashemite Kingdom of Jordan. Our company is owned by Finnish telecom investors and we are independent of operator groups and network vendors. Delivering Omnitele Experience The Omnitele Way The company was founded in 1988 to set up world’s first GSM network. Since then we have completed over 1000 projects in over 80 countries around the globe. Always delivering Omnitele Experience – a fact proven by our long lasting client relationships. Our unique way of working sets us apart from the competition and gives us a strong identity in the world of telecommunications. We call this the Omnitele Way, which means being Straightforward, Trusted and Intelligent. www. omnitele. com ©Omnitele Ltd. 2015 3

our services Technology Strategy Design and Optimisation Audit and Benchmark Performance Management maximised customer experience minimised network cost ©Omnitele Ltd. 2015 4

agenda ©Omnitele Ltd. 2015 5 1. Introduction − About Omnitele 2. Customer Experience Challenge KPIs that capture End-User experience 3. Methodologies for Monitoring End-User Qo. S Different approaches – one goal 4. Take-outs Summary

Monitoring End-User Qo. S in… …pre-HSPA networks General user experience depends on two basic services Voice Services Data Services MOS, Success Rates, Drop Rates, Setup Time Throughput ©Omnitele Ltd. 2015 6 Generally available bitrates are not sufficient for data services. As throughput is the bottleneck, it becomes the measure to define the Qo. S

Customer Experience Challenge Bitrate is no longer the ultimate mobile network quality indicator Qo. S: WWW page waiting time [s] 1000 KB WWW Page Waiting Time vs. Bitrate HSPA 10 LTE 8 6 4 2 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 NW Performance: Bitrate [Mbit/s] ©Omnitele Ltd. 2015 7

Customer Experience Challenge big difference in NW performance… …thin margins in Quality of Service bitrate [Mbit/s] 40 35 30 25 20 15 10 5 0 buffering time [s] 2 1. 5 1 0. 5 0 T-Mobile Vodafone KPN T-Mobile Vodafone higher bitrate, but KPN faster You. Tube ©Omnitele Ltd. 2015 8 Vodafone KPN

Customer Experience Challenge QUALITY OF EXPERIENCE How well service quality meets expectations? overall Satisfaction, e. g. 1 -5 ”stars” Highly market dependent and user segment specific QUALITY OF SERVICE How well does a service perform Measured application/service KPIs product of one or more network performance factors NETWORK PERFORMANCE Technical measurements of the network e. g. , delay, jitter, packet loss, throughput. . . ©Omnitele Ltd. 2015 9 RSCP Ec/N 0 latency jitter packet loss RSRP RLC throughput RSRQ RAB setup success rate modulation coding TX power G -factor Channel C/I Timing Advance Mobile TX Power Voice Codec Usage Handover Success Rate MCS Usage Distribution Time Slot Utilization Block Error Ratio Active Set Size SHO Success Rate ISHO Success Rate CQI E -DPDCH Throughput PDSCH modulation MAC DL BLER MAC UL Retransmission Rate

Customer Experience Challenge all-IP PS network OFDMA access scheme spectral efficiency 15 bit/s/Hz SC-FDMA uplink adaptative modulation low latency 300 Mbps 20 MHz bandwidth Carrier Aggregation improved cell-edge performance 8 x 8 MIMO ©Omnitele Ltd. 2015 10

Customer Experience Challenge all-IP PS network news refreshing time? OFDMA access scheme app download spectral efficiency 15 bit/s/Hz time? SC-FDMA uplink it takes too long to adaptative upload a pic in modulation Instagram… low latency 300 Mbps ©Omnitele Ltd. 2015 11 20 MHz bandwidth facebook Carrier Aggregation refreshing improved cell-edge performance time? 8 x 8 MIMO

agenda ©Omnitele Ltd. 2015 12 1. Introduction − About Omnitele 2. Customer Experience Challenge KPIs that capture End-User experience 3. Methodologies for Monitoring End-User Qo. S Different approaches – one goal 4. Take-outs Summary

Monitoring Methodologies in mobile networks pasive monitoring PMS/Network Counters CDR Crowdsourcing ©Omnitele Ltd. 2015 13 active monitoring Probes (static/mobile) Drive Testing

Monitoring Methodologies in mobile networks pasive monitoring PMS/Network Counters CDR Crowdsourcing ©Omnitele Ltd. 2015 14 Vast amount of data 24/7 live monitoring Big picture of network utilisation trends Not available KPIs on End-User level Challenging implementation (Crowdsourcing) Average results do not tell much

Monitoring Methodologies in mobile networks active monitoring Probes (static/mobile) Drive Testing ©Omnitele Ltd. 2015 15 Controlled environment Reproduces End-User cases Collects essential data for troubleshooting Amount of samples is limited Limited coverage information Expensive exercise (Drive Testing)

Monitoring Methodologies best methodology? PMS/Network Counters Drive Testing Probes End-User Monitoring Crowdsourcing a combination of all of them ©Omnitele Ltd. 2015 16 CDR

Monitoring Methodologies set up a process ©Omnitele Ltd. 2015 17 4. Correlate data from different sources 1. What do I want to Monitor? 3. What tools do I have available? 2. How my subscribers use the service?

”Be-the-Customer” Approach Methodology for capturing the true end-user experience when using mobile networks WHERE | Dense population areas independently selected by Omnitele WHEN | Test focus on peak hours, no empty network testing HOW | Terminals and test cases as per real subscriber behaviour ©Omnitele Ltd. 2015 18

Latvia Benchmark – So. W Country-wise Benchmark 3 Operators VS ©Omnitele Ltd. 2015 19

Latvia Benchmark – So. W Hotspot Testing Web Browsing You. Tube Drive Testing Facebook File Transfers Instagram Twitter Dropbox App Testing Network Performance Testing Tests to be carried out with latest Samsung Galaxy S terminal (4 G/3 G/2 G floating mode) ©Omnitele Ltd. 2015 20 Voice Calls

Latvia Benchmark – So. W Hotspot Testing Drive Testing 212 locations +2 000 km Hotspot to Hotspot Routes Main Connecting Roads all towns +10 k inhabitants ©Omnitele Ltd. 2015 21

agenda ©Omnitele Ltd. 2015 22 1. Introduction − About Omnitele 2. Customer Experience Challenge KPIs that capture End-User experience 3. Methodologies for Monitoring End-User Qo. S Different approaches – one goal 4. Take-outs Summary

Summary 1. Data throughput is not longer the unique KPI to take into consideration when monitoring Mobile Data Qo. S 2. Different Monitoring Methodologies give you insights to different type of data – everything is valuable! 3. Set a process that helps achieving your goals, taking into consideration the actual users of the service ©Omnitele Ltd. 2015 23

thank you for your attention! questions ©Omnitele Ltd. 2015 24

We answer Contact details Omnitele Ltd Name Alberto Pérez Phone +358 9 695 991 Title Business Development Consultant Email contact@omnitele. com Phone +358 40 537 80 52 Website www. omnitele. com Email alberto. perez@omnitele. com Address Omnitele Ltd. Mäkitorpantie 3 B P. O. Box 969 00101 Helsinki, Finland ©Omnitele Ltd. 2015 25

max imi se d cus tomer exp erience minim is ed ne twork cost ©Omnitele Ltd. 2015 26 www. omnitele. com