ITUTQSDG DUBAI ASCOM PRESENTATION QOS AND QOE IN

[ ] ITU-TQSDG - DUBAI ASCOM PRESENTATION: QOS AND QOE IN LTE MOBILE NETWORKS Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014

Who We Are Independent Auditor +40 1 patents worldwide Next-generation technology 650 Customers Globally More than 30 patents worldwide Innovative Leader Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014 Wireless network performance ITU-T ETSI VQEG Formative Player

AGENDA § LTE Qo. E- Qo. S- KPI MAPPING RELATIONSHIPS § CHALLENGES EMERGING FROM LTE § ASCOM LTE Qo. S/Qo. E TESTING APPROACHES § SHARING EXPERIENCE: SOME USE CASES § CONCLUSIONS Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014

[ ] LTE QOE-QOS KPI MAPPING RELATIONSHIPS Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014

ASCOM’s APPROACH ON Qo. S/Qo. E CYCLE => SAME AS ITU’s (ITU-T G. 1000) MNO Subscriber Qo. E Requirements of Subscriber Qo. S Targeted by MNO (Target SLA) Execution gap Value gap Qo. E Perceived by Subscriber Alignment gap Qo. E/Qo. S Cycle Perception gap KPIs, Qo. S TEMS Qo. S Delivered by MNO (Achieved SLA) Qo. E Subscriber-centric Quality (Qo. E) Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014 TEMS Network-centric Quality (Qo. S)

LTE Qo. E- Qo. S - KPIs MAPPING Network Performance Customer Experience Qo. E Application Layer (accessibility, retainability, integrity) Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014 Qo. S KPIs Upper Layers (IP/UDP, IP/TCP) (throughput, delay Layer 2&3 MAC, RLC, LLC (reports related to Qo. E dimensions = PDP context, HO info, codecs) Physical layer (coverage, interference)

COST EFFICIENT TOP DOWN APPROACH ENABLED BY PREDEFINED LTE Qo. E-Qo. S-KPIs MAPPINGS Increased operational efficiency Top-down customer experience centric approach Voice Service: ‘ 97 Auryst, ‘ 03 PESQ, ‘ 12 POLQA Video-Audio Services: ‘ 08 VSQI/MTQI, ’ 12 PEVQ, ‘ 12 VQmon Application Layer (accessibility, retainability, integrity) Qo. E Upper Layers (IP/User Data Protocol , IP/Transport Control Protocol): Throughput, Delay, Packet Loss, Rebuffering Qo. S Layers 2 & 3 messaging reports related to Qo. E dimensions like PDP context, HO info, codec usage Network performance KPIs Physical layer details like coverage and interference Reduce troubleshooting time by using Qo. E centric mapping to Qo. S and KPIs per service type Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014 7

[ ] CHALLENGES EMERGING FROM LTE Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014

Vo. LTE-Vi. LTE-RCS EXPERIENCE Call Quality as Perceived by Subscribers Codecs § MOS / ITU-T P. 863; J. 247 & P. 120 x. x based (video) (KPI) § On device Vo. LTE client: re-buffering/time scaling for voice; error concealment type for video § Voice path delay (“mouth to ear”), echo, video – voice lip sync § RTP Packet loss, latency, jitter § HOIT (LTE HO, e. SRVCC) Devices Network Clients (Qo. S /KPI) (KPI) Call Experience (Qo. E) § Throughput § Voice /video codec type and bit rates Call control performance § Session set-up: SIP signaling statistics / IMS Registration, RTT § Session Accessibility, Retainability § QCI allocation verification § LTE RRC connection and HO statistics Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014

MOBILE VIDEO STREAMING EXPERIENCE Codec types: Devices & Clients § High § Different form factors § Low § Different 2 D/3 D displays § Adaptive bit rates § Different adaptive error concealment schemes Network centric: Service-Centric § Packet loss Wide variety of content & bit rates § Jitter § Allocated GBR and QCI § HTTP/TCP vs. RTP/RTSP Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014

Vo. LTE Qo. E AND ITS ROOT CAUSES (Qo. S, KPIs SOURCES) Perceived frequency Spectrum (Qo. E) Interruptions (Qo. E) (incl. time clipping) Network (Qo. S): Limited Bandwidth Network (Qo. S/KPIs) Device (KPIs): Spectral shaping Reverberations Device based signal processing (KPIs) (e. g. (IP/IMS loss, jitter, RAN erroneous bits, RAN HO) NR, EC) Codec/client (KPIs): PL concealment schemes Aggressive VAD schemes Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014 “Mouth to ear delay” (Qo. E) Network (Qo. S) (IMS path can be key contributor) Codec/device signal processing (KPIs) Noisiness (Qo. E) (incl. musical noise) Network (Qo. S): Limited BWD with noisy speech (speech contamination) Device (KPIs): Imperfect NR (musical noise) Loudness during silent periods Non-optimal loudness levels Codec/client (KPIs) PLC interpolation based (“additive artifacts”) Perceived call session performance (Qo. E) IMS network (Qo. S/KPIs): SIP statistics, IMS registration ITU-T on going work: G. Vo. LTE, P. TCA (Technical Cause Analysis)

MOBILE VIDEO STREAMING Qo. E AND ITS ROOT CAUSES (Qo. S, KPIs SOURCES) Low throughput (limited bandwidth) Qo. S Packet loss, discard, late arrival IP delays / jitter KPIs Possible transcoding Video resolutions/re-scaling Encoding/transcoding rates Compression schemes KPIs Limited coverage, interference Bandwidth estimation algorithms impact the bit rate selection Content server load, (competing video streams) Content complexity (variable movement) KPIs Client under/over flow; improper buffer lengths/adaptation length Initial buffering settings Display resolutions, form factors Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014 Visual impairments: Blockiness, Bluryness, Jerkiness, Freezing with and/or without skipping Perceived service accessibility / access time Qo. E

[ ] ASCOM LTE QOE/QOE TESTING APPROACHES Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014

UNDERSTAND USE ITU-T BASED Qo. E METRICS BEST FITTED TO THE SERVICE BEING TESTED LTE VOICE AND MOBILE VIDEO SERVICES LTE Broadcast Streaming server Video stream (You. Tube, e. MBMS) Encoding Cellular Voice & Data Network IP recordings (RTP KPIs/Qo. S) synchronized with RAN KPIs Solutions Voice: MOS P. OLQA Video: MOS PEVQ Decoding Conversational Video Voice/Conversational Video stream IP transport/payload parameters Full Reference listening media quality evaluation (intrusive, perceptual) measurement Direct RTP KPIs/Qo. S Solutions Voice: MOS - ITU P. 564 based Video-audio: MOS – P. 120 x. x based (VQmon) Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014 MOS, media based KPIs Non Reference listening media quality evaluation (non-intrusive, parametric) MOS, network based KPIs

[ ] SHARING EXPERIENCE: SOME USE CASES Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014

Vo. LTE MOS SCORE and Vo. LTE KPI’s/Qo. S TEMS Discovery Client Network PIs : K sure Mea TEMS Investigation TEMS Pocket Deliver Vo. LTE Stack Optimization POLQA; speech path delay, volume, echo Client Information (re-buffering, codec) ESM configuration (QCI, Ro. HC) Real time IP trace & L 3 logging IP recording (RTP stats), SIP stats RAN (HOIT, Scheduling, RSRP, CINR, CQI, PMI/RI, UE category, MTU Size, Protocol stack configuration) Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014 Qo. E Evaluation & Troubleshooting MOS Speech & Client centric reasons Network centric reasons Speech

ADAPTIVE STREAMING Qo. E AND Qo. S/KPIs Dynamically adaptive HTTP/TCP Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014 ETSI TS 102. 250 -2, ETSI TR 101. 578 Measurement Guidance for TCP based video services

[ ] CONCLUSIONS Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014

A FULL Qo. E/QOS CYCLE SOLUTION FOR LTE NETWORK AND SERVICES My Customer Experience Data Automated data correlation and presentation My competitors’ customers (Benchmark) Providing network and customer experience centric diagnoses for voice (Vo. LTE, OTT, Vo. HSPA), and video-audio services (OTT, RCS-e/Joyn) My customers, real field agents Automated “what, ” “why” and engineertrusted “how” scenarios Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014 Built-in best practices scripted data analytics provide automated root cause analysis

CONCLUSIONS § PLEASE VISIT OUR WEBSITE: http: //www. ascom. com/nt/en/index-nt/about-us-network-testing/nt-about-usresources. htm/ White papers: Vo. LTE, Video Streaming, Het. Nets, Carrieir Aggregation and. . . watch the space: LTE Broadcast testing to come soon Informa Webinar: Advanced testing with Ascom in LTE networks Webinars: Vo. LTE, Video Streaming Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014

[ ] THANK YOU Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014