Performance Characterization Call Reliability Diagnosis Support for Voice

Performance Characterization & Call Reliability Diagnosis Support for Voice over LTE Yunhan Jack Jia, Qi Alfred Chen, Z. Morley Mao, Jie Hui†, Kranthi Sontinei†, Alex Yoon†, Samson Kwong†, Kevin Lau† University of Michigan, T-Mobile US Inc. † 1 1 The views presented in this paper are as individuals and do not necessarily reflect any position of T-Mobile.

Your voice call needs an upgrade Ø Data network evolution: Ø 2 G -> 3 G -> 4 G/LTE Ø Carrier’s voice call: Ø All circuit-switched before 2014 Ø Moving to a data-centric world Ø Voice over LTE Illustration: Serge Bloch

Voice over LTE ØDeliver voice service as data flows within LTE network Vo. LTE Packet-Switched Core Internet Circuit-Switched Core Telephony Network ENode. B Legacy call Node. B ØFor operators: reduce cost. ØPerformance benefit for users is unclear 1

Challenge 1: Guarantee Vo. LTE performance ØGuaranteeing Qo. S is challenging Default Bearer Internet User Dedicated Bearer ØHigh user expectation on Vo. LTE Ø Goal: Replacing legacy call 2 Gateway Bit rat: 50 kbps, Delay: 100 ms,

Challenge 2: Diagnose Vo. LTE problems ØVo. LTE is a complex service C\\\ LTE Coverage Constraints 3 G/2 G Network C\\\ Cross-layer Interaction C\\\ Device-network Interactions Multiple Layers LTE Network C\\\ Mobility Support ØExisting approach: User tickets Ø subjective, less accurate, coarse-grained 3

Problem statement * Definition: Quality of Experience (Qo. E) • Quality as seen by the end-user • E. g. , network call setup time vs. user perceived call setup time Ø Insufficient understanding of Qo. E of deployed Vo. LTE services Ø No effective support to capture and diagnose Vo. LTE problems 4

Contributions Ø Systematic study of Vo. LTE in commercial deployment Ø Qo. E quantification Ø Empirical comparisons with legacy call & OTT Vo. IP ØDiagnosis support for Vo. LTE reliability problems Ø Devise tool to capture audio experience problems efficiently Ø Covers three major symptoms in user tickets Ø Uncover potential causes lying in the Vo. LTE protocols Ø E. g. , Up-to-50 -second muting caused by mis-coordination between two different standards 5

Outline Ø Performance characterization Ø Methodology overview Ø Result summary Ø Diagnosis support for Vo. LTE reliability problems Ø Capturing audio experience problems Ø Audio quality monitor Ø Backend diagnosis engine Ø Stress testing approach & diagnosis Ø Case studies Ø Discussion 7

Methodology overview Vo. LTE service providers OP-III Legacy call Comparing entities Skype Hangouts Voice ØMetrics we study Ø Smooth audio experience Ø audio quality (MOS), mouth-to-ear delay and more Ø Energy consumption Ø Bandwidth requirement Ø Reliability Ø Call setup success rate Ø Call drop rate 8

Result overview Ø Vo. LTE delivers excellent audio quality with Ø Ø low bandwidth requirement less user-perceived call setup time low energy consumption won’t be affected by background traffic Ø Reliability still lags behind legacy call Ø Higher call drop rate (5 X) Ø Higher call setup failure rate (8 X) 9

Call reliability support of Vo. LTE ØChallenge: Unsatisfying and varying network conditions Ø Vo. LTE reliability support 2 G/3 G Core 2 G/3 G Ø Circuit-switched fall back Ø Single Radio Voice call Continuity IMS LTE Core LTE CSFB Procedure ØHowever, SRVCC Procedure Vo. LTE still fails to achieve a comparable reliability with legacy call Not all Vo. LTE problems are captured by traffic-analysis based approach 12

Audio quality monitor overview ØUse audio channel to detect Qo. E problems in real-time ØThree types of Vo. LTE reliability problems Ø Audio experience related problems Ø Muting, garbled audio, intermittent audio, one-way audio Ø Call setup failure Ø Unintended call drop Normal Voice Call Audio Quality Monitor Sampler Context Collector Muting Intermittent audio

Audio quality monitor evaluation Ø Implementation based on Android Audio. Record API Ø Accuracy: FP: 0. 65%, FN: 3. 7%. Ø Energy Overhead: +7% during Vo. LTE call Ø Complementary to traffic-based anomaly detection Ø Closer to user experience, easier to deploy. Ø Useful diagnostic tool for operators Ø Capture end-user audio problems objectively and accurately. More important: Understand the underlying causes of the problems 15

Stress testing approach & diagnosis Ø Motivation Ø Producing more problematic cases Ø Gathering critical logs in lab settings Audio Quality Monitor Device Logging Signal Strength Anomaly Detection Automation Multi-Layer Logs Cross-layer Diagnosis Network Logs Network Events Lab settings 20 Potential Causes

Diagnose long audio muting problem Ø Problem capturing Ø Ø Up-to-50 -second audio muting [Audio quality monitor] Triggered by signal strength degradation [Context collector] Ø Problem diagnosing Ø Gap between radio link layer timeout and RTP layer timeout Application RTP RRC RLC Control Vo. LTE call session Transmit voice packet stream Control the radio link connection Transmit low level protocol data unit

Lacking of coordination in cross-layer interactions RTP Timeout : Recommended minimum value = 360/bandwidth(kbps) 30 to 50 seconds! Application RTP Muting Start RRC RLC Radio Link Disconnection Muting End RTP Timeout Go to RRC_IDLE Reestablishment … Radio Link Failure Less than 5 seconds Max. Retx Threshold Radio Layer Timeout = RTT * max. Retx. Threshold + min{T 301, T 311} 25

Lacking of coordination in cross-layer interactions Ø RTP layer makes wrong assumption on the radio layer failure recovery Ø Cause: Gap between RTP (defined in RFC) and RRC/RLC (defined in 3 GPP) protocol Ø Also causing similar problems in Skype and Hangouts Ø Suggested solutions Ø Reporting radio link events directly to application layer Ø Other case studies detailed in the paper 26

Discussion Ø Limitation of diagnosis support Ø Coverage Ø Not fully automated Ø Follow-Up Ø Integrating OEM support for Qo. E problem diagnosis Ø Adding diagnosis support into protocols 27

Summary Ø First systematic study of Vo. LTE Qo. E in the commercial deployment Ø Provide diagnosis support for Vo. LTE Ø Audio quality monitor to capture problems Ø Stress testing approach to collect essential information Ø Cross-layer diagnosis support to understand problems 29

Thank you! Questions?