Fault and Performance Management Fault for Next Generation

Fault and Performance Management Fault for Next Generation IP IP Communication Next Alan Clark, Telchemy Alan

Outline • • • Problems affecting Vo. IP performance Tools for Measuring and Diagnosing Problems Protocols for Reporting Qo. S Performance Management Architecture What to ask for/ integrate?

Enterprise Vo. IP Deployment IP Phones IP VPN Branch Office Teleworker Gateway IP Phone

Vo. IP Deployment - Issues IP Phones ROUTE FLAPPING, LINK FAIL IP Phone IP VPN CODEC DISTORTION Gateway LAN CONGESTION, DUPLEX MISMATCH, LONG CABLES…. ECHO ACCESS LINK CONGESTION IP Phone

Call Quality Problems • • Packet Loss Jitter (Packet Delay Variation) Codecs and PLC Delay (Latency) Echo Signal Level Noise Level

Packet Loss and Jitter Buffer IP Network Codec Distorted Speech Packets lost in network Packets discarded due to jitter

Routers, Loss and Jitter Queuing delay Arriving packets Processing delay Input queue Queuing delay Serialization delay Output queue Prioritize/ Route Packet loss due to buffer Overflow or RED Voice packet delayed by one or more data packets

Queuing Delays Added delay due to wait for data packets to be sent = Jitter

Jitter Average jitter level (PPDV) = 4. 5 m. S Peak jitter level = 60 m. S

Wi. Fi can also cause jitter

Effects of Jitter • Low levels of jitter absorbed by jitter buffer • High levels of jitter o o lead to packets being discarded cause adaptive jitter buffer to grow - increasing delay but reducing discards • If packets are discarded by the jitter buffer as they arrive too late they are regarded as “discarded” • If packets arrive extremely late they are regarded as “lost” hence sometimes “lost” packets actually did arrive

Packet Loss Average packet loss rate = 2. 1% Peak packet loss = 30%

Packet Loss is bursty • Packet loss (and packet discard) tends to occur in sparse bursts - say 20 -30% in density and one second or so in length • Terminology o o o Consecutive burst Sparse burst Burst of Loss vs Loss/Discard

e los s Example Packet Loss Distribution tiv rst) Co n se cu bu e s r a sp 2 cent r e p 0 bu ity ( s n e rst d

Loss and Discard • Loss is often associated with periods of high congestion • Jitter is due to congestion (usually) and leads to packet discard • Hence Loss and Discard often coincide • Other factors can apply - e. g. duplex mismatch, link failures etc.

Example Loss/Discard Distribution

Leads To Time Varying Call Quality High jitter/ loss/ discard

Packet Loss Concealment Estimated by PLC • Mitigates impact of packet loss/ discard by replacing lost speech segments • Very effective for isolated lost packets, less effective for bursty loss/discard • But isn’t loss/discard bursty? • Need to be able to deal with 10 -20 -30% loss!!!

Effectiveness of PLC Codec distortion Impact of loss/ discard and PLC

Call Quality Problems • • Packet Loss Jitter (Packet Delay Variation) Codecs and PLC Delay (Latency) Echo Signal Level Noise Level

Effect of Delay on Conversational Quality

Causes of Delay Accumulate and encode Echo Control CODEC RTP IP UDP TCP Network delay Jitter buffer, decode and playout RTP IP UDP TCP CODEC Echo Control External delay

Cause of Echo Gateway IP Echo Canceller Round trip delay - typically 50 m. S+ Acoustic Echo Line Echo Additional delay introduced by Vo. IP makes existing echo problems more obvious Also - “convergence” echo

Echo problems • Echo with very low delay sounds like “sidetone” • Echo with some delay makes the line sound hollow • Echo with over 50 m. S delay sounds like…. Echo • Echo Return Loss o o 55 d. B or above is good 25 d. B or below is bad

Call Quality Problems • • Packet Loss Jitter (Packet Delay Variation) Codecs and PLC Delay (Latency) Echo Signal Level Noise Level

Signal Level Problems Amplitude Clipping occurs -- speech sounds loud and “buzzy” 0 d. Bm 0 -36 d. Bm 0 Temporal Clipping occurs with VAD or Echo Suppressors -- gaps in speech, start/end of words missing

Noise • Noise can be due to o o Low signal level Equipment/ encoding (e. g. quantization noise) External local loops Environmental (room) noise • From a service provider perspective - how to distinguish between o o room noise (not my problem) Network/equipment/circuit noise (is my problem)

Measuring Vo. IP performance Vo. IP Specific Active Test - Measure test calls Passive Test - Measure live calls VQmon ITU G. 107 VQmon ITU P. VTQ Analog signal based ITU P. 862 (PESQ) ITU P. 563

“Gold Standard” - ACR Test 4 3 2 2 • Speech material o o o Phonetically balanced speech samples 8 -10 seconds in length Test designed to eliminate bias (e. g. presentation order different for each listener) Known files included as anchors (e. g. MNRU) • Listening conditions o o Panel of listeners Controlled conditions (quiet environment with known level of background noise)

Example ACR test results • Extract from an ITU subjective test • Mean Opinion Score (MOS) was 2. 4 • • • 1=Unacceptable 2=Poor 3=Fair 4=Good 5=Excellent

Packet based approaches Test Call Vo. IP Test System IP Measure call Live Call Vo. IP End System VQmon, G. 107. P. VTQ Vo. IP End System IP Passive Test

Packet based approaches • ITU G. 107 o o R = Ro - Is - Ie - Id + A Really a network planning tool Missing many essential monitoring features • VQmon o o ITU G. 107 + ETSI TS 101 329 -5 Annex E +……. Proprietary but widely used (Superset of G. 107 & P. VTQ) • ITU P. VTQ o Available late 2005, very limited functionality

Extended E Model - VQmon 4 State Markov Model Gather detailed packet loss info in real time Arriving packets Loss/ Discard events Discarded Jitter buffer CODEC Signal level Noise level Echo level Metrics Calculation Call Quality Scores Diagnostic Data

Modeling transient effects Ie(burst) Measured Call quality User Reported Call quality Ie(VQmon) Ie(gap) 10 15 20 25 Time (seconds) 30 35

VQmon - computational model Burst loss rate Perceptual model Calculate R-LQ MOS-LQ Ie mapping Gap loss rate ETSI TS 101 329 -5 Signal level Noise level Calculate Ro, Is Echo Delay Calculate Id Recency model ITU-T G. 107 Calculate R-CQ MOS-CQ

Accuracy: Non-bursty conditions

Accuracy: Bursty conditions • G. 107 o o o • Well established model for network planning No way to represent jitter Few codec models Inaccurate for bursty loss Conversational Quality only VQmon o o o Extended G. 107 Transient impairment model Wide range of codec models Narrow & Wideband Jitter Buffer Emulator Listening and Conversational Quality on m Q V E Model Comparison of VQmon and E Model for severely time varying conditions

Signal based approaches Test Call Vo. IP End System P. 862 Tester IP Vo. IP End System P. 862 is an Active Test Approach Vo. IP End System IP P. 563 is a Passive Test Approach Vo. IP End System P. 563 Tester

ITU P. 862 - Active testing Tested segment of connection IP PESQ Audio files Time align FFT… Compare FFT… PESQ Score

ITU P. 862 - Active testing • Send speech file • Compare received file with original using FFT • Takes typically 50 -100 MIPS per call • MOS-like score in the range 0. 5 to 4. 5 • Widely used within the industry Results for G. 729 A codec for a set of speech files (i. e. for each packet loss rate the only thing changed is the speech source file)

ITU P. 563 - Passive monitoring • Analyses received speech file (single ended) • Produces a MOS score • Correlates well with MOS when averaged over many calls • Requires 100 MIPS per call Comparison of P. 563 estimated MOS scores with actual ACR test scores. Each point is average per file ACR MOS with 16 listeners compared to P. 563 score

Performance Monitoring - Passive Test Embedded Monitoring Function RTCP XR SIP Qo. S Report

SLA Monitoring - Active Test call Active Test Functions

Active or Passive Testing? • Active testing o works for pre-deployment testing and on-demand troubleshooting • But!!!! o IP problems are transient • Passive monitoring o o o Monitors every call made - but needs a call to monitor Captures information on transient problems Provides data for post-analysis • Therefore - you need both

Vo. IP Performance Management Framework Network Management System Call Server and CDR database Signaling Based Qo. S Reporting Network Probe, Analyzer or VQ Router Vo. IP Endpoint SNMP Reporting VQ VQ Vo. IP Gateway RTP stream (possibly encrypted) Embedded Monitoring Media Path Reporting (RTCP XR) Embedded Monitoring

Vo. IP Performance Management Framework • Embedded monitoring function in IP phones, residential gateways…. o o Close to the user Least cost + widest coverage • Protocol support developed o o RTCP XR (RFC 3611), SIP, MGCP, H. 323, Megaco Draft SNMP MIB • Works in encrypted environments • Already being deployed by equipment vendors

The role of RTCP XR (RFC 3611) 1. Provides a useful set of metrics for Vo. IP performance monitoring and diagnosis 2. Supports both real time monitoring and post-analysis 3. Extracts signal level, noise level and echo level from DSP software in the endpoint 4. Exchanges info on endpoint delay and echo to allow remote endpoint to assess echo impact 5. Provides midstream probes/ analyzers access to analog metrics if secure RTP is used 6. Goes through firewalls………

RFC 3611 - RTCP XR Loss Rate Discard Rate Burst Density Gap Density Burst Duration (m. S) Gap Duration (m. S) Round Trip Delay (m. S) End System Delay (m. S) Signal level RERL Noise Level Gmin R Factor Ext R MOS-LQ MOS-CQ Rx Config - Jitter Buffer Nominal Jitter Buffer Max Jitter Buffer Abs Max

SIP Service Quality Reporting Event PUBLISH sip: collector@example. com SIP/2. 0 Via: SIP/2. 0/UDP pc 22. example. com; branch=z 9 h. G 4 b. K 3343 d 7 ……… Content-Type: application/rtcpxr Content-Length: . . . VQSession. Report Local. Metrics: Time. Stamps=START: 10012004. 18. 23. 43 STOP: 10012004. 18. 26. 02 Session. Desc=PT: 0 PD: G. 711 SR: 8000 FD: 20 FPP: 2 PLC: 3 SSUP: on Call. ID=1890463548@alice. uac. chicago. com ……… Signal=SL: 2 NL: 10 RERL: 14 Quality. Est=RLQ: 90 RCQ: 85 EXTR: 90 MOSLQ: 3. 4 MOSCQ: 3. 3 Qo. EEst. Alg: VQMonv 2. 1 Dialog. ID: 38419823470834; to-tag=8472761; from-tag=9123 dh 311

RTCP XR MIB Session table History table Basic parameters Call quality metrics Alerting

Passive Monitoring Framework VQ VQ IP Phones VQ VQ VQ IP VPN VQ VQ SNMP NMS Branch Office VQ Gateway RT CP Teleworker XR SIP Qo. S Report VQ VQ IP Phone

What to Implement/ Ask For • Embedded monitoring functionality in IP Phones and Gateways (e. g. VQmon) • RTCP XR for mid-call data exchange between endpoints • SIP Service Quality Events for reporting end of call quality • RTCP XR MIB for SNMP support

Summary • • • Problems affecting Vo. IP performance Tools for Measuring and Diagnosing Problems Protocols for Reporting Qo. S Performance Management Architecture What to ask for/ integrate?