Workshop on Monitoring Quality of Service and Quality

Workshop on “Monitoring Quality of Service and Quality of Experience of Multimedia Services in Broadband/Internet Networks” (Maputo, Mozambique, 14 -16 April 2014) Update on ITU-T & ETSI standardization activities for LTE Qo. S / Qo. E Dr. Peter Počta Peter. Pocta@fel. uniza. sk University of Žilina, Slovakia Joachim Pomy Consultant@joachimpomy. de OPTICOM, Germany Maputo - Mozambique - 14 - 16 April 2014 Version : 11 December 2008

§ ITU-T SG 12, Question 11: Ø"Performance interworking and traffic management for Next Generation Networks " Maputo - Mozambique - 14 - 16 April 2014 2

G. Vo. LTE (1) ØC-127: Inputs for G. Vo. LTE (Huawei Technologies Co. Ltd. , France Télécom Orange) ØAt last SG 12 meeting, two contributions (C. 074 from Orange and C. 076 from Ascom) addressed the need for starting the development of specific requirements on end-to-end Qos for voice over 4 G mobile networks. The decision has been taken to develop a new Recommendation, provisionally called G. Vo. LTE, under Q. 11 (see the report R. 4 of WP 3), with the following description: Maputo - Mozambique - 14 - 16 April 2014 3

G. Vo. LTE (2) ØThis Recommendation describes the key aspects impacting end-to-end performance of managed voice applications over LTE networks. Maputo - Mozambique - 14 - 16 April 2014 4

G. Vo. LTE (3) ØThis Recommendation makes assumptions about relevant Qo. S mechanisms used to manage the voice service, and studies the impact of such mechanisms on the end-to-end network performance. Some typical end-to-end scenarios are described, involving cases with LTE access at both sides of the communication, or with a different access technology at one side (wireless or wireline access). Maputo - Mozambique - 14 - 16 April 2014 5

G. Vo. LTE (4) ØA typical reference connection is defined in this Recommendation, composed of various segments, including: terminal, wireless access, backhaul network, core network. Considerations regarding the sharing of the budget of some key parameters across these segments, such as latency, are provided. Maputo - Mozambique - 14 - 16 April 2014 6

Qo. S of connections from current technologies to LTE for delay sensitive applications (ETSI STF 437) Maputo - Mozambique - 14 - 16 April 2014 7

Outline Introduction of the ETSI STF 437 project (scope, team), § Qo. S problems encountered by operators, § Shortcomings of standards and implementations, § Possible solutions (actions for standards and implementations), Maputo - Mozambique - 14 - 16 April 2014 8

Introduction of the ETSI STF 437 § Project scope: Ø address LTE related Qo. S problems for delay sensitive applications, Ø determine possible shortcomings of the standards and possible shortcomings of implementations Ø propose possible solutions Maputo - Mozambique - 14 - 16 April 2014 § Project team: Ø Peter Počta (leader) § University of Žilina, Slovakia Ø Jan Holub § Mesaqin. com s. r. o. (Ltd. ), Czech Republic Ø Joachim Pomy § Telecommunications & Int’l Standards, Germany Ø Catherine Quinquis § France Telecom, France 9

Qo. S problems encountered by operators § Contacted operators and stakeholders only trial LTE technology § Trials mostly focus on a performance of data services (lab implementation). § One of the contacted operators focused on voice service in its trial, following problems have been encountered: § Some problems with CS Fallback (was not working properly), § Problem with routing between radio part (LTE network) and IMS, § All problems have been fixed by vendor (wrong implementation). § Conclusion: operators have not already started to deploy Voice over LTE in their networks. § To sum up current situation, according to market observers and visionaries: § Network operators are planning to deploy Vo. LTE in upcoming years (2013 or 2014). § The potential subscribers of LTE expect better performing data services than in 3 G and services that take advantage of the lower latency, such as voice and video. (Quality of service – crucial role in an adaptation process of Vo. LTE). § Implication: Qo. S should be properly implemented and tested in trials to ensure sufficient quality level to speed up/drive the adaptation process of Vo. LTE. Maputo - Mozambique - 14 - 16 April 2014 10

Shortcomings of standards and implementations § Shortcomings of standards: Ø Standards focusing on IMS Multimedia Telephony, IMS Profiles for voice service, SMS and conversational video service: § According to ETSI TS 126 114, the MTSI client should only support AMR codec for voice service. § The jitter-buffer management mechanisms of video service described in the ETSI TS 126 114 seem to be very weak. § The quality of experience metrics defined in clause 16 of ETSI TS 126 114 can be more or less only considered as Qo. S parameters. Maputo - Mozambique - 14 - 16 April 2014 11

Shortcomings of standards and implementations § Shortcomings of standards: Ø Standards describing resource and admission control mechanisms developed by 3 GPP and ETSI (3 GPP Policy and Charging Control (PCC) and TISPAN Resource and Admission Control Subsystem (RACS)): § 3 GPP PCC and TISPAN RACS vary considerably in architecture, supported networks and node types (big functional differences). § To harmonize them, a following activity called “Framework for Gq' / Rx harmonization” was started in 2007 by 3 GPP. § In 2009, the harmonization activity mentioned above has been abandoned and all published specifications has been withdrawn (the developed solutions has been lost). § Main problem: no interface is available for communication between the TISPAN RACS and the 3 GPP PCC. Consequently, they can not interact and negotiate with each other in order to finish the operations required for continuous Qo. S guarantee (e. g. resource reservation). Maputo - Mozambique - 14 - 16 April 2014 12

Shortcomings of standards and implementations § Shortcomings of implementations: Ø Interim solutions: § Circuit-switched fallback (CSFB): Uses 2 G/3 G for voice and requires interruption of the LTE connection; this is already affected by the signalling of an incoming call. § Voice over LTE Generic Access (Vo. LGA): Tunnels the voice call from the 2 G/3 G to the LTE terminal via an LTE data channel; requires additional equipment in the network; lack of Qo. S. § Fast Track Vo. LTE: Proprietary solution providing voice service via the LTE technology; adds SIP stack to existing mobile network equipment. § Over-the-Top solutions (OTT): Vo. IP applications operating via an LTE data channel, running on the LTE terminal; lack of Qo. S. Ø Migration of voice service to LTE: § Before commercial deployment, a number of lab, field and market trials should be conducted. § User speech quality expectations are very high. If voice services do not deliver the necessary level of quality and reliability, users will revert back to existing circuit-switched options or, in some cases, simply rely on over -the-top solutions. Maputo - Mozambique - 14 - 16 April 2014 13

Shortcomings of standards and implementations § Shortcomings of implementations: Ø Vo. LTE Interop testing: § was organised by the Multi-Service Forum (MSF) and backed by the GSMA in September 2011 and October 2012. § a couple of different aspects has been tested, mainly focusing on signalling issues, multivendor interoperability. § MSF reports in the white paper summarizing Vo. LTE Interop event 2011: " The event proved that Vo. LTE is a viable solution for providing voice services over LTE access technology, and can be deployed with roaming and interconnect functionality to provide equivalent service as CS-based voice services today. “ – What exactly has been tested? – What kind of understanding of the terms "Quality of Service" and "service" have been prevalent? § After a detailed view into the scenarios of this interoperability event, the STF 437 team has concluded that Qo. S of the media channel is not tested at all. Maputo - Mozambique - 14 - 16 April 2014 14

Possible solutions (actions for standards and implementations) § Actions for standards: Ø Codec aspects: § In principle, there are three possible solutions: – Tandeming (lower quality and increased delay), – End-to-end connection based on AMR codec (lower quality), – using other codecs supported by IMS for end-to-end connections. § STF 437 team advice: MTSI client defined in ETSI TS 126 114 should have the possibility to access every codec supported by SIP. § Action point: appropriate standard should be extended accordingly. Ø Jitter-buffer Management Mechanisms of video service: – are currently not or at least not sufficiently standardized. Further study in this area is of critical importance. Maputo - Mozambique - 14 - 16 April 2014 15

Possible solutions (actions for standards and implementations) § Actions for standards: Ø Quality of Experience metrics: § There are still no clear standards in place, which can support network operators in selecting those parameters in the Qo. E extension. § It is proposed by STF 437 team to follow the approach of a Human Factors Extension to the Seven-Layer OSI Reference Model proposed by Bauer and Patrick: – Main idea: three additional HCI (Human Computer Interface) layers capture the additional quality parameters introduced by the concept of Qo. E over Qo. S, whereas the classical 7 OSI layers are synonymous to classical Qo. S parameters. – The above mentioned three additional layers focusing on HCI are: – Layer No. 8: Display (keyboard, GUI/CLI, vocal, Bpp, Ppi, Ppm, etc. ), – Layer No. 9: Human Performance (perception, cognition, memory, motor control, social, etc. ), – Layer No. 10: Human Needs (communication, education, acquisition, security, entertainment, etc. ). Maputo - Mozambique - 14 - 16 April 2014 16

Possible solutions (actions for standards and implementations) § Actions for standards: Ø Framework for Gq’/Rx harmonization: § Crucial to reopen the activity called “Framework for Gq’/Rx harmonization”: – define interactive interface supporting sufficient exchange of information between TISPAN RACS and 3 GPP PCC – harmonize the communication between the two different resource and admission control solutions to satisfy the subscribers’ service experience Maputo - Mozambique - 14 - 16 April 2014 17

Possible solutions (actions for standards and implementations) § Actions for implementations: Ø Migration of voice service to LTE: § Delay values achieved in trials and real implementations have to be carefully observed. § Sufficient algorithms/techniques should be deployed in the networks in order to ensure continuous data sessions for incoming and outgoing voice/video calls. § A quality of service perceived by the end-user should be carefully measured in trials and monitored in real implementations to ensure the best quality provided to the users. Ø Vo. LTE Interop testing: § TC INT has begun to consider inclusion of Qo. S testing aspects into the specifications forthcoming Vo. LTE interoperability events. § STF 437 team has proposed an example of improved test scenario presented on next slide (Fig. 1). § While this is a good starting point, the challenge of Qo. S in heterogeneous scenarios interconnecting current technologies with LTE may require additional effort in the future. § Another option: the organization of a pure Speech & Video Quality Test Event for LTE. Maputo - Mozambique - 14 - 16 April 2014 18

Possible solutions (actions for standards and implementations) Figure 1: Example of improved test scenario for Vo. LTE interop test Maputo - Mozambique - 14 - 16 April 2014 19

TOWARDS QUALITY AND QOS ASSESSMENT FOR VOLTE IN ETSI TC INT Chair: Giulio Maggiore, Telecom Italia giulio. maggiore@telecomitalia. it Maputo - Mozambique - 14 - 16 April 2014 © ETSI 2013. All rights reserved 20

TC INT (Core Network and Interoperability Testing) § Development of Core Network test specifications (interoperability, conformance, performance, security) based on 3 GPP and E 2 NA specifications. § To initiate and supervise interoperability events (such as Plugtests) related to Core Networks as well as other events (workshops and seminars) § To coordinates interoperability efforts with other organisations GSMA, OMA, IMS/NGN Forum, IMTC, etc. § Endorsement of test specifications from/to other SDOs e. g. ITU-T. Maputo - Mozambique - 14 - 16 April 2014 21 © ETSI 2013. All rights reserved 21

TC INT Interoperability Specification Work R 6 TS 186 011 V 1. 1. 1 2007 R 1 R 7 TS 186 011 V 2. 1. 1 2008 R 2 Late R 7 TS 186 011 V 2. 2. 1 2009 Late R 2 Maputo - Mozambique - 14 - 16 April 2014 22 R 8 TS 186 011 V 3. 1. 1 TS 102 901 V 1. 1. 1 2010 R 9 R 10 TS 186 011 V 4. 1. 1 TS 102 901 V 2. 1. 1 TS 186 011 V 5. 0. 2 TS 102 901 V 5. 0. 6 TS 103 129 V 2. 0. 4 2011 2012 2013 E 2 NA 22

2013 – VOLTE QOS ASSESSMENT TECHNOLOGY EVALUATION EVENT ETSI STF#453 STF Leader: Peter Schmitting, FSCom peter. schmitting@fscom. fr Maputo - Mozambique - 14 - 16 April 2014 23 © ETSI 2013. All rights reserved 23

Vo. LTE Qo. S Assessment event overview § Scope: Technology evaluation Plugtests aiming at validating the different methodologies for end to end Qo. S Assessment for Vo. LTE described in ETSI TS 103 189 over a deployed LTE/IMS network. § Dates: 18 -20 November 2013 § Venue: ETSI, Sophia Antipolis, France § Web: http: //www. etsi. org/news-events/698 -volte-qosassessment-2013 Maputo - Mozambique - 14 - 16 April 2014 24 © ETSI 2013. All rights reserved 24

Vo. LTE Qo. S Assessment event scope § End to end tests performed with several different UEs over a deployed LTE / IMS test network. § 4 Test Setups Ø Ø Acoustic to acoustic Acoustic to electrical Electrical to acoustic Electrical to electrical § Voice quality tests based on instrumental assessment of speech samples performed on different combinations of the acoustic and electrical interfaces which allows evaluating the different methodologies and setups. § Parallel network performance evaluation Maputo - Mozambique - 14 - 16 April 2014 25 © ETSI 2013. All rights reserved 25

2013 – Vo. LTE Qo. S Assessment Technology Evaluation event – test description TS 103 189 § End-to-end Tests based on Instrumental Assessment of Speech Samples (POLQA model deployed): Ø Ø Mean Opinion Score (MOS)-Listening Quality Objective Delay variation over time Difference in loudness level MOS stability § End-to-end Tests based on Instrumental Assessment of Video Samples (PEVQ model deployed): Ø Ø Ø DMOS (Difference Mean Opinion Score) Delay and Quality variation over time Jerkiness and Blockiness Blur, Chrominance and Luminance Temporal Distortion Frozen and Skipped Frames Maputo - Mozambique - 14 - 16 April 2014 26

2013 – Vo. LTE Qo. S Assessment Technology Evaluation event – test description TS 103 189 § Other end-to-end tests in the voice channel: Ø Ø Ø End-to-end frequency response Overall loudness rating End-to-end delay Quality of echo cancellation Double talk performance § End-to-end Tests based on functional parameters: Ø Telephony Call Setup Time Ø Streaming Service Access Time § Network Performance Parameters Ø UNI-to-UNI delay over time Ø UNI-to-UNI jitter Ø UNI-to-UNI packet loss Maputo - Mozambique - 14 - 16 April 2014 27

28 Vo. LTE Qo. S Assessment – Setup 1 Acoustic-to-Acoustic Maputo - Mozambique - 14 - 16 April 2014 © ETSI 2013. All rights reserved 28

Vo. LTE Qo. S Assessment – Setup 2 Acoustic to Electrical Maputo - Mozambique - 14 - 16 April 2014 29 © ETSI 2013. All rights reserved 29

Vo. LTE Qo. S Assessment – Setup 3 Electrical to Acoustic Maputo - Mozambique - 14 - 16 April 2014 30 © ETSI 2013. All rights reserved 30

Vo. LTE Qo. S Assessment – Setup 4 Electrical to Electrical Maputo - Mozambique - 14 - 16 April 2014 31 © ETSI 2013. All rights reserved 31

What’s next? § Validation of 76% of the test cases in TS 103 189 § A number of gaps and potential improvements in TS 103 189 have been identified § A new work item has been open for the update of TS 103 189 with the new setup, acoustic-to-electrical identified during the event § Next event will be launched when Vo. LTE Native Handset and PEVQ tools will be available for testing Maputo - Mozambique - 14 - 16 April 2014 32 © ETSI 2013. All rights reserved 32

Further steps with regard to ETSI TS 103 189 § Verification of remaining voice quality test cases Øtbd § Verification of video quality test cases (RTPbased video streaming): ØTest houses willing to do so are needed § Extension of video quality test cases towards MPEG-DASH ØITU-T SG 12 Question 14 currently running a WI called P. NATS Maputo - Mozambique - 14 - 16 April 2014 33

ITU-T SG 12 Question 14 Project P. NATS § P. NATS focusing on a development of a model Ø consisting of a collection of objective parametric quality assessment modules Ø predicting the impact of observed IP network impairments on quality experienced by the end-user § in multimedia mobile streaming and fixed network applications § using progressive download, including adaptive streaming methods Ø The development is divided into 2 parts, namely: § Track No. 1: bitstream-based parametric models (payload and/or header information) § Track No. 2: full-reference and hybrid models for the video coding quality module. Ø Track No. 2 is of our interest as it correlates well with an aim of Vo. LTE Qo. S Test event Ø Models for both tracks are supposed to be available at the end of this year. Maputo - Mozambique - 14 - 16 April 2014 34

Any questions ? Contact: Consultant@joachimpomy. de Maputo - Mozambique - 14 - 16 April 2014 35