ITU Workshop on Voice and Video over LTE

ITU Workshop on “Voice and Video over LTE” Geneva, Switzerland, 1 December 2015 Considerations for end to end video quality Qo. E assessment as a means of verifying interoperability Paul Coverdale Consultant, Huawei Technologies Co. Ltd. coverdale@sympatico. ca

Outline • Background • Methodologies for assessing video Qo. E • Key requirements for an objective video Qo. E assessment tool • Current ITU-T Standards • Summary

Background • Today LTE is deployed by 360 operators in 124 countries. Out of them, 80 operators are investing in Voice over LTE (Vo. LTE) and 14 have already launched Vo. LTE services. • But Vo. LTE also allows for video over LTE (Vi. LTE) with managed quality of service, since the network is managed by the operator allowing them to prioritize real-time communication over data and reserve resources for specific services. • But how to ensure interoperability of Vi. LTE among different service providers?

Interoperability as perceived by the end-user • Important to consider interoperability from the viewpoint of the end-user – They are the one who pays the bill • The need to reliably establish and close a video session is clearly important, but so also is the video quality (Qo. E) during the session

Methodologies for assessing video Qo. E • In principle, assessment of Quality of Experience (Qo. E) must be performed using subjective tests, with metrics such as the mean opinion score (MOS) • However, it is also possible to estimate Qo. E based on objective measurements and associated quality estimation models • Subjective testing needs more resources and effort, because it requires human subjects, and is not so convenient in a live-service setting • Objective measurement and Qo. E calculation is generally much faster and more convenient, but the accuracy of the final evaluation depends on the accuracy of these models, and an understanding of the important human factors

surveilla nce TV call Vo. D Live Evolution of Video Quality Experience: Different Screens, Services, and Networks 1970 s– 1980 s Black&white Colorful Analog Digital Meeting basic requirements 1990 s-2010 s VCR 8 K 2020 s Plane Stereoscopic Static 3 D Dynamic 3 D (holographic) Multi-Screen Experience to Eye’s Extreme (Scope of the current experience standards) Interactive and immersed experience

Application scenario Performance Metrics Research method Requirements for objective video Qo. E assessment Subscriber survey Determine top factors and their weights. 1 3 Human factor engineering experiments Measure subjective perception. Experience modeling Network requirements BTV Vo. D Phone Pad Fixed network ITU-T Recommendations Determine the calculation method and formulae 2 4 Video communication TV Measurement indicators Network architecture Quantified collection Experience assessment Network assessment Security surveillance Camera Mobile network Continuous evolution Planning guidance

Subscriber Survey, Highlighting Top 3 Factors That Affect User Experience Subscriber survey Determine top factors and their weights. Top 3 factors that affect user experience: video source quality, video loading speed, and video view experience Weight importance Note: A larger weight value indicates greater importance. Interaction Performance Contents l l Focus group interview Questionnaire 3000+ survey samples l Video quality = Content quality (ultra HD, and SD) l Initial loading = Content loading duration (initial channel zapping or VOD loading response time) l View experience = Broadcasted video quality (frame freezing or blocking) Source: Huawei, UCD center, 2015

Experience targets Experience factors Video Qo. E Modelling Top Three Factors for Video Experience Video source Playback process Viewing process Video quality Interactive experience View experience 0 distortion High Frame definition Color rate Gamut 0 waiting Initial Loading time Zapping time 0 damage 0 Blocking 0 Stalling

Video MOS modeling factors Video Qo. E Assessment Panorama Video quality Interactive experience Viewing experience s. Quality* s. Interaction s. View Video MOS = f (s. Quality, s. Interaction, s. View) Application scenario * s=score BTV Vo. D Phone Pad Fixed network Video communication TV Security surveillance Camera Mobile network Objective multi-screen, multi-network, and multi-service(BTV, VOD Video communication Video surveillance etc) video experience standard

Video quality Video Quality Factors: Definition, Motion, and Fidelity Interaction experience Viewing experience The definition depends on viewing angle and resolution. 8000 The frame rate affects video motion. True colors Resolution: 7680 x 4320 PPI: 87 High fidelity means true colors, delicate pictures, and complete details. 100" 7000 5000 6" 60" Resolution: 5120 x 2880 PPI: 87 9. 7" Soap opera broadcast 42" Resolution: 3840 x 2160 PPI: 725 Resolution: 4096 x 2160 PPI: 87 2000 Typical viewing distance of a mobile phone with a 6" screen: 30 cm Typical viewing distance of an i. Pad with a 9. 7" screen: 41 cm Typical viewing distance of a TV screen: 2. 5 m Rec. BT. 2020 12 bits 8 bits 10 bits 12 bits SDR 8 bits HDR 10 bits HDR 12 bits Complete details 3000 Resolution: 5120 x 2880 PPI: 536 Rec. BT. 2020 10 bits Delicate pictures Marathon broadcast 6000 4000 Rec. BT. 709 8 bits Maximum frame rates for screen sizes

Video quality Interaction experience Viewing experience Interactive Response Factors: Video Loading and Switching Duration Interactive Operations and Acceptable User Experience 1 s channel switching 2 s video loading 0. 5 s fast forwarding/rewinding Ultimate Experience Objective: 0 Wait Time 100 ms = immediate response

Video quality Viewing Experience Factors: Erratic Display/Video Freeze Interaction experience Viewing experience Current criteria (VOD freeze): l l VOD freeze duration on mobile phone/pad screens <= 10% VOD freeze duration on TV screens <= 1% Current criteria (erratic display during live broadcast): l Resolution higher than 720 p: 1 erratic display/4 hours; l Resolution lower than 720 p: 1 erratic display/2 hours; Ultimate experience: 0 erratic display/video freeze l Same requirement for TV, mobile phone, and pad screens Source: DSL Forum TR-126 Ultimate user experience: 0 occurrence of video freeze and erratic display

Anticipated Video Service Experience Evolution The video industry technical can help the 3 top factors reach the following requirements. Video quality l TV: 1080 p @ 60 fps, 8 bit, ITU-R Rec. BT. 709 l TV: 4 k @ 120 fps, HDR, 12 bit, ITUR Rec. BT. 2020 l TV: 8 k @ 240 fps, HDR, 12 bit, ITU-R Rec. 2020 l Mobile phone: 720 p @ 30 fps, 8 bit, ITU-R Rec. BT. 709 l Mobile phone: 2 k @ 120 fps, HDR, 12 bit, ITU-R Rec. BT. 2020 l Mobile phone: 4 k @ 120 fps, HDR, 12 bit, ITU-R Rec. BT. 2020 l TV: Interaction experience l ü channel switching time < 1 s ü Initial VOD wait time: < 2 s Mobile phone: ü initial VOD wait time < 3 s l ü channel switching time < 500 ms ü Initial VOD wait time: < 1 s Mobile phone: ü Viewing experience l Initial VOD wait time: < 1 s ü channel switching time < 100 ms ü Initial VOD wait time: < 100 ms Mobile phone: ü initial VOD wait time < 100 ms l Number of erratic displays during a live video stream<= 2 l Number of erratic displays during live broadcast <= 1 l Number of erratic displays during live broadcast <= 0 l VOD freeze duration on mobile phone/pad screens <= 10% (within the 1 -minute statistical period) l VOD freeze duration on mobile phone/pad screens <= 5% (within the 1 -minute statistical period) l VOD freeze duration on mobile phone/pad screens <= 0% (within the 1 -minute statistical period) l VOD freeze duration on TV screens <= 1% (within the 45 -minute statistical period) l VOD freeze duration on TV screens <= 0% (within the 45 -minute statistical period) Now 2018 2020+

Current ITU-T Video Qo. E Models l l IPTV/Mobile TV Services HTTP Progressive Download (Rec. P. 1201/P. 1202) (Rec. P. 1201 Appendix III) Measures the quality degradation l due to compression l due to packet-loss l due to rebuffering Provides packet-level (P. 1201)/bit stream-level (P. 1202) assessment algorithm l l l Formulated by adding the initial buffering time and video freeze impact to the P. 1201 standard Without consideration of user interactions (such as pausing, seeking, user initiated quality change, user initiated play or user initiated end) are NOT considered at all

Summary • Successful interoperability of Vi. LTE depends on ensuring end-user satisfaction with the resulting video quality (high Qo. E) • Standardized objective models for predicting video Qo. E are becoming available, and will play an important rule in ensuring Vi. LTE interoperability