Repeatability of Video Stream Traffic Generation Using Internet
Repeatability of Video Stream Traffic Generation Using Internet and Local Sources June 2018
Prior Internet Video Steaming Results – In SE 45(18)14, HPE presented several internet video streaming measurements conducted with You. Tube and Netflix over the public Internet – With You. Tube 1080 P, the resulting average data rate was 4. 5 Mbps and peak data rate was 215 Mbps – With Netflix 1080 P, the resulting average data rate was 8. 0 Mbps and peak data rate was 300 Mbps – Streaming started with a 3 sec high data rate initial buffering period – For You. Tube, subsequent burst period was about 10 seconds – For Netflix, subsequent burst period was about 4 seconds You. Tube 1080 P. Bitrate (Mbps) - Average: 4. 5, Peak: 215 Netflix Video Bitrate 4420. Bitrate (Mbps) - Average: 8. 04, Peak: 300 2
Internet Streaming Issue #1 - Repeatability – The identical You. Tube capture was attempted in May with different results – The average data rate dropped to 4. 0 Mbps, and the peak data rate dropped to 168 Mbps. – Furthermore, the burst periods were 5 seconds. – Investigation revealed multiple potential sources of variation: – Client Device Hardware and Concurrent Streaming Count – You. Tube adapts the video to the clients bandwidth, display, and processing capability. (https: //youtube-eng. googleblog. com/2018/04/makinghigh-quality-video-efficient. html). Also, the video quality/bitrate can and will change dynamically during playback for many reasons. – Adding more streaming clients can change the bitrate experienced by some or all of the clients. – Web Browser – The March HPE video streaming tests were conducted with Chrome browser, which as a default uses their QUIC protocol. – However, the same experiment run using Windows Internet Explorer results in variable burst periods ranging from 3 – 9 seconds. – Furthermore, the average data rate of the stream after the initial buffering period is 1 Mbps higher with IE than with Chrome. One possible source of difference is that IE uses TCP+TLS. – Internet Path Topology & Latency – Video services use Content Delivery Networks to cache copies closer to users. – End-to-end latency and datapath topology will be different between any two test locations. – Video Encoders & Playback Ladders – Video providers continuously re-encode their libraries based on R&D and compression advances. ( ”Per-Title Encode Optimization, ” Netflix Tech Blog, December 14, 2015) – There is no guarantee that a test conducted at two different times will stream the same file – We conclude that reproducing results with Internet streaming will be extremely challenging (or impossible) 3
Internet Streaming Issue #2 – Averaging Initial Buffering – The March internet streaming results were captured over a 5 minute interval – With Internet streaming, buffering caused “lumpy” transmissions with long quiet periods. – In the discussion at SE 45#3, two problems were identified: – Over how long a period should the measurement be taken, to fairly allocate the heavy initial buffer period? – How should the initial buffer period be amortized across the measurement period to estimate duty cycle? – It was proposed that new Internet streaming sessions would be captured over a 10 minute interval in order to provide more time to average out the initial buffering period – (average data rate comparisons between old and new data were done with the same 5 min interval) 4
Using Intranet Streaming – There are many advantages with video streaming from a local server in order to achieve repeatability: – Rather than randomly selecting a You. Tube video and hoping the data rate is high or low enough, we can select a video file and post process it to the specific format and data rate that we desire – In this case, Cisco has provided a video of a first person view when riding a motorcycle – The original file was post processed to 1080 P at 4. 5 Mbps – Three different 5 minute clips were generated to get independent videos for three client testing – We can select specific software to stream video. We propose to use VLC which is free and standard. – With VLC, we selected HTTP streaming, providing TCP base transport – With VLC intranet streaming there is no noticeable initial buffering period – With no initial buffering period, the is no contention as to how long to average over – In other words, the initial buffering period is already amortized 5
VLC HTTP Packet Capture with 1080 P 4. 5 Mbps Source Average: 4. 8 Mbps, Peak: 9. 9 Mbps 6
Intranet Testing Produces More Conservative Result 5 Minute You. Tube capture 10 Minute You. Tube capture 5 Minute VLC capture – 1080 P – No beacons in packet capture – Beacons in packet capture – 6 Mbps AP beacon and basic rate – 6 Mbps AP beacon and basic rate – One client result – Average data rate 3. 98 Mbps – Average data rate 3. 38 Mbps – Average data rate: 4. 5 Mbps – Airtime utilization: 0. 71% – Airtime utilization: 0. 59% – Airtime utilization: 1. 17% – Three client result – Average data rate 14. 84 Mbps – Average data rate 11. 71 Mbps – Average data rate: 13. 5 Mbps – Airtime utilization: 0. 83%/client (2. 49% overall) – Airtime utilization: 0. 65%/client (1. 96% overall) – Airtime utilization: 0. 93%/client (2. 78% overall) VLC based testing provides similar airtime utilization with a repeatable test procedure 7
- Slides: 7