Video Streaming via Transcoding Jianping Fan Department of

Video Streaming via Transcoding Jianping Fan Department of Computer Science University of North Carolina at Charlotte, NC 28223 http: //www. cs. uncc. edu/~jfan

1. Why we need transcoding? query object Networks? feature extraction Cluster 1 Subcluster i 1 Subregion ij 1 Cluster i Subcluster ij Subregion ijl Cluster n Subcluster im Subregion ijr Object ijrm Disk for cluster 1 Disk for cluster i

1. Why we need transcoding? Networks How can I access video in database over networks? a. Videos in database are encoded, fixed and stored. b. Network conditions are always changing! Streaming Videos via Trancoding!

2. Key parameters for transcoding a. Video coding format used in database system. Bit rate for the retrieved video! b. Network conditions such as bandwidth! c. Client’s Qo. S parameters: Quality such as resolution Delay Quality of Presentation of video

3. Basic requirements Original bit stream transcoding Bit stream sent to client via network a. Bit rate after transcoding should be close to network bandwidth! b. Quality of transcoded video should be accepted by clients! Not induce too much quantization error again! c. Easy for implementation!

4. Case study: Streaming MPEG-1 video over ISDN Error 1 Transcoder ISDN Network MPEG-1 Stream Error 2 MPEG-1 player MPEG-1 Encoder Real-Time Transcoder network Error 1 + Error 2 should be accepted by clients!

5. Quality of Video Parameters for defining Quality of Video: a. Frame rate-The number of still pictures displayed per second to give the viewer perception of motion. b. Color depth-The number of bits per pixel for representing color information. c. Frame resolution-Typically expressed as the width and height in pixels.

6. Simple Transcoding Real-time decoding and real-time re-encoding real-time MPEG-1 Stream ISDN Network MPEG-1 Decoder Real-time H. 263 Encoder Too Expensive & Too much Errors!

6. Simple Transcoding Where the bit rate comes from? a. Frame rate b. Frame resolution c. Color depth

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: resolution reduction What will happen when we reduce MPEG video’s resolution?

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: GOP 1 GOP 2 GOP 3 GOP 4 time I B B P B B B P GOP

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: Sample filtering? New I frame What will happen if do like this?

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: How to do it? How to code I frame? Y U I frame V

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: Sample filtering I-macroblock We can do this easily on original image, can we this sample filtering on compressed domain (DCT coefficients)? DC and AC coefficients!

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: Sample filtering

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: If we do resolution reduction on I frames, there are two questions: (1) how we can do this on P & B frames? (2) what’s the influence on motion compensation?

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: ? Original I Frame Original P Frame How we can use transcoded I frame in the decoder side to get P frame? Transcoding I frame

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: Transcoding I frame ? Original P Frame Transcoding P frame transcoding Original I Frame

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: I or P frame Motion vectors & DCT coefficients of compensated errors How to sample the motion vectors?

8. Transcoding via Temporal Sampling Temporal Resolution Reduction (frame dropping): Frame dropping (1) Which frame should be dropped? (2) What is the influence if we drop some frames?

7. Transcoding via Temporal Sampling Temporal Resolution Reduction (frame dropping): (1) It is no problem for us dropping B frames, but bit rate for them is low! (1) We cannot drop I frames in a GOP (group of pictures). (2) If we drop a P frame, how about its B frames? Drop B frame together or refine motion vectors? I B P

8. Transcoding via Temporal Sampling Temporal Resolution Reduction (frame dropping): I B B P trancoding I B B P How we can get compensation of B from another P frame?

8. Transcoding via Temporal Sampling I P Motion vector 1 I Motion vector 3 P Motion vector 2 P DCT (1) What’s the relationship between motion vector 3 and motion vector 1 and motion vector 2? (2) What’s the relationship between the DCT coefficients between I frame and frame after one P is dropped?

9. Transcoding via Averaging Videos Select DC coefficient for each block! DC DC AC 63

9. Transcoding via Averaging Videos If we just select DC coefficients for I frame, what’s influence on P frames and B frames? I averaging ? P

10. Transcoding via Reducing Color Depth Color depth reduction via binary: binary 8 bits 1 bit (1) How to do this on the compressed domain? (2) What’s the influence on P and B frames?

11. Transcoding via Re-Quantization is an efficient way to reduce the redundancy information, but also induce errors in the picture! In current video and image coding standards such as MPEG-*, H. 26*, JPEG-*, they use two parameters to define quantization: (1) Scale; (2) Quantization table. We can only change scale to realize re-quantization! What’s the influence for motion compensation?

12. Transcoding with Network Condition ISDN Network MPEG-1 Stream Adaptive Transcoder Parameters should be used for adaptive trancoder design: a. Frame rate b. resolution c. Color depth d. Averaging

12. Transcoding with Network Condition ISDN Network MPEG-1 Stream Adaptive Transcoder Problem: Adaptive Transcoder Design Subject to: Network bandwidth (changing) Source video bit rate ? Network should be used for selecting parameters!

13. Transcoding with Client Qo. S ISDN Network MPEG-1 Stream Adaptive Transcoder Problem: Adaptive Transcoder Design Subject to: Client’s Qo. S Source video bit rate ?

14. Open Discussion a. Why we select streaming for video not download? b. How transcoder can adapt? c. Can we do transcode fully on compressed domain?

How can we support content-adaptive video transcoding?

Do we keep the dominant video contents?