MPEG 4 Codec for Access Grids National Center

MPEG 4 Codec for Access Grids National Center for High Performance Computing Speaker: Barz Hsu Email: page 1 barz@nchc. org. tw 1/23/2005 11/11/2004

Outline • • Definition of HDTV Brief Introduction to MPEG-4 Standard History and Problems of VIC Contributions page 2 1/23/2005

What’s High Definition Video? • Frame Resolution: >= 720 x 480 or 640 x 480 • Frame Rate >= 20 fps • Peak Signal to Noise Ratio (PSNR) >= 30 d. B page 3 1/23/2005

Another Key Factors for Access Grids • Real-Time – Minimize latency & jitter • Error Resilience – RTP over UDP is unreliable • Large Scale Video Conferencing – Selection of suitable coding tools – Optimization • Adaptive – network bandwidth – computing resources page 4 1/23/2005

Status of MPEG 4 • Part 2: Advanced Simple Profile (ASP) • Part 10: Advanced Video Coding (AVC or H. 264) • Match MPEG-2 quality at up to half data rate • Deliver excellent video quality across entire bandwidth spectrum – from 3 G to HD – (from 40 Kbps to 10 MBps) page 5 1/23/2005

Performance Comparison page 6 1/23/2005

Interoperability Issues • Communicate with Polycom, i. Chat AV (Tiger) • Transport Stream – RFC 3016: RTP Payload Format for MPEG-4 Audio/Visual Streams. – RFC 3640: RTP Payload Format for Transport of MPEG-4 Elementary Streams – MPEG 2 TS page 7 1/23/2005

VIC: Video Conferencing Tool • Developed by Network Research Group in U. C. Berkeley • Mbone Tools used by Access Grid, in. SORS, and VRVS – vic for video and rat for audio • • Based on RTP/RTCP to provide real-time video Support Multicast/Unicast Protocol Hybrid of Tcl/Tk and C++ Supported codecs – H. 261, H. 263++, MJPEG, nv, and etc. page 8 1/23/2005

The Problems of VIC • Hard to maintain • Lack of high-quality video codec – only H. 261 is relative useable • Limitation on frame resolution – Only QCIF (176 x 144), CIF (352 x 288) • Limitation on viewing resolution • Lack of exhaustive optimization on codec, render, grabber, and etc page 9 1/23/2005

Our Goal • Broadcast in Television Quality – Leverage with Existing Open Source Resources • VLC, MPlayer, and FFmpeg – Providing “Sense of Presence” – As Real-Time As Possible – High-Definition, High Quality – Incorporate with up-to-date codecs such as MPEG 2, MPEG 4 (ASP), H. 264 – Less Compression Artifact – Error Resilience/Concealment page 10 1/23/2005

MPEG 4 Codec • Related Open Source Project – FFmpeg, xvid, and sklmp 4 • FFmpeg/libavcodec – High performance of CPU usage – Support various codec • MPEG 4, MPEG 2, MPEG 1 • H. 263++, H. 263, H. 261 – Provide error resilience tools • slices, data partition – Cross Platform • Linux, Windows, and Mac OS page 11 1/23/2005

Benefits from MPEG 4 codec • Support arbitrary sizes of a video stream – in contrast to H. 261 – the largest size depends on your video grabber card and camera • Less obvious block artifacts page 12 1/23/2005

MPEG 4 Performance • Typical bit-rate is 1 Mbps – 720 x 480 (NTSC) with 25 frames/sec and PSNR value is about 40 d. B • Consistent bit-rate – adjusting bit-rate only affects to the quality; not influence the frame-rate • Up to 40 CIF, or 10 NTSC video streams. page 13 1/23/2005

Deinterlace • Convert interlaced video into progressive one page 14 1/23/2005

Efficient Color Conversion • VIC didn’t give correct color representation. • Efficient Color Conversion – Grabber (YUYV) to Encoder (YUV 420) – Decoder (YUV 420) to Display (RGB 15/RGB 16/RGB 24) page 15 1/23/2005

Scaling Viewing Windows • Original VIC – only support limited sizes of viewing windows – based on unrealistic pixel replication method • NCHC-VIC – – scaling viewing windows arbitrarily without too much overhead interpolation by 9 tap FIR filter hardware assisted by Xvideo extension in XFree 86 full screen mode page 16 1/23/2005

Full Screen Snapshot page 17 1/23/2005

Real-Time Video Streaming • Real-time – the compressed data are transferred at a speed that matches the coded video source rate. • Latency, Jitter – average end-to-end delay – delay variation page 18 1/23/2005

Error Resilience • Error Progratation page 19 1/23/2005

Error Resilience (cont’d) • Resynchronization marker – the data between the error location and the first following resynchronization marker would have to be discarded. • Data Partition – motion vectors and related syntactic elements are transmitted first, followed by syntactic elements like MB headers, motion vectors, and DCT coefficients. page 20 1/23/2005

Minimize Latency • Multithread Encoding – Take advantage of symmetric multiple processor architecture • Pipeline the latency of transmission and encoding • SIMD Instruction Ses – Optimized by SSE 2/SSE, MMX page 21 1/23/2005

Future Work • Support H. 264 codec • Support MPEG-4 High Efficient AAC (HE-AAC) page 22 1/23/2005

Conclusion • Project Website: • http: //vic. nchc. org. tw page 23 1/23/2005

Thanks for your attendance page 24 1/23/2005