CIS 679 Multimedia Basics r Multimedia data type

CIS 679: Multimedia Basics r Multimedia data type r Basic compression techniques

Multimedia Data Type r Audio r Image r Video

Audio r Digitization m Sampling m Quantization m Coding r Higher sampling rate -> higher quality m Nyquist sampling theorem: for lossless digitization, the sampling rate should be at least twice the maximum frequency responses r Higher bits per sample -> higher quality r Sampling at 8 KHz, 8 bit samples -> 64 kbits/sec r CD-quality audio m Sampling at 44. 1 KHz, 16 bit samples -> 705. 6 kbits/sec

Image/Video r Digitization m Scan a picture frame m Digitize every pixel r Color represented by RGB r Normally converted to Y (black and white TV), U and V m m Luminance Y = 0. 30 R + 0. 59 G + 0. 11 R Chrominance U = (B-Y) * 0. 493 V = (R-Y) * 0. 877

Video Transmission Standards r NTSC m Y = 0. 30 R + 0. 59 G + 0. 14 B m I = 0. 60 R + 0. 28 G + 0. 32 B m Q = 0. 21 R + 0. 52 G + 0. 21 B r PAL

Studio-quality TV r NTSC m 525 lines at 30 frames/second m Y sampled at 13. 5 MHz, Chrominance values at 6. 75 MHz m With 8 -bit samples, m Data rate = (13. 5 + 6. 75) * 8 = 216 Mbps

Summary of Multimedia Data Types r Audio data rate = 64 kbps, and 705. 6 kbps r Video date rate = 216 Mbps r Compression is required!

Can Multimedia Data Be Compressed? r Redundancy can be exploited to do compression! r Spatial redundancy m correlation between neighboring pixels in image/video r Spectral redundancy m correlation among colors r Psycho-visual redundancy m Perceptual properties of human visual system

Categories of Compression r Lossless m No distortion of the original content m Used for computer data, medical images, etc. r Lossy m Some distortion m Suited for audio and video

Compression Techniques Run-length Coding Entropy Encoding Huflfman Coding Arithmetic Coding DPCM Prediction DM FFT Transformation DCT Source Coding Bit Position Layered Coding Subsampling Sub-band Coding Vector Quantization J PEG MPEG Hybrid Coding H. 261 DVI RTV, DVI PLV

Entropy Encoding Techniques r Lossless compression r Run-length encoding m Represent stream as (c 1, l 1), (c 2, l 2), …, (ck, lk) m 11111333332222444444 = (1, 10) (3, 5) (2, 4) (4, 5) m Or ABCCCCDEFGGG = ABC!8 DEFGGG r Pattern Substitution m Substitute smaller symbols for frequently used patterns

Huffman Coding r Use variable length codes r Most frequently used symbols coded with fewest bits r Codes are stored in a codebook r Codebook transferred with the compressed stream

Source Encoding Techniques r Transformation encoding m Transform the bit-stream into another domain m Data in the new domain more amenable to compression m Type of transformation depends on data r Image/video transformed from time domain into frequency domain (DCT)

Differential/Predictive Encoding r Encoding the difference between actual value and a prediction of that value r Number of Techniques m m m Differential Pulse Code Modulation (DPCM) Delta Modulation (DM) Adaptive Pulse Code Modulation (APCM) r How they work? m When consecutive change little m Suited for audio and video

Vector Quantization r Divide the data stream into blocks or vectors m One or two dimensional blocks r Use codebooks r Find the closest symbol in codebook for a given sample r Transmit the reference to that symbol r Codebook present at sender/receiver r When no exact match, could send the error m Lossy or lossless r Useful with known signal characteristics r Construct codebooks that can match a wide range of symbols

Major Steps of Compression r Preparation m Uncompressed analog signal -> sampled digital form r Processing m Source coding m DCT typically used: Transform from time domain -> frequency domain r Quantization m Quantize weights into integer codes m Could use different number of bits per coefficient r Entropy encoding m Lossless encoding for further compression

Conclusion r Multimedia data types r Why multimedia can be compressed? r Categories of compression r Compression techniques m Entropy encoding m Source encoding m Hybrid coding r Major steps of compression r What’s next? m JPEG m MPEG