Multimedia Compression q Audio image and video require
- Slides: 59
Multimedia Compression q Audio, image and video require vast amounts of data q 320 x 240 x 8 bits grayscale image: 77 Kb q 1100 x 900 x 24 bits color image: 3 MB q 640 x 480 x 24 x 30 frames/sec: 27. 6 MB/sec q Low network’s bandwidth doesn't allow for real time video transmission q Slow storage or processing devices don't allow for fast playing back q Compression reduces storage requirements E. G. M. Petrakis Multimedia Compression 1
Classification of Techniques q. Lossless: recover the original representation q. Lossy: recover a representation similar to the original one qhigh compression ratios qmore practical use q. Hybrid: JPEG, MPEG, px 64 combine several approaches E. G. M. Petrakis Multimedia Compression 2
Compression Standards Furht at. al. 96 E. G. M. Petrakis Multimedia Compression 3
Lossless Techniques Furht at. al. 96 E. G. M. Petrakis Multimedia Compression 4
Lossy Techniques Furht at. al. 96 E. G. M. Petrakis Multimedia Compression 5
JPEG Modes of Operation q. Sequential DCT: the image is encoded in one left-to-right, top-to-bottom scan q. Progressive DCT: the image is encoded in multiple scans (if the transmission time is long, a rough decoded image can be reproduced) q. Hierarchical: encoding at multiple resolutions q. Lossless : exact reproduction E. G. M. Petrakis Multimedia Compression 6
JPEG Block Diagrams Furht at. al. 96 E. G. M. Petrakis Multimedia Compression 7
JPEG Encoder q. Three main blocks: q. Forward Discrete Cosine Transform (FDCT) q. Quantizer q. Entropy Encoder q. Essentially the sequential JPEG encoder q. Main component of progressive, lossless and hierarchical encoders q. For gray level and color images E. G. M. Petrakis Multimedia Compression 8
Sequential JPEG q Pixels in [0, 2 p-1] are shifted in [-2 p-1, 2 p-1 -1] q The image is divided in 8 x 8 blocks q Each 8 x 8 block is DCT transformed E. G. M. Petrakis Multimedia Compression 9
DCT Coefficients q. F(0, 0) is the DC coefficient: average value over the 64 samples q. The remaining 63 coefficients are the AC coefficients q. Pixels in [-128, 127]: DCTs in [-1024, 1023] q. Most frequencies have 0 or near to 0 values and need not to be encoded q. This fact achieves compression E. G. M. Petrakis Multimedia Compression 10
Quantization Step q. All 64 DCT coefficients are quantized q. Fq(u, v) = Round[F(u, v)/Q(u, v)] q. Reduces the amplitude of coefficients which contribute little or nothing to 0 q. Discards information which is not visually significant q. Quantization coefficients Q(u, v) are specified by quantization tables q. A set of 4 tables are specified by JPEG E. G. M. Petrakis Multimedia Compression 11
Quantization Tables Furht at. al. 96 q for (i=0; i < 64; i++) for (j=0; j < 64; j++) Q[i, j] = 1 + [ (1+i+j) quality]; qquality = 1: best quality, lowest compression qquality = 25: poor quality, highest compression E. G. M. Petrakis Multimedia Compression 12
DC Coefficients q Predictive coding of DC Coefficients q Adjacent blocks have similar DC intensities q Coding differences yields high compression E. G. M. Petrakis Multimedia Compression 13
AC Coefficients q The 63 AC coefficients are ordered by a “zig-zag” sequence q Places low frequencies before high frequencies q Low frequencies are likely to be 0 q Sequences of such 0 coefficients will be encoded by fewer bits E. G. M. Petrakis Multimedia Compression Furht at. al. 96 14
Entropy Encoding q Encodes sequences of quantized DCT coefficients into binary sequences q AC: (runlength, size) (amplitude) q DC: (size, amplitude) q runlength: number consecutive 0’s, up to 15 qtakes up to 4 bits for coding q(37, 4)(12) = (15, 0)(7, 4)(12) q amplitude: first non-zero value q size: number of bits to encode amplitude q 0 0 0 476: (6, 9)(476) E. G. M. Petrakis Multimedia Compression 15
Huffman coding q. Converts each sequence into binary q. First DC following with ACs q. Huffman tables are specified in JPEG q. Each (runlength, size) is encoded using Huffman coding q. Each (amplitude) is encoded using a variable length integer code q(1, 4)(12) => (11111101101100) E. G. M. Petrakis Multimedia Compression 16
Example of Huffman table Furht at. al. 96 E. G. M. Petrakis Multimedia Compression 17
JPEG Encoding of a 8 x 8 block Furht at. al. 96 E. G. M. Petrakis Multimedia Compression 18
Compression Measures q Compression ratio (CR): increases with higher compression q. CR = Original. Size/Compressed. Size q Root Mean Square Error (RMS): better quality with lower RMS q. Xi: original pixel values qxi: restored pixel values qn: total number of pixels E. G. M. Petrakis Multimedia Compression 19
Furht at. al. 96 E. G. M. Petrakis Multimedia Compression 20
JPEG Decoder q. The same steps in reverse order q. The binary sequences are converted to symbol sequences using the Huffman tables q. F’(u, v) = Fq(u, v)Q(u, v) q. Inverse DCT E. G. M. Petrakis Multimedia Compression 21
Progressive JPEG q When image encoding or transmission takes long there may be a need to produce an approximation of the original image which is improved gradually Furht at. al. 96 E. G. M. Petrakis Multimedia Compression 22
Progressive Spectral Selection q. The DCT coefficients are grouped into several bands q. Low-frequency bands are first qband 1: DC coefficient only qband 2: AC 1, AC 2 coefficients qband 3: AC 3, AC 4, AC 5, AC 6 coefficients qband 4: AC 7, AC 8 coefficients E. G. M. Petrakis Multimedia Compression 23
Lossless JPEG q. Simple predictive encoding Furht at. al. 96 prediction schemes E. G. M. Petrakis Multimedia Compression 24
Hierarchical JPEG q Produces a set of images at multiple resolutions q Begins with small images and continues with larger images (down-sampling) q The reduced image is scaled-up to the next resolution and used as predictor for the higher resolution image E. G. M. Petrakis Multimedia Compression 25
Encoding 1. Down-sample the image by 2 a in each x, y 2. Encode the reduced size image (sequential, progressive. . ) 3. Up-sample the reduced image by 2 4. Interpolate by 2 in x, y 5. Use the up-sampled image as predictor 6. Encode differences (predictive coding) 7. Go to step 1 until the full resolution is encoded E. G. M. Petrakis Multimedia Compression 26
Furht at. al. 96 E. G. M. Petrakis Multimedia Compression 27
JPEG for Color images q. Encoding of 3 bands (RGB, HSV etc. ) in two ways: q. Non-interleaved data ordering: encodes each band separately q. Interleaved data ordering: different bands are combined into Minimum Coded Units (MCUs) q. Display, print or transmit images in parallel with decompression E. G. M. Petrakis Multimedia Compression 28
Interleaved JPEG q Minimum Coded Unit (MCU): the smallest group of interleaved data blocks (8 x 8) Furht at. al. 96 E. G. M. Petrakis Multimedia Compression 29
Video Compression q. Various video encoding standards: Quick. Time, DVI, H. 261, MPEG etc q. Basic idea: compute motion between adjacent frames and transmit only differences q. Motion is computed between blocks q. Effective encoding of camera and object motion E. G. M. Petrakis Multimedia Compression 30
MPEG q. The Moving Picture Coding Experts Group (MPEG) is a working group for the development of standards for compression, decompression, processing, and coded representation of moving pictures and audio q. MPEG groups are open and have attracted large participation q http: //mpeg. telecomitalialab. com E. G. M. Petrakis Multimedia Compression 31
MPEG Features q. Random access q. Fast forward / reverse searches q. Reverse playback q. Audio – visual synchronization q. Robustness to errors q. Auditability q. Cost trade-off E. G. M. Petrakis Multimedia Compression 32
MPEG -1, 2 q. At least 4 MPEG standards finished and more under construction q. MPEG-1: storage and retrieval of moving pictures and audio on storage media q 352 x 288 pixels/frame, 25 fps, at 1. 5 Mbps q. Real-time encoding even on an old PC q. MPEG-2: higher quality, same principles q 720 x 576 pixels/frame, 2 -80 Mbps E. G. M. Petrakis Multimedia Compression 33
MPEG-4 q. Encodes video content as objects q. Based on identifying, tracking and encoding object layers which are rendered on top of each other q. Enables objects to be manipulated individually or collectively on an audiovisual scene (interactive video) q. Only a few implementations q. Higher compression ratios E. G. M. Petrakis Multimedia Compression 34
MPEG-7 q. Standard for the description of multimedia content q. XML Schema for content description q. Does not standardize extraction of descriptions q. MPEG 1, 2, and 4 make content available q. MPEG 7 makes content semantics available E. G. M. Petrakis Multimedia Compression 35
MPEG-1, 2 Compression q Compression of full motion video, interframe compression, stores differences between frames q A stream contains I, P and B frames in a given pattern q Equivalent blocks are compared and motion vectors are computed and stored as P and B frames Furht at. al. 96 E. G. M. Petrakis Multimedia Compression 36
Frame Structures q I frames: self contained, JPEG encoded q. Random access frames in MPEG streams q. Low compression q P frames: predicted coding using with reference to previous I or P frame q. Higher compression q B frames: bidirectional or interpolated coding using past and future I or P frame q. Highest compression E. G. M. Petrakis Multimedia Compression 37
Example of MPEG Stream Furht at. al. 96 q B frames 2 3 4 are bi-directionally coded using I frame 1 and P frame 5 q. P frame 5 must be decoded before B frames 2 3 4 q. I frame 9 must be decoded before B frames 6 7 8 q. Frame order for transmission: 1 5 2 3 4 9 6 7 8 E. G. M. Petrakis Multimedia Compression 38
MPEG Coding Sequences q. The MPEG application determines a sequence of I, P, B frames q. For fast random access code the whole video as I frames (MJPEG) q. High compression is achieved by using large number of B frames q. Good sequence: (IBBPBBPBB). . . E. G. M. Petrakis Multimedia Compression 39
Motion Estimation q. The motion estimator finds the best matching block in P, B frames q. Block: 8 x 8 or 16 x 16 pixels q. P frames use only forward prediction: a block in the current frame is predicted from past frame q. B frames use forward or backward or prediction by interpolation: average of forward, backward predicted blocks E. G. M. Petrakis Multimedia Compression 40
Motion Vectors block: 16 x 16 pixles Furht at. al. 96 q One or two motion vectors per block q. One vector forward predicted P or B frames or backward predicted B frames q. Two vectors for interpolated B frames E. G. M. Petrakis Multimedia Compression 41
MPEG Encoding q I frames are JPEG compressed q P, B frames are encoded in terms of future or previous frames q Motion vectors are estimated and differences between predicted and actual blocks are computed q. These error terms are DCT encoded q. Entropy encoding produces a compact binary code q. Special cases: static and intracoded blocks E. G. M. Petrakis Multimedia Compression 42
MPEG encoder JPEG encoding Furht at. al. 96 E. G. M. Petrakis Multimedia Compression 43
MPEG Decoder Furht at. al. 96 E. G. M. Petrakis Multimedia Compression 44
Motion Estimation Techniques q. Not specified by MPEG q. Block matching techniques q. Estimate the motion of an nxm block in present frame in relation to pixels in previous or future frames q. The block is compared with a previous or forward block within a search area of size (m+2 p)x(n+2 p) qm = n = 16 qp = 6 E. G. M. Petrakis Multimedia Compression 45
Block Matching Furht at. al. 96 q Search area in block matching techniques q. Typical case: n=m=16, p=6 q. F: block in current frame q. G: search area in previous (or future) frame E. G. M. Petrakis Multimedia Compression 46
Cost functions The block has moved to the position that minimizes a cost function q I. Mean Absolute Difference (MAD) q q F(i, j) : a block in current frame G(i, j) : the same block in previous or future frame (dx, dy) : vector for the search location q q E. G. M. Petrakis dx=(-p, p), dy=(-p, p) Multimedia Compression 47
More Cost Functions II. Mean Squared Difference (MSD) III. Cross-Correlation Difference (CCF) E. G. M. Petrakis Multimedia Compression 48
More cost Functions IV. Pixel Difference Classification (PDC) q q q t: predefined threshold each pixel is classified as a matching pixel (T=1) or a mismatching pixel (T=0) the matching block maximizes PDC E. G. M. Petrakis Multimedia Compression 49
Block Matching Techniques q. Exhaustive: very slow but accurate q. Approximation: faster but less accurate q. Three-step search q 2 -D logarithmic search q. Conjugate direction search q. Parallel hierarchical 1 -D search (not discussed) Pixel difference classification (not discussed here) E. G. M. Petrakis Multimedia Compression 50
Exhaustive Search q. Evaluates the cost function at every location in the search area q. Requires (2 p+1)2 computations of the cost function q. For p=6 requires 169 computations per block!! q. Very simple to implement but very slow E. G. M. Petrakis Multimedia Compression 51
Three-Step Search q. Computes the cost function at the center and 8 surrounding locations in the search area q. The location with the minimum cost becomes the center location for the next step q. The search range is reduced by half E. G. M. Petrakis Multimedia Compression 52
Three-Step Motion Vector Estimation (p=6) Furht at. al. 96 E. G. M. Petrakis Multimedia Compression 53
Three–Step Search 1. Compute cost (MAD) at 9 locations • Center + 8 locations at distance 3 from center 2. Pick min MAD location and recompute MAD at 9 locations at distance 2 from center 3. Pick the min MAD locations and do same at distance 1 from center • • • The smallest MAD from all locations indicates the final estimate M 24 at (dx, dy)=(1, 6) Requires 25 computations of MAD E. G. M. Petrakis Multimedia Compression 54
2 -D Logarithic Search q. Combines cost function and predefined threshold T q. Check cost at M(0, 0), 2 horizontal and 2 vertical locations and take the minimum q. If cost at any location is less than T then search is complete q. If no then, search again along the direction of minimum cost - within a smaller region E. G. M. Petrakis Multimedia Compression 55
Furht at. al. 96 q if cost at M(0, 0) < T then search ends! q compute min cost at M 1, M 2, M 3, M 4; take their min; q if min cost < M(0, 0) q if (cost less than T) then search ends! q else compute cost at direction of minimum cost (M 5, M 6 in the example); else compute cost at the neighborhood of min cost within p/2 (M 5 in the example) E. G. M. Petrakis Multimedia Compression 56
Conjugate Direction Search Furht at. al. 96 q Repeat q find min MAD along dx=0, -1, 1 (y fixed): M(1, 0) in example q find min MAD along dy=0, -1, 1 starting from previous min (x fixed): M(2, 2) q search similarly along the direction connecting the above mins E. G. M. Petrakis Multimedia Compression 57
Other Compression Techniques q. Digital Video Interactive (DVI) qsimilar to MPEG-2 q. Fractal Image Compression q. Find regions resembling fractals q. Image representation at various resolutions q. Sub-band image and video coding q. Split signal into smaller frequency bands q. Wavelet-based coding E. G. M. Petrakis Multimedia Compression 58
References q B. Furht, S. W. Smoliar, H-J. Zang, “Video and Image Processing in Multimedia Systems”, Kluwer Academic Pub, 1996 E. G. M. Petrakis Multimedia Compression 59
- Video compression techniques in multimedia
- Fundamentals of image compression
- Variable length coding in digital image processing
- Chest compression test
- Audio compression techniques
- Lossless compression algorithms in multimedia
- Lossy compression
- Lossless compression in multimedia
- Spatial redundancy in video compression
- Compression moulding video
- Xxx oooo video
- Spatial redundancy in video compression
- Digital representation in multimedia
- Adding sound to multimedia project
- Menggabungkan audio kedalam sajian multimedia
- Digital audio in multimedia
- Pengertian dari audio
- Animation audio video
- Mi collab
- Audio video interleave
- Animation audio video
- Audio vs video
- Bahan audio
- Video outline
- Avi to gif windows
- Quantization in data compression
- Coding redundancy
- 472
- Yuvpak compressed fractal image
- Jpeg: still image data compression standard
- Jpeg still image data compression standard
- Singular value decomposition image compression
- Lossy compression in digital image processing
- Signal image compression
- What is multimedia?
- Analog and digital video in multimedia
- Pre production planning for video film and multimedia
- Analog vs digital video
- Multimedia becomes interactive multimedia when
- Multimedia becomes interactive multimedia when
- Esa multimedia.esa.int./multimedia/virtual-tour-iss
- Video berasal dari bahasa latin, yaitu
- How video works in multimedia
- What is video shooting in multimedia
- Interaktif multimedia
- Justin sconza
- Types of video in multimedia
- Heanet filesender
- Videos yandex search
- Gravity yahoo
- Yahoo search video
- The frame size of a video refers to the video’s
- Image and video upscaling from local self examples
- Jpg type of multimedia
- Hevc full form
- Analog image and digital image
- Do electromagnetic waves require a medium
- Carries energy through matter
- It monitor users behavior and offer contextual advice
- Facilitated diffusion high to low