Information Management DIG 3563 Lecture 11 Compression Lossy

Information Management DIG 3563 – Lecture 11 Compression – Lossy and Lossless J. Michael Moshell University of Central Florida Original image* by Moshell et al. Imagery is from. Wikimedia except where marked with *. Licensing is listed. 1

Before we begin: Review: binary & hexadecimal numbers 132 (decimal) means 1 - $100 bill (102) 3 - $10 bills (101) 2 - $1 bill (100) for a total value of 100 30 2 132 (decimal, or base-10) -2 -

Binary numbers at the of computing 1101 (binary) means 1 - $8 bill (23) 1 - $4 bill (22) 0 - $2 bill (21) 1 - $1 bill (20) decimal value 8 4 0 1 for a total value of 13 (decimal, or base-10) -3 -

Your task: convert bin to dec or decimal to binary, by this method 1101 (binary) means 1 - $8 bill (23) 1 - $4 bill (22) 0 - $2 bill (21) 1 - $1 bill (20) decimal value 8 4 0 1 for a total value of 13 (decimal, or base-10) -4 -

Practice problems (do it!) Convert these binary numbers to decimal: 1000 1001 1101 0011 Convert these decimal numbers to binary: 31 17 129 -5 -

Whoa. . . decimal to binary? An example: x= 15 decimal is. . . what in binary? (call it y) You need to know powers of 2, up to 210, like this 2 4 8 16 32 64 128 256 512 1024 Now: is x>=16? No, so y= 0 is x>=8? Yes, so y= 01 and x=x-8 now x=7. Is x>=4? Yes, so y=011, x=x-4 now x=3. Is x>=2? Yes, so y=0111, x=x-2 now x=1. Is x>=1? Yes, so y=0111 -6 -

Binary is. . messy! binary value symbol 0000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 1 2 3 4 5 6 7 8 9 A B C D E F 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 So we ‘encode’ it with convenient symbols so we would represent 1000 1011 as 8 B in “Hexadecimal” (base 16) and convert like this: 8 x 16 = 12810 B (or 1110) = 1110 total = 13910 -7 -

Practice problems 2 Convert these binary numbers to hexadecimal: 1000 1001 1101 0011 Convert these decimal numbers to hexadecimal: 31 17 129 -8 -

Practice problems 2 Convert these hex numbers to decimal: AA 16 1216 -(sometimes we write 12 x for he. Xadecimal) A convention: 8 bits is a byte; represented as 2 hex digits, between 01 x and FFx =25510 -9 -

And now: compression We begin with TV, the first BIG data • Media evolved through these steps -35 mm film played by a film chain to produce a video signal -2 inch reel-to-reel videotape www. wikipedia. org - GNU FDL -10 -

Media Asset Management in TV (These are mostly analog media) - cassettes of various sizes: 3 / 4 inch dominated - analog media: - feasible in 1970’s - no high RAM needs - no high speed CPU -BUT: - Limited generation copying www. wikipedia. org - GNU FDL -11 -

Media Asset Management in TV - The arrival of digital videotape- Digital Video - professional: DVCAM-L - professional: DVCPRO-M - consumer: mini. DV www. wikipedia. org - GNU FDL -12 -

Media Asset Management in TV - The arrival of digital videotape- Digital Video - professional: DVCAM-L - professional: DVCPRO-M - consumer: mini. DV Better than analog, but not Totally loss-less. (Why? ) www. wikipedia. org - GNU FDL -13 -

Media Asset Management in TV - The arrival of digital videotape-The problem of realtime capture and delivery: Error detecting and correcting codes require “repair” if errors are found. If too much data is missing from a tape, repair may be impossible or take too long. So: Digital Video uses “masking” & “fill-in”. -14 -

Media Asset Management in TV - The arrival of true video with RAID -RAID: Redundant Array of Inexpensive Disks * Designed for no-single-point-of-failure. * Many strategies. Here’s a simple one. * Odd Parity: 8 data bits 1 0 1 1 0 0 -15 -

Media Asset Management in TV - The arrival of true video with RAID -RAID: Redundant Array of Inexpensive Disks * Designed for no-single-point-of-failure. * Many strategies. Here’s a simple one. * Odd Parity: 8 data bits 1 0 1 1 0 0 1 plus 1 parity bit - - - - -/ added so that the WHOLE ROW has an odd number of 1 s in it. -16 -

Media Asset Management in TV - How error correcting codes work 1 0 1 0 0 0 1 1 0 0 1 Longitudinal Parity: 1 0 0 1 1 0 1 -17 -

Media Asset Management in TV - How error correcting codes work 1 0 1 0 0 0 0 0 1 1 0 0 1 0 1 Longitudinal Parity: 1 0 0 1 1 0 1 Now: A bit goes bad. Can you find it? Can you fix it? -18 -

Media Asset Management in TV - How error correcting codes work 1 0 1 0 0 0 0 0 1 1 0 0 1 0 1 Longitudinal Parity: 1 0 0 1 1 0 1 Now: A bit goes bad. Can you find it? Can you fix it? Yes. . . we can find its row and its column. -19 -

Media Asset Management in TV - Practice: Find the bad bit a b c p 1 0 1 q 1 1 1 r 1 0 0 s 0 0 0 t 1 1 1 u 0 0 1 v 0 0 1 w 0 0 1 x 1 1 0 d=Parity: 1 0 0 1 0 1 -20 -

Media Asset Management in TV - Practice: Find the bad bit a b c p 1 0 1 q 1 1 1 r 1 0 0 s 0 0 0 t 1 1 1 u 0 0 1 v 0 0 1 w 0 0 1 x 1 1 0 d=Parity: 1 0 0 1 0 1 -21 -

Media Asset Management in TV - Practice: Find the bad bit a b c p 1 0 1 q 1 1 1 r 1 0 0 s 0 0 0 t 1 1 1 w 0 0 1 x 1 1 0 d=Parity: 1 0 0 1 0 1 And so we correct it to 0. u 0 0 1 v 0 0 0 -22 -

Media Asset Management in TV RAID Technology … is a BIT (well, a little) more complex than that. But you get the idea, I hope. -23 -

Media Asset Management in TV RAID Technology … is a BIT (well, a little) more complex than that. But you get the idea, I hope. Imagine NINE disks to store EIGHT bits of info. If we put one bit of each "word" on 8 disks and the parity checksum on #9, then if any ONE disk fails, we could reconstruct its contents. -24 -

Media Asset Management in TV RAID Technology … is a BIT (well, a little) more complex than that. But you get the idea, I hope. Imagine NINE disks to store EIGHT bits of info. If we put one bit of each "word" on 8 disks and the parity checksum on #9, then if any ONE disk fails, we could reconstruct its contents. -25 -

Media Asset Management in TV RAID Technology … is a BIT (well, a little) more complex than that. But you get the idea, I hope. X Imagine NINE disks to store EIGHT bits of info. If we put one bit of each "word" on 8 disks and the parity checksum on #9, then if any ONE disk fails, we could reconstruct its contents. -26 -

Media Asset Management in TV RAID Technology … is a BIT (well, a little) more complex than that. But you get the idea, I hope. X Imagine NINE disks to store EIGHT bits of info. If we put one bit of each "word" on 8 disks and the parity checksum on #9, then if any ONE disk fails, we could reconstruct its contents. -27 -

Media Asset Management in TV RAID Technology … is a BIT (well, a little) more complex than that. But you get the idea, I hope. X Imagine NINE disks to store EIGHT bits of info. If we put one bit of each "word" on 8 disks and the parity checksum on #9, then if any ONE disk fails, we could reconstruct its contents. -28 -

Data Compression Key concepts: Lossless compression: perfectly reversible Essential for computer programs, forensics, etc. Lossy compression: some info is lost, but much space is saved. Used when the resulting product is "good enough" for human consumption. And remember: the original may be archived, too. -29 -

Data Compression Data can be compressed if it contains redundency. What is that? • An image contains redundency if a given pixel's color allows you to predict the color of the adjacent ones with greater than random chance of success. • The first compression method we study is called Run Length Encoding. -30 -

Run Length Encoding (RLE) A run is a sequence of identical values. 4 4 4 is a 'run' of five values, all of which = 4 A code block is two numbers, like 5 4. RLE uses code blocks to represent data-with-runs. DATA: 5 5 5 2 2 2 7 7 7 7 - total of 17 numbers Coded: 6 5 3 2 8 7 - total of 6 numbers -31 -

Run Length Encoding (RLE) You practice: write down the RLE code for this sequence: 4 4 4 21 21 21 18 18 -32 -

Run Length Encoding (RLE) You practice: write down the RLE code for this sequence: 4 4 4 21 21 21 18 18 And. . . the answer is. . . 3 4 5 21 2 18 -33 -

Run Length Encoding (RLE) You practice: Decode this code message Message: 5 4 3 2 6 47 -34 -

Run Length Encoding (RLE) You practice: Decode this code message Message: 5 4 3 2 6 47 And. . . the answer is. . . 4 4 4 2 2 2 47 47 47 -35 -

Run Length Encoding (RLE) The question of memory use. If 'number' means 'byte', then 0<=n<=255. So, code blocks represent runs of length <=255, and you need 3 bytes per color, if 24 bit color. You can either jointly or separately RLE the RGB parts Code blocks would be 4 bytes long: n R G B if joint. If you do all the R then all the G then all the B, your code blocks are 2 bytes long * 3 sets of 'em. -36 -

Run Length Encoding (RLE) The question of efficiency. Are there any images that CANNOT be RLE compressed? Sure! Those whose average run length is less than 2. Data: 1 3 4 5 5 7 6 = nine bytes "Compressed": 1 1 1 3 1 4 4 5 1 7 1 6 = twelve bytes -37 -

Run Length Encoding (RLE) The question of efficiency. Are there any images that CANNOT be RLE compressed? Sure! Those whose average run length is less than 2. Data: 1 3 4 5 5 7 6 = nine bytes "Compressed": 1 1 1 3 1 4 4 5 1 7 1 6 = twelve bytes ? ? What would such a 'hard to compress' image look like? ? -38 -

Run Length Encoding (RLE) The question of efficiency. Are there any images that CANNOT be RLE compressed? Sure! Those whose average run length is less than 2. Data: 1 3 4 5 5 7 6 = nine bytes "Compressed": 1 1 1 3 1 4 4 5 1 7 1 6 = twelve bytes ? ? What would such a 'hard to compress' image look like? ? Answers : (a) NATURAL images, full of gradients (b) NOISY or RANDOM images -39 -

Run Length Encoding (RLE) So, what would you do about such bumpy data? You COULD make up a special message, with code 0, like this: 0 6 32 43 56 77 33 22 Where '0' means "Random block coming" and '6' means "the next six numbers are as-is" So the output would be 32 43 56 77 33 22 This beats straight RLE, but is still not super efficient. -40 -

Huffman Coding Idea: most frequent messages get short code symbols. Early example: Morse Code Typesetter's sequence: etaoin e t . a o . - i . . -. n shrdlu <-High frequency letters --- -41 -

Huffman Coding Idea: most frequent messages get short code symbols. Early example: Morse Code Typesetter's sequence: etaoin e t . a o . - i . . -. n --- shrdlu <-High frequency letters Low frequency letters Q --. Y -. -Z --. . -42 -

Huffman Coding So, to Huffman code a sequence of symbols, first you compute their Frequency Histogram. 5 8 5 5 8 8 14 16 8 5 etc. . . 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0 0 1 0 0 0 -43 -

Huffman Coding So, to Huffman code a sequence of symbols, first you compute their Frequency Histogram. 5 8 5 5 8 8 14 16 8 5 etc. . . 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0 0 10 0 0 0 -44 -

Huffman Coding So, to Huffman code a sequence of symbols, first you compute their Frequency Histogram. 5 8 5 5 8 8 14 16 8 5 etc. . . 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0 0 2 0 0 10 0 0 0 -45 -

Huffman Coding So, to Huffman code a sequence of symbols, first you compute their Frequency Histogram. 5 8 5 5 8 8 14 16 8 5 etc. . . 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0 0 3 0 0 10 0 0 0 -46 -

Huffman Coding So, to Huffman code a sequence of symbols, first you compute their Frequency Histogram. 5 8 5 5 8 8 14 16 8 5 etc. . . 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0 0 3 0 0 20 0 0 0 -47 -

Huffman Coding So, to Huffman code a sequence of symbols, first you compute their Frequency Histogram. 5 8 5 5 8 8 14 16 8 5 etc. . . 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0 0 30 0 0 0 -48 -

Huffman Coding So, to Huffman code a sequence of symbols, first you compute their Frequency Histogram. 5 8 5 5 8 8 14 16 8 5 etc. . . 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0 0 30 0 0 1 0 0 -49 -

Huffman Coding So, to Huffman code a sequence of symbols, first you compute their Frequency Histogram. 5 8 5 5 8 8 14 16 8 5 etc. . . You continue this until you know pretty well, how frequent all the symbols are. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0 0 30 0 0 1 0 0 -50 -

Huffman Coding Then you assign the shortest code symbols to the most frequently occurring message items. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0 0 30 0 0 1 0 0 -51 -

Dictionary Based Encoding Consider the problem of 24 bit color, in artificial images. A blue sky might contain many copies of (00 00 A 0), and the brown ground might have lots of (A 0 80 00). We can make a "dictionary" or color palette, or look-up table (LUT) like this: -52 -

Dictionary Based Encoding Now you can RLE the information, starting from only one byte per pixel, instead of 3. We can make a "dictionary" or color palette, or look-up table (LUT) like this: -53 -

GIF (Graphics Interchange Format) GIF uses a color palette of 256 colors, so code=1 byte. But. . what if there are MORE than 256 colors in an image? GIF compresser/decompressers (codecs) make a palette of 256 most common colors in the image, and then map all other colors onto these colors. This is a lossy kind of compression. GIF is good for artificial images (especially those without gradients), as they seldom have >256 colors. -54 -

GIF also uses LZW And what, may you ask, is LZW? It's a compression technique originally meant for TEXT. Here's a conceptual picture: In: “the story begins at the beginning, and. . . 000001111122222333333 012345678901234567890 -55 -

GIF also uses LZW And what, may you ask, is LZW? It's a compression technique originally meant for TEXT. Here's a conceptual picture: In: “the story begins at the beginning, and. . . 000001111122222333333 012345678901234567890 Out: “the story 3 Repeat what was in location 3 (a blank) -56 -

GIF also uses LZW And what, may you ask, is LZW? It's a compression technique originally meant for TEXT. Here's a conceptual picture: In: “the story begins at the beginning, and. . . 000001111122222333333 012345678901234567890 Out: “the story 3 Red numbers are stored in a special way, distinct from characters Repeat what was in location 3 (a blank) -57 -

GIF also uses LZW And what, may you ask, is LZW? It's a compression technique originally meant for TEXT. Here's a conceptual picture: In: “the story begins at the beginning, and. . . 000001111122222333333 012345678901234567890 Out: “the story 3 b 2 gin 4 -58 -

GIF also uses LZW And what, may you ask, is LZW? It's a compression technique originally meant for TEXT. Here's a conceptual picture: In: “the story begins at the beginning, and. . . 000001111122222333333 012345678901234567890 Out: “the story 3 b 2 gin 43 a 53(0: 3) Start at 0, copy 3 characters -59 -

GIF also uses LZW And what, may you ask, is LZW? It's a compression technique originally meant for TEXT. Here's a conceptual picture: In: “the story begins at the beginning, and. . . 000001111122222333333 012345678901234567890 Out: “the story 3 b 2 gin 43 a 53(0: 3)3(10: 5). . . begin -60 -

LZW – Document is its OWN LUT! “the story begins at the beginning, and. . . 000001111122222333333 012345678901234567890 Out: “the story 3 b 2 gin 43 a 53(0: 3)3(10: 5). . . Begin The further we go, the more repetitions occur in the text, and so the compression gets better and better. -61 -

LZW – Document is its OWN LUT! “the story begins at the beginning, and. . . 000001111122222333333 012345678901234567890 Out: “the story 3 b 2 gin 43 a 53(0: 3)3(10: 5). . . Begin The further we go, the more repetitions occur in the text, and so the compression gets better and better. -62 -

LZW – Document is its OWN LUT! “the story begins at the beginning, and. . . 000001111122222333333 012345678901234567890 Out: “the story 3 b 2 gin 43 a 53(0: 3)3(10: 5). . . Begin The further we go, the more repetitions occur in the text, and so the compression gets better and better. GIF uses the similarity between scan lines to great effect. (Much easier for artificial, than for natural images. ) -63 -

JPEG (Joint Photo Experts Group) JPEG uses the frequency domain for compression. But what is the frequency of a picture? Low frequency information == "long wavelength" info e. g. is the left side of picture darker than the right side? Brightness A scan line Low frequency -64 -

JPEG (Joint Photo Experts Group) JPEG uses the frequency domain for compression. But what is the frequency of a picture? High frequency information = small details like this picket fence. . Brightness A scan line High frequency -65 -

JPEG (Joint Photo Experts Group) JPEG is smart about throwing away high frequency detail in the COLOR domain, while keeping it in the BRIGHTNESS domain. Wierd though it seems, humans are fuzzier about seeing color than they are about brightness. So we perceive the edges of things by brightness, and then we fill in the color info, based on "neural faith". "There's the door frame. So the door color MUST end at the edge of the door, right? " -66 -

JPEG (Joint Photo Experts Group) The Discrete Cosine Transformation * Takes an 8 x 8 block of pixels * Produces an 8 x 8 block of frequency information * You can then selectively degrade (store less bits of) the high frequency parts, and then RLE the results. Low freq. --> This corner. wikipedia. org -67 -

JPEG (Joint Photo Experts Group) The Discrete Cosine Transformation * Takes an 8 x 8 block of pixels * Produces an 8 x 8 block of frequency information * You can then selectively degrade (store less bits of) the high frequency parts, and then RLE the results. Low freq. --> This corner. wikipedia. org -68 -

JPEG (Joint Photo Experts Group) The Discrete Cosine Transformation Is a special form of the Fourier Transformation Think of 'high frequency' as FINE DETAIL information. "Low Frequency" is GROSS DETAIL information. -69 -

And what shall we DO with this? Let's have a look at a whole bunch of file formats used in Digital Media. under_scores! #$%^& You can see them at http: //en. wikipedia. org/wiki/Graphics_file_format_summary Your task: Look through the TECHNICAL section (bottom of the chart) – find as many terms as you can, that we have discussed today. We will ask a series of questions relevant to data storage. -70 -

Questions such as. . . • What are the relative advantages of TIFF and GIF as archival storage media for scanned images? • Can I recover Illustrator models from a JPEG file? • What is 'color management' and why do I care? • What is interlacing and what does it have to do with progressive scan display devices? YOUR SPECIFIC TASK: Make up AND ANSWER at least one question in the style of the above, about that chart! -71 -