Compression Compression q Compression ratio how much is

  • Slides: 26
Download presentation
Compression

Compression

Compression q Compression ratio: how much is the size reduced? q Symmetric/asymmetric: time difference

Compression q Compression ratio: how much is the size reduced? q Symmetric/asymmetric: time difference to compress, decompress? q Lossless; lossy: any information lost in the process to compress and decompress? q Adaptive/static: does the compression dictionary change during processing?

Huffman encoding q Statistical encoding q Requires knowledge of relative frequency of elements in

Huffman encoding q Statistical encoding q Requires knowledge of relative frequency of elements in the string q Sender and receiver must both know encoding chosen q Create a tree structure that assigns longest representations to most rarely used symbols

Huffman example q First, start with statistics about occurrence of symbols in the text

Huffman example q First, start with statistics about occurrence of symbols in the text to be compressed. Ø That assumption might not be right for every message. Ø Sometimes expressed as percentage, sometimes as relative frequencies q A(7) E(9) S(8) I(4) D(4) G(3) N(2) P(2) R(1) W(1) q We want shorter codes for A, E, S longer codes for R, W to minimize the overall message lengths q We are saying that in analysis of a large body of typical text, we find that the occurrence of E is 9 times more common than the occurrence of W, for example

Constructing the Code First, combine the least frequently used symbols The weight (frequency) of

Constructing the Code First, combine the least frequently used symbols The weight (frequency) of the pair (R, W) is 2, of the pair (N, P) is 4 and of the existing tree combined with N is also 2 4 2 OR N(2) R(1) W(1) N(2) P(2) R(1) W(1) Either will work. The second version gets us closer to the canonical form. A(7) E(9) S(8) I(4) D(4) G(3) N(2) P(2) R(1) W(1)

4 2 R(1) 5 N(2) P(2) G(3) W(1) Insert the next least frequently used

4 2 R(1) 5 N(2) P(2) G(3) W(1) Insert the next least frequently used symbol (P, with weight=2). We could attach the 2 to the 4 in the tree to get a node with weight = 6 or we could join the P(2) and G(3) nodes to get a combined node with weight = 5. We choose the lower weight. A(7) E(9) S(8) I(4) D(4) G(3) N(2) P(2) R(1) W(1)

4 2 R(1) 5 N(2) P(2) 8 G(3) D(4) I(4) W(1) Next, we have

4 2 R(1) 5 N(2) P(2) 8 G(3) D(4) I(4) W(1) Next, we have D and I at weight 4 each. We could add one of them to the 4 node in the tree for a new weight of 8 or add them to each other for a weight of 8. To get the canonical form, let’s join D and I. A(7) E(9) S(8) I(4) D(4) G(3) N(2) P(2) R(1) W(1)

4 2 R(1) 5 N(2) P(2) 8 G(3) D(4) I(4) W(1) Now, we need

4 2 R(1) 5 N(2) P(2) 8 G(3) D(4) I(4) W(1) Now, we need to insert the A at weight 7. What are the options? Join the 4 + 5 = 9 or A(7)+4 = 11, A(7)+5 = 12, A(7)+8 = 15, 7+S(8) = 15. Where shall we put the A? A(7) E(9) S(8) I(4) D(4) G(3) N(2) P(2) R(1) W(1)

9 15 4 2 R(1) 5 N(2) P(2) 8 G(3) D(4) A(7) I(4) W(1)

9 15 4 2 R(1) 5 N(2) P(2) 8 G(3) D(4) A(7) I(4) W(1) Our lowest weight possible is 4=5 = 9, so we join those trees. Then our option is to join the A(7) to the 9 or to the 8, so we choose the 8. A(7) E(9) S(8) I(4) D(4) G(3) N(2) P(2) R(1) W(1)

24 9 4 2 R(1) 8 5 N(2) 17 15 P(2) G(3) D(4) A(7)

24 9 4 2 R(1) 8 5 N(2) 17 15 P(2) G(3) D(4) A(7) S(8) E(9) I(4) W(1) Continuing, we need to insert the S(8) and the E(9). We can get a weight of 17 by joining them together. We could have inserted them in a different place, but would have broken the canonical form. A(7) E(9) S(8) I(4) D(4) G(3) N(2) P(2) R(1) W(1)

41 0 24 0 0 R(1) 2 1 4 1 N(2) 9 1 1

41 0 24 0 0 R(1) 2 1 4 1 N(2) 9 1 1 0 P(2) 1 5 1 G(3) 0 D(4) 0 0 8 15 1 1 S(8) A(7) I(4) W(1) A(7) E(9) S(8) I(4) D(4) G(3) N(2) P(2) R(1) W(1) 17 1 E(9)

Completed code q q q q q S(8)= E(9)= A(7)= I(4)= N(2)= P(2)= G(3)=

Completed code q q q q q S(8)= E(9)= A(7)= I(4)= N(2)= P(2)= G(3)= D(4)= R(1)= W(1)= 10 11 0101 0010 0011 0100 00001 q Average code length: q A has weight 7 and length 3, etc. q 8*2 + 9*2 +7*3 +4*4 +2*4 + 3*4 +4*4 + 1*5 +1*5 = 125 q 125/41 =3. 049 A(7) E(9) S(8) I(4) D(4) G(3) N(2) P(2) R(1) W(1)

In class exercise q Working in pairs, encode a message of at least 10

In class exercise q Working in pairs, encode a message of at least 10 letters using the code we just generated. q Do not leave any spaces between the letters in your message. q Pass the message to some other team. q Make sure you give and get a message. q Decode the message you received.

Entropy per symbol Entropy, E, is information content Entropy is inversely proportional to the

Entropy per symbol Entropy, E, is information content Entropy is inversely proportional to the probability of occurrence E = -∑pi log 2 pi i=1, n where n is the number of symbols and pi is the probability of occurrence of the ith symbol This is the lower bound on weighted compression -the goal to shoot for. How well did we do in our code? 2. 3 compared to 3. 049

Properties of the Huffman code q q Variable length code Prefix property Average bits

Properties of the Huffman code q q Variable length code Prefix property Average bits per symbol (entropy) Huffman codes approach theoretical limit for amount of information per symbol q Static coding. Code must be known by sender and receiver and used consistently

Dynamic Huffman Code q Build the code as the message is transmitted. q The

Dynamic Huffman Code q Build the code as the message is transmitted. q The code will be the best for this particular message. q Sender and receiver use the same rules for building the code.

Constructing the tree q Sender and receiver begin with an initial tree consisting of

Constructing the tree q Sender and receiver begin with an initial tree consisting of a root node and a left child with a null character and weight = 0 q First character is sent uncompressed and is added to the tree as the right branch from the root. The new node is labeled with the character, its weight is 1 and the tree branch is labeled 1 also. q A list shows the tree entries in order

Example q banana: r Initial tree *(0) r Transmit b *(0) Weight (1) =

Example q banana: r Initial tree *(0) r Transmit b *(0) Weight (1) = number of times that character has occurred so far b(1) *(0) b(1) List version of the tree

A new character seen q Whenever a new character appears in the message, it

A new character seen q Whenever a new character appears in the message, it is sent as follows: Ø send the path to the empty node Ø send the uncompressed representation of the new character. Ø Place the new character into the tree and update the list representation. r Null node moves down to make room for the new node 1 as its sibling *(0) a(1) 1 b(1) a (1) List is formed by reading the tree left to right, bottom level to top level ba

Another character r 2 q n 1 *(0) n(1) 1 a(1) 2 b(1) *(0)

Another character r 2 q n 1 *(0) n(1) 1 a(1) 2 b(1) *(0) b(1) a (1) n(1) List entries are not in non decreasing order. Adjust the list and show the corresponding tree. r b(1) 2 *(0) n(1) 1 a(1) b(1) 2 1 *(0) a (1) n(1) (Note all left branches are coded as 1, all right branches as 0) ban

Our first repeated character r b(1) q a 3 1 *(0) Again there is

Our first repeated character r b(1) q a 3 1 *(0) Again there is a problem. The numbers in the list do not obey the requirement of non decreasing order 2 1 *(0) a (2) n(1) r a(2) *(0) n(1) 1 a(2) b(1) 3 b(1) n(1) Adjust the list and make the tree match *(0) n(1) 1 b(1) a(2) 2 Note that the 3 changed to a 2 as a result of the tree restructuring. bana

Another repeat Code sent for this n will be 101 corresponding to the original

Another repeat Code sent for this n will be 101 corresponding to the original position of n. Then the restructuring will be done. r q n a(2) 3 2 *(0) b(1) *(0) n(2) 2 b(1) a(2) 3 n(2) Another misfit. Move b(1) and adjust the tree as needed. r a(2) 3 1 n(2) *(0) b(1) 1 n(2) a(2) 3 *(0) b(1) banan

One more letter q a This a is encoded as 0. No restructuring of

One more letter q a This a is encoded as 0. No restructuring of the tree is needed. r a(3) 3 1 *(0) n(2) b(1) *(0) b(1) 1 n(2) a(3) 3 banana

In class exercise q Create the dynamic Huffman code for the “message” = Tennessee

In class exercise q Create the dynamic Huffman code for the “message” = Tennessee

Summary q Compression seeks to minimize the amount of transmission by making efficient representations

Summary q Compression seeks to minimize the amount of transmission by making efficient representations for the data. q Static compression keeps the same codes and depends on consistency in the distribution of characters to code q Dynamic compression adjusts as it works to allow the most efficient compression for the current message.

Some extra resources q Huffman coding resources: http: //www. dogma. net/Data. Compression/Huffman. shtml

Some extra resources q Huffman coding resources: http: //www. dogma. net/Data. Compression/Huffman. shtml