Comp Sci 201 Huffman Greedy Algorithms Jeff Forbes

Comp. Sci 201 Huffman & Greedy Algorithms Jeff Forbes April 11, 2018 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 1

V is for … • Virtual Memory • When 4 Gb becomes the cloud • Virtual Reality • Augmenting IRL • Viral Video • From meme to everywhere • vi and vim • Nothing matters less than … 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 2

PFTD • Review of Huffman Compression • Key aspects of compression at a high-level • Decompression with more detail • Looking at bits and information • What's in a. jpeg compared to. mp 3 • Greedy Algorithms & Vote Rigging 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 3

Properties of Huffman coding • Want to minimize weighted path length L(T)of tree T • • wi is the weight or count of each encoding i • di is the leaf corresponding to encoding i • How do we calculate character (encoding) frequencies? • Huffman coding creates pretty full bushy trees? • When would it produce a “bad” tree? • How do we produce coded compressed data from input efficiently? 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 4

Compressing kjv 10. txt Encoding Length # values with this length 3 1, 159, 124 13 1, 108 4 1, 487, 471 14 664 5 712, 325 15 476 6 485, 333 16 225 7 261, 611 17 71 8 84, 107 18 44 9 81, 467 19 22 10 48, 019 20 11 11 21, 065 21 3 12 1, 863 22 6 23 6 Comp. Sci 201, Spring 2018, Greedy 5

Decompressing public void decompress(Bit. Input. Stream in, Bit. Output. Stream out) 1. Check that file is compressed? Read magic number. 2. read. Tree. Header: Recreate tree from header 3. read. Compressed. Bits: Parse compressed data from input stream and write decoded output to output stream • Check if correct? 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 6

Uncompression with Huffman • We need the trie to uncompress • 00010011001101111 • As we read a bit, what do we do? • Go left on 0, go right on 1 • When do we stop? What to do? • How do we get the trie? • Could store 256 counts, use same code • Instead store trie: read and write 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 7

Reading and Writing Huff Trie • Similar to concept/techniques in Tree APTs • Distinguish interior and leaf nodes • In huff we label with 0 and 1 respectively • In Tree APT we store "null" explicitly • 8 4 x 6 x x 12 10 x x 15 x x • Number? Read two subtrees • X ? Return null, no recursion • 8 [ 4 x 6 x x] [12 10 x x 15 x x] • 12 [10 x x] [15 x x] 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 8

Compressing public void compress(Bit. Input. Stream in, 1. 2. 3. 4. 5. Bit. Output. Stream out) read. For. Counts: counts the number of occurrences of each character make. Tree. From. Counts: constructs the Huffman tree make. Codings. From. Tree: generates the encodings for each character write. Header: writes the magic number and a pre-order traversal of the Huffman tree write. Compressed. Bits: writes the encoded bits for each character in the uncompressed text • Check if correct? 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 9

Making a Tree from Counts? • Begin with a forest of single-node trees (leaves) • Node stores two values: character and count • Repeat until there is only one node left: root of tree • Remove two minimally weighted trees from forest Greedy! • Create new tree with minimal trees as children, • New tree root's weight: sum of children (character ignored) Priority. Queue<Huff. Node> forest = new Priority. Queue<>(); for (int i = 0; i < 256; i++) if (frequencies[i] > 0) // computed elsewhere forest. add(new Huff. Node(i, frequencies[i])); while (forest. size() > 1) { Huff. Node left = forest. remove(); Huff. Node right = forest. remove(); forest. add(new Huff. Node(-1, left. weight()+right. weight(), left, right)); } Huff. Node root = forest. remove(); 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 10

Writing code out to file • How do we go from characters to encodings? • Build Huffman tree • Root-to-leaf path generates encoding • Recall Leaf. Trails APT • Need way of writing bits out to file • Platform dependent? • Complicated to write bits and read in same ordering • See Bit. Input. Stream and Bit. Output. Stream classes • How do we know bits come from compressed file? • Store a magic number 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 11

Trie used to encode & decode • Compress: create encodings for each char/leaf • Similar to Leaf. Trails APT • Each 8 -bit chunk/char mapped to encoding, e. g. , in an array with codings[‘A’] == “ 010101” • Decompress: Use trie/tree to decompress bits • Trie unique to each file, part of compressed file • Compress: write trie, Decompress: read trie 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 12

Good Comp. Sci 201 Assignment? • Array of character/chunk counts, or is this a map? • Map character/chunk to count, why array? • Priority Queue for generating tree/trie • Do we need a heap implementation? Why? • Tree traversals for code generation, uncompression • One recursive, one not, why and which? • Deal with bits and chunks rather than ints and chars • The good, the bad, the ugly • Create a working compression program • How would we deploy it? Make it better? • Benchmark for analysis • Whats a corpus? Comp. Sci 201, Spring 2018, Greedy 13

Other methods • Adaptive Huffman coding • Lempel-Ziv algorithms • Build the coding table on the fly while reading document • Coding table changes dynamically • Protocol between encoder and decoder so that everyone is always using the right coding scheme • Works well in practice (compress, gzip, etc. ) • More complicated methods • Burrows-Wheeler (bunzip 2) • PPM statistical methods Comp. Sci 201, Spring 2018, Greedy 14

Data Compression Year Scheme Bit/Char 1967 ASCII 7. 00 1950 Huffman 4. 70 1977 Lempel-Ziv (LZ 77) 3. 94 1984 Lempel-Ziv-Welch (LZW) – Unix compress 3. 32 1987 (LZH) used by zip and unzip 3. 30 1987 Move-to-front 3. 24 1987 gzip 2. 71 1995 Burrows-Wheeler 2. 29 1997 BOA (statistical data compression) 1. 99 12/4/2020 Comp. Sci 201, Spring 2018, Greedy • Why is data compression important? • How well can you compress files losslessly? • Is there a limit? • How to compare? • How do you measure how much information? 15

Anita borg: 1949 -2003 “Dr. Anita Borg tenaciously envisioned and set about to change the world for women and for technology. … she fought tirelessly for the development technology with positive social and human impact. ” “Anita Borg sought to revolutionize the world and the way we think about technology and its impact on our lives. ” http: //www. youtube. com/watch? v=1 y. Pxd 5 jqz_Q 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 16

Reasoning Formally about APTs • Buying votes with minimal expenditures • http: //www. cs. duke. edu/csed/newapt/voterigging. html • I have N votes. Other candidates: v 1, v 2, …, vm • Which candidate do I take a vote from? • When do I stop taking votes? • Will I have enough votes to win? • Will I spend the least amount of money? 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 17

Greed is good, greed is right • https: //www. youtube. com/watch? v=VVx. YOQS 6 ggk 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 18

Greedy Algorithms • Locally optimal decision leads to global optimum • Make change with minimal coins: $0. 63, $0. 32 • What if we’re out of nickels? $0. 63, $0. 32 • Autocomplete: Heaviest path in Trie is best, minimal exploration • Huffman coding: pick smallest trees • Maximal vote-holder is best to buy, minimize $$ 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 19

Vote Rigging: from idea to code • Where do we store purchased votes? • From whom do we buy a vote? • Do this with minimal code to write? • Not concerned with efficiency, but realizing idea • One easy idea to find from whom we buy vote? • Ideas from everyone 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 20

Buying Votes • Buy one, buy another, buy the last vote: winner! $$ 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 21

Reasoning Formally • When this loop finishes, what do we know? while (v[0] <= maximum(v[1]…v[N])) • In loop body, from whom do we buy a vote? What do we do with the vote? Reverse. Sort(v[1], …, v[N]) V[0]++ V[1]--; • Last re-sort? 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 22

Knowing and Using the API • java. util. List has a sub. List method • “backed” by original list, look at examples • Benefits and Drawbacks • From int[] to List<Integer> for sub. List • Use a. set(0, a. get(0)+1) not a[0]++ 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 23

One edge-case from all green List<Integer> list = new Array. List<>(); for(int k : votes) list. add(k); int stolen = 0; List<Integer> others = list. sub. List(1, list. size()); while (list. get(0) <= Collections. max(others)) { Collections. sort(others, Comparator. reverse. Order()); list. set(0, list. get(0) + ; (1 others. set(0, others. get(0) - 1); stolen += 1; { return stolen; 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 24

WOTO http: //bit. ly/201 -s 18 -0411 -1 • Greedy algorithms • Huffman Coding • Dijkstra with Priority Queue • A* with Priority Queue 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 25

Test 2 • Median: 47/63 (74. 6%) • Std Dev: 8. 07 (12. 82%) • Regrade requests must be submitted by Sunday at 11: 59 pm • Regrade requests must address criteria • Review solutions 12/4/2020 Comp. Sci 201, Spring 2018, Greedy 26