Greedy Algorithms 1 Let PA denote the problem

Greedy Algorithms

一個活動選擇問題 (設計 1) *Let P(A) denote the problem with A as the given set of proposed activities and S denote an optimal solution of P(A). For any activity i in A, we have 1. i S S is an optimal solution of P(A{ i }). 2. i S S { i } is an optimal solution of P(AN[ i ]) but not necessary an optima solution of P(A{ i }). N[ i ]={j A: Ij Ii } N[ i ] : Greedy Algorithms i 5

一個活動選擇問題 (設計 2) *What kind of activity i in A will be contained in an optimal solution of P(A) : an activity with 1. minimum fi si or 2. minimum |N[ i ]| or 3. minimum fi or 4. minimum si. Answer : . Proof : Let f 1 = min { fi } and S be an optimal solution of P(A). If 1 S then there is one and only one activity in S, say j, such that Ij I 1 . Then S { j} { 1} is also an optimal solution. Greedy Algorithms 6

一個活動選擇問題 (程式+例子) Input: i si 1 1 2 3 3 0 4 5 5 3 6 5 7 6 8 8 9 8 10 9 11 12 fi 4 5 6 7 8 9 10 11 12 13 14 Greedy-ASP(s, f, n) { /* f[1] f[2] … f[n]*/ Ans = {1}; for(i=2, j=1; i n; i++) if(s[ i ] f[ j ]) {Ans = Ans { i }; j = i; } } 1 4 9 2 7 3 8 5 time Greedy Algorithms 11 10 6 7

Greedy 演算法的要素 * Optimal substructure (a problem exhibits optimal substructure if an optimal solution to the problem contains within it optimal solutions to subproblems) * Greedy-choice property * Priority queue or sorting Greedy Algorithms 8

Knapsack Problem (Greedy vs. DP) Given n items: weights: w 1 w 2 … wn values: v 1 v 2 … vn a knapsack of capacity W Find the most valuable load of the items that fit into the knapsack Example: item weight value Knapsack capacity W=16 1 2 $20 2 5 $30 3 10 $50 4 5 $10 Greedy Algorithms 9

Knapsack Problem Given n items: weights: w 1 w 2 … wn values: v 1 v 2 … vn a knapsack of capacity W T[i, j]: the optimal solution using item 1, . . . , i with weight at most j. If wi> j T[i, j] = T[i-1, j]; otherwise T[i, j] = max{ T[i-1, j], wi +T[i-1, j - wi]}. How good is this method? Greedy Algorithms 10

0 -1 and Fractional Knapsack Problem * Constraints of 2 variants of the knapsack problem: Ø 0 -1 knapsack problem: each item must either be taken or left behind. Ø Fractional knapsack problem: the thief can take fractions of items. * The greedy strategy of taking as mush as possible of the item with greatest vi / wi only works for the fractional knapsack problem. Greedy Algorithms 11

Huffman Codes * A very effective technique for compressing data * Consider the problem of designing a binary character code * Fixed length code vs. variable-length code, e. g. : Alphabet: a b c Frequency in a file 45 13 12 Fixed-length codeword 000 001 010 Variable-length codeword 0 101 100 d e f 16 9 5 011 100 101 1101 1100 file length 1 = 300; file length 2 = 224 Compression ratio = (300 224)/300 100% 25% Greedy Algorithms 12

Prefix Codes & Coding Trees * We consider only codes in which no codeword is 0 also a prefix of some a: 45 other codeword. * The assumption is crucial for decoding variable-length code (using a binary tree). E. g. : 1 1 0 0 1 c: 12 b: 13 Greedy Algorithms 0 1 d: 16 0 1 f: 5 e: 9 13

Optimal Coding Trees * For a alphabet C, and its corresponding coding tree T, let f(c) denote the frequency of c C in a file, and let d. T(c) denote the depth of c’s leaf in T. (d. T(c) is also the length of the codeword for c. ) * The size required to encode the file is thus: B(T) = c C f(c) d. T(c) * We want to find a coding tree with minimum B(T). Greedy Algorithms 14

Observation 1 * Any optimal coding tree for C, |C| > 1, must be a full binary tree, in which every nonleaf node has two children. E. g. : for the fixed-length code: a: 45 b: 13 c: 12 d: 16 Greedy Algorithms e: 9 f: 5 15

Observation 2 * Assume C = { c 1, c 2, … , cn}, and f(c 1) f(c 2) … f(cn). Then there exists an optimal coding tree T such that : T: c 1 and d. T(c 1) = d. T(c 2) = maxc C d. T(c) c 2 Greedy Algorithms 16

Observation 3 * If is T an optimal coding tree for C , then T' is an optimal coding tree for C {c 1, c 2} {c'} with f(c') = f(c 1) + f(c 2). T: T': c' c 1 c 2 c 1 Greedy Algorithms c 2 17

Huffman’s Algorithm (例) f: 5 e: 9 c: 12 14 b: 13 f: 5 14 f: 5 b: 13 a: 45 d: 16 a: 45 e: 9 25 d: 16 e: 9 d: 16 c: 12 Greedy Algorithms a: 45 b: 13 18

Huffman’s Algorithm (例-續1) 14 f: 5 25 d: 16 e: 9 c: 12 25 c: 12 a: 45 b: 13 a: 45 30 14 b: 13 f: 5 d: 16 e: 9 Greedy Algorithms 19

Huffman’s Algorithm (例-續2) 25 c: 12 a: 45 30 b: 13 f: 5 14 d: 16 e: 9 a: 45 55 25 c: 12 Greedy Algorithms 30 b: 13 14 f: 5 d: 16 e: 9 20

Huffman’s Algorithm (例-續3) a: 45 55 25 c: 12 0 a: 45 30 b: 13 14 f: 5 0 c: 12 Greedy Algorithms 1 55 0 25 d: 16 e: 9 100 1 1 0 b: 13 30 14 1 d: 16 0 1 f: 5 e: 9 21

Huffman’s Algorithm (pseudo-code) Huffman(C) Q C // Q : priority queue while(|Q| > 1) z Allocate-Node( ) x left[z] Extract-Min(Q) y right[z] Extract-Min(Q) f [z] f [x] + f [y] insert(Q, z) return Extract-Min(Q) Time efficiency: Greedy Algorithms . 22