MS 101 Algorithms Instructor Neelima Gupta nguptacs du

MS 101: Algorithms Instructor Neelima Gupta ngupta@cs. du. ac. in

TOC • Lower Bounds (adversary)

Lower Bounds • Any algorithm to find min and max of n keys by comparison of keys must do at least 3 n/2 – 2 comparisons in the worst case. • Proof: – Assume that the keys are distinct – Let x denotes max and y denotes min – We must know that every key other than x has lost to some comparison ----- n-1 unit of information required. – Also, we must know that every key other than y has won in some comparison ----- n-1 unit of information required. – Total : 2 n – 2 unit of information required to declare that x is max and y is min.

Adversary • Devises a strategy that gives away minimum information. Key status W L WL N Meaning Has won at least one comparison and never lost Has lost at least one comparison and never won Has won and lost at least one comparison Has not yet participated in a comparison

Status of keys x and y compared by an algo Adversary chooses N, N W, N/ WL, N W, W L, L W, L/WL, L/W, W L WL, WL x >y x<y x>y x>y New status W, L …, WL WL, … No change Consistent with No assigned values Change Units of new information 2 1 1 0 0

• Make sure that input constructed is consistent.

Example: Constructing an input using adversary’s rules. Comparison x 1 x 2 X 3 x 4 x 5 x 6 status value status value INITIAL N * N * N * N X 1. X 2 X 1, X 5 X 3, X 4 X 3, X 6 X 3, X 1 X 2, X 4 X 5, X 6, X 4 TABLE 5. 3 *

Example: Constructing an input using adversary’s rules. Comparison x 1 x 2 X 3 x 4 x 5 x 6 status value status value INITIAL N * N * N * N X 1. X 2 X 1, X 5 X 3, X 4 X 3, X 6 X 3, X 1 X 2, X 4 X 5, X 6, X 4 TABLE 5. 3 *

Example: Constructing an input using adversary’s rules. Comparison x 1 status value INITIAL N * X 1, X 2 W 20 x 2 X 3 x 4 x 5 x 6 status value status value N * N * N L 10 N * X 1, X 5 X 3, X 4 X 3, X 6 X 3, X 1 X 2, X 4 X 5, X 6, X 4 TABLE 5. 3 N * N * *

Example: Constructing an input using adversary’s rules. Comparison x 1 status value INITIAL N * X 1, X 2 W 20 X 1, X 5 W 20 x 2 X 3 x 4 x 5 x 6 status value status value N * N * N L 10 N * N L X 3, X 4 X 3, X 6 X 3, X 1 X 2, X 4 X 5, X 6, X 4 TABLE 5. 3 * 5 N * *

Example: Constructing an input using adversary’s rules. Comparison x 1 status value INITIAL N * X 1, X 2 W 20 X 1, X 5 W 20 X 3, X 4 x 2 X 3 x 4 x 5 x 6 status value status value N * N * N L 10 N * N L W 15 X 3, X 6 X 3, X 1 X 2, X 4 X 5, X 6, X 4 TABLE 5. 3 L 8 * 5 N * *

Example: Constructing an input using adversary’s rules. Comparison x 1 status value INITIAL N * X 1, X 2 W 20 X 1, X 5 W 20 x 2 X 3 x 4 x 5 x 6 status value status value N * N * N L 10 N * N L X 3, X 4 W 15 X 3, X 6 W 15 X 3, X 1 X 2, X 4 X 5, X 6, X 4 TABLE 5. 3 L * N * * 5 8 L 12

Example: Constructing an input using adversary’s rules. Comparison x 1 status value INITIAL N * X 1, X 2 W 20 X 1, X 5 W 20 x 2 X 3 x 4 x 5 x 6 status value status value N * N * N L 10 N * W 15 X 3, X 6 W 15 WL 20 * N L X 3, X 4 X 3, X 1 N W 25 X 2, X 4 X 5, X 6, X 4 TABLE 5. 3 L * N * * 5 8 L 12

Example: Constructing an input using adversary’s rules. Comparison x 1 status value INITIAL N * X 1, X 2 W 20 X 1, X 5 W 20 x 2 X 3 x 4 x 5 x 6 status value status value N * N * N L 10 N * W 15 X 3, X 6 W 15 X 2, X 4 WL * N L X 3, X 4 X 3, X 1 N 20 W WL L X 6, X 4 TABLE 5. 3 * 5 L L X 5, X 6 N 8 25 10 * * 8 12

Example: Constructing an input using adversary’s rules. Comparison x 1 status value INITIAL N * X 1, X 2 W 20 X 1, X 5 W 20 x 2 X 3 x 4 x 5 x 6 status value status value N * N * N L 10 N * W 15 X 3, X 6 W 15 X 2, X 4 WL * N L X 3, X 4 X 3, X 1 N 20 W WL L * N * * 5 8 L 12 L 3 25 10 L X 5, X 6 8 WL X 6, X 4 TABLE 5. 3 5

Example: Constructing an input using adversary’s rules. Comparison x 1 status value INITIAL N * X 1, X 2 W 20 X 1, X 5 W 20 x 2 X 3 x 4 x 5 x 6 status value status value N * N * N L 10 N * W 15 X 3, X 6 W 15 X 2, X 4 WL * N L X 3, X 4 X 3, X 1 N 20 W WL L * N * * 5 8 L 12 L 3 WL 3 25 10 L 8 X 5, X 6 WL X 6, X 4 L 5 2 Key values X 1=20, X 2=10, X 3=25, X 4=2, X 5=5, X 6=3. MAX= X 3 MIN =X 4 TABLE 5. 3

EXPLANATION: 1. First column shows a sequence of comparison that may be carried out by some algorithm 2. Remaining columns shows the status and value assigned to the keys by the adversary. 3. Each row after first comparison contains only the entries relevant to current comparison. 4. In 5 th comparison when X 3 and X 1 are compared adversary increases the value of X 3 because it is supposed to win. 5. In 8 th it changes the value of x 4 and x 6 consistent with its rules. 6. After first five comparison X 3 has lost at least once so it is max. 7. After all comparison X 4 is the only key that has never won so it is min. Result: In this example the algorithm did eight comparison. the worst case lower bound for six keys is (3/2) *6 -2=7

Proof of theorem 5. 1 • We have to show that the adversary rule will force an algorithm to do at least 3 n-2 comparison to get 2 n-2 unit of information. • Algorithm can get two units of information only when two keys have not been included in any previous comparison. • Suppose n is even: Algorithm does n/2 comparison for unseen keys, so it can get at most n unit of information

Proof of theorem 5. 1 cont. • Algorithm needs n-2 additional unit of information so it must do at least n-2 more comparison • Thus to get 2 n-2 information it must do at least n/2+n-2=3 n/2 -2 comparisons. • For odd n: Total no of comparison is 3 n/2 -3/2

Proof of theorem 5. 1 cont. • Algorithm needs n-2 additional unit of information so it must do at least n-2 more comparison • Thus to get 2 n-2 information it must do at least n/2+n-2=3 n/2 -2 comparisons. • For odd n: Total no of comparison is 3 n/2 -3/2

Lower bound for Largest n Second Largest • Any algorithm that works by comparing keys to find the second largest in a list of n keys must do at least n + log n – 2 comparisons. • If we can show that `max’ participated in at least log n comparisons we are done because – Any algorithm that computes second largest must compute the largest. Why? • To know that x is second largest, it must know that it is not the largest. That is it must loose to at least one comparison ( and in fact exactly one, since if it looses to two or more it is not the second largest. ). And, when x looses to exactly one key , it knows that key must be the largest. – If ‘max’ looses to only one key, we know that key must be the second largest. But what if ‘max’ looses’ to more than one key. – So, any algorithm must do at least n-1 comparisons (where ‘max’ is compared with the rest of the keys ) and compare the “loosers to max” to get the second largest.

Adversary • Assign weight w(x) to each key x. • Initially w(x) = 1 for all x.

Adversary modifies the weights as per the following strategy. Case w(x) > w(y) Adversary response x>y w(x) = w(y) > 0 x > y w(y) > w(x) y>x w(x) = w(y) = 0 consistent with previous results New weights w(x)←w(x) + w(y), w(y) ← 0 w(y)←w(x) + w(y), w(x) ← 0 No change

Observations • A key has lost a comparison iff its weight is zero. i. e. once a key has lost a comparison, its weight will never become non zero in future. Proof: Notice that once a key y has lost a comparison its weight becomes zero. Once its weight becomes it can never be compared with a key x whose weight w(x) is less than w(y). Hence w(y) never changes hereafter.

Observations • The sum of the weights is always n. This is true initially and it is preserved when weights are updated. • When the algorithm stops, only one key has a non-zero weight. Otherwise there would be at least two keys (say a and b) that never lost a comparison. The adversary can give arbitrarily high and distinct values to these two keys. – Now, if the algorithm declares a as largest and b as second largest, adversary can always choose the values the other way round and make the algorithm’s choice of largest and second largest incorrect.

• Claim: In any algorithm to compute the largest , the largest must be compared with at least ceil(log n) keys in the worst case. • Proof: Let x be the key that has non-zero weight when the algorithm stops. Then clearly x = largest. We’ll show that x has directly won over at least ceil(log n) distinct keys.

• Let wk = w(x) be the weight of x after the kth comparison won by x against a previously undefeated key. Then, we’ll prove that wk ≤ 2 wk-1 – If x has won against few previously defeated keys in between, its weight does not change. So, – wk = wk-1 + w(y) where y is the previously undefeated key – Clearly, w(y) ≤ wk-1 for else y wouldn’t loose to x. – Thus, wk ≤ 2 wk-1 • Let K be the number of comparisons x wins against previously undefeated keys, then n = w K ≤ 2 K w 0 = 2 K Thus K ≥ log n or K ≥ ceil(log n )

Assignment 5 • Generate a worst case input for largest and second largest problem using the adversary strategy discussed here.