Amortized Analysis The average cost of a sequence

Amortized Analysis The average cost of a sequence of n operations on a given Data Structure. Aggregate Analysis Accounting Method CS 333 / Cutler Amortized Analysis 1

Amortized Analysis • Amortized analysis computes the average time required to perform a sequence of n operations on a data structure • Often worst case analysis is not tight and the amortized cost of an operation is less that its worst case. • Analogy (making coffee) CS 333 / Cutler Amortized Analysis 2

Applications of amortized analysis • • Vectors/ tables Disjoint sets Priority queues Heaps, Binomial heaps, Fibonacci heaps • Hashing CS 333 / Cutler Amortized Analysis 3

Difference between amortized and average cost • To do averages we need to use probability • For amortized analysis no such assumptions are needed • We compute the average cost per operation for any mix of n operations CS 333 / Cutler Amortized Analysis 4

Operations on Data Structures • A data structure has a set of operations associated with it. • Example: A stack with – push(), pop() and Multi. Pop(k). • Often some operations may be slow while others are fast. • push() and pop() are fast. • Multi. Pop(k) may be slow. • Sometimes the time of a single operation can vary CS 333 / Cutler Amortized Analysis 5

Methods • Aggregate analysis- the total amount of time needed for the n operations is computed and divided by n • Accounting - operations are assigned an amortized cost. Objects of the data structure assigned a credit • Potential – The prepaid work (money in the “bank”) is represented as “potential” energy that can be released to pay for future operations CS 333 / Cutler Amortized Analysis 6

Aggregate analysis • n operations take T(n) time • Amortized cost of an operation is T(n)/n CS 333 / Cutler Amortized Analysis 7

Stack - aggregate analysis • A stack with operations • Push, Pop and Multipop(S, k) while not empty(S) and k>0 do Pop(S); k: =k-1 end while CS 333 / Cutler Amortized Analysis 8

Stack - aggregate analysis • Push and Pop are O(1) (to move 1 data element) • Multipop is O(min(s, k)) where s is the size of the stack and k the number of elements to pop. • Assume a sequence of n Push, Pop and Multipop operations CS 333 / Cutler Amortized Analysis 9

Stack - aggregate analysis • Each object can be popped only once for each time it is pushed • So the total number of times Pop can be called ( directly or from Multipop) is bound by the number of Pushes <=n. CS 333 / Cutler Amortized Analysis 10

Stack - aggregate analysis • A sequence of n Push and Pop operations is therefore O(n) and the amortized cost of each is O(n)/n=O(1) CS 333 / Cutler Amortized Analysis 11

Stack Example: Op/Moves <=2 Operation Stack Start pop c a b a c b a pop b a push b push c a b a c b a Multipop(3) pop a 6 Operation: 6 Moves CS 333 / Cutler Amortized Analysis 4 Operation: 6 Moves 12

Accounting Method • Charge each operation an (invented) amortized cost. – Often different from actual run time cost. Some operations may have an amortized cost larger than runtime, others may have less. – Unlike businesses we do not want to make a profit – We want to cover the actual cost • Amount charged but not used in performing an operation is stored with objects of the data structure CS 333 / Cutler Amortized Analysis 13

Accounting method • Later operations can use stored amount to pay for their actual cost • Credit balance must not go negative (always enough to pay for performance of future operations) CS 333 / Cutler Amortized Analysis 14

Stack - amortized analysis • We assign the amortized costs: $2 for Push $0 for both Pop and Multipop • For a sequence of n Push and Pop operations the total amortized cost is at most 2 n or O(n) CS 333 / Cutler Amortized Analysis 15

Stack - amortized analysis • Each time we do a Push we pay $1 for the actual cost of the Push and the element has a credit of $1. • Each time an element is popped we take the $1 credit to pay for it • Thus the balance is always nonnegative CS 333 / Cutler Amortized Analysis 16

k bit binary counter Increment (A) i=0 while i < length[A] and A[i] = 1 A[i] = 0 i ++ if i < length[A] A[i] = 1 • Initially the counter contains 0 • Eventually it becomes 2 k -1 • Next it is reset to 0 A k -1 1 CS 333 / Cutler Amortized Analysis 0 17

3 bit counter k=3 Bit 2 1 0 D No. 0 000 1 001 2 010 3 011 4 100 5 101 6 110 7 111 Flips 2 4 8 Aggregate analysis • Count number of times a bit is flipped. • Let number increments n = 2 k (if n < 2 k analysis similar) • A[0] flipped n times • A[1] flipped n/21 times • … • A[k - 1] flipped n/2 k-1 times CS 333 / Cutler Amortized Analysis 18

Accounting method • Charge amortized cost of $2 to set a bit to 1 • When a bit is set to 1, pay $1 for actual cost and store $1 with bit • Note: at all times a bit with value 1 has $1 • When a bit is reset to 0 use $1 to pay for actual cost • $2 per Increment operation CS 333 / Cutler Amortized Analysis 19

Accounting method • Let the value stored in the counter be: $1 $1 $0 $0 $1 $1 $1 1 1 0 0 1 1 1 • After increment and a payment of $2 = $1 +$1: $1 $1 $0 $0 $0 1 1 0 0 0 CS 333 / Cutler Amortized Analysis 20

Dynamic table (object table, hash table, vector, etc) • The table is dynamic • We can’t predict its maximum size • We would like to avoid allocating a lot of unused space (reasonable load balance) • May not be able to avoid table overflow. • Overflow should not cause run time failure CS 333 / Cutler Amortized Analysis 21

java. util. Vector • A built in class which is a “growable” array. • The user can set: – initial. Capacity - the initial capacity of the vector. – capacity. Increment - the amount by which the capacity is increased when the vector overflows. • The default for capacity. Increment is to double the size CS 333 / Cutler Amortized Analysis 22

Dynamic table • Idea: Allocate more memory as needed. • Duplicate the size of the table after each overflow. • After each duplication must copy elements from old to new table • We assume for now the only operation is insert and calculate amortized cost • Table sizes: 1, 2, 4, 8, …, 2 k CS 333 / Cutler Amortized Analysis 23

Aggregate method Op. Size Cost The table before after 1 0 1 1 1 2 2 2 1+1 3 3 2 4 2+1 Copy 4 4 1 5 4 8 4+1 6 8 8 1 7 8 8 1 24/9 8 8 8 1 9 8 16 CS 3338+1 / Cutler Amortized Analysis 1 2 3 4 5 6 7 8 9. . 24

Aggregate Analysis • Let the number of inserts be 2 k < n 2 k+1 • (Note: 2*2 k = 2 k+1 < 2 n) • At this point the size of the table is 2 k+1 • After n inserts: – The total cost for copy operations only is 1 +2 + 4 +. . . + 2 k = 2 k+1 - 1 < 2 n – The total cost for n inserts (without copy) is n. • Total < 3 n • Therefore the amortized time is O(1). CS 333 / Cutler Amortized Analysis 2 k n 2 k 25

Accounting analysis • Charge each insert $3. • When the table is not full, use $1 for the cost of insert, and store $2 with element • When the table doubles from m to 2 m: – m/2 elements that never moved before have $2 credit, – m/2 elements which already moved have $0 CS 333 Amortized 26 – After copy all m/ Cutler elements have $0 credit Analysis

Accounting analysis Size = 8 4 copied elements Size = 4 with $0 2 copied elements $0 $0 with $0 Size =8 $0 $0 2 new elements 4 copied elements $0 $0 with $2 $0 $0 with $0 $0 4 new elements $2 $0 with $2 $2 $2 CS 333 / Cutler Amortized Analysis 27

java. util. Vector fixed increment • Let the number of inserts n satisfy c 0 +(m-1)c < n c 0 +mc • So (n- c 0 )/c m <1+ (n- c 0 )/c and m = (n) • At this point the size of the array is c 0 +mc • After the nth insert: – The total time for copy operations is ? Amortized 28 – The total time. CS 333 for/ Cutler n inserts (without copy) Analysis

java. util. Vector fixed increment Initial capacity c 0 0 c 0 1 2 Capacity increment c c 0 c c m-1 c 0 c c Cost for m vector copies = mc 0 + c (1 + 2+ 3 + … +(m-1)) = mc 0 + c m(m-1)/2= (m 2) = (n 2) CS 333 / Cutler Amortized 29 Analysis

java. util. Vector fixed increment • After the nth insert: – The total time for copy operations is (n 2) – The total time for n inserts (without copy) is n. • Average insert time is (n) CS 333 / Cutler Amortized Analysis 30