Abstract Data Types Stack Queue Amortized analysis 1

Abstract Data Types Stack, Queue Amortized analysis 1

ADT is an interface • It defines – the type of the data stored – operations, what each operation does (not how) – parameters of each operation 2

ADT חוזה בין מתכנת האפליקציה ומיישם מבנה הנתונים Application ממשק Implementation of the Data structure 3

Example: Stacks • • • Push(x, S) : Insert element x into S Pop(S) : Delete the last element inserted into S Empty? (S): Return yes if S is empty Top(S): Return the last element inserted into S Size(S) Make-stack() 4

The Stack Data Abstraction push 5

The Stack Data Abstraction push pop Last in, First out. push 6

A stack application Infix Postfix (2+ 3) * 5 2 3 + 5 * ( (5 * (7 / 3) ) – (2 * 7) ) 5 7 3 / * 2 7 *- • Evaluate an expression in postfix or Reverse Polish Notation 7

A stack application 2 3 + 5 * 3 2 8

A stack application 2 3 + 5 * 5 5 9

A stack application 2 3 + 5 * 25 10

Pseudo-code S ← make-stack() while ( not eof ) do B ← read the next data; if B is an operand then push(B, S) else X ← pop(S) Y ← pop(S) Z ← Apply the operation B on X and Y push(Z, S) return(top(S)) 11

Implementation • We will be interested in algorithms to implement the ADT. . • And their efficiency. . 12

Using an array t A 12 1 3 A[0] A[1] A[2] A[N-1] The stack is represented by the array A and variable t 3 1 12 13

Using an array t A 12 1 3 A[0] A[1] A[2] A[N-1] The stack is represented by the array A and variable t make-stack(): Allocates the array A, which is of some fixed size N, sets t ← -1 14

Operations t A 12 1 3 A[0] A[1] A[2] A[N-1] size(S): return (t+1) empty? (S): return (t < 0) top(S): if empty? (S) then error else return A[t] 15

Pop t A 12 1 3 A[0] A[1] A[2] A[N-1] pop(S): if empty? (S) then error else e ←A[t] t←t– 1 return (e) pop(S) 16

Pop t A 12 1 3 A[0] A[1] A[2] A[N-1] pop(S): if empty? (S) then error else e ←A[t] t←t– 1 return (e) pop(S) 17

Push t A 12 1 3 A[0] A[1] A[2] A[N-1] push(x, S): if size(S) = N then error else t ←t+1 A[t] ← x push(5, S) 18

Push t A 12 1 5 A[0] A[1] A[2] A[N-1] push(x, S): if size(S) = N then error else t ←t+1 A[t] ← x push(5, S) 19

Implementation with lists top size=3 12 x 1 5 x. next x. element 20

Implementation with lists top size=3 12 1 5 make-stack(): top ← null size ← 0 21

Operations top size=3 12 1 5 size(S): return (size) empty? (S): return (top = null) top(S): if empty? (S) then error else return top. element 22

Pop top size=3 12 1 5 pop(S): if empty? (S) then error else e ←top. element top ← top. next size ← size-1 return (e) pop(S) 23

Pop top size=2 12 1 5 pop(S): if empty? (S) then error else e ←top. element top ← top. next size ← size-1 return (e) pop(S) 24

Garbage collection top size=2 1 5 pop(S): if empty? (S) then error else e ←top. element top ← top. next size ← size-1 return (e) pop(S) 25

Push top size=2 1 5 push(x, S): n = new node n. element ←x n. next ← top ← n size ← size + 1 push(5, S) 26

Push top size=2 5 1 5 push(x, S): n = new node n. element ←x n. next ← top ← n size ← size + 1 push(5, S) 27

Push top size=2 5 1 5 push(x, S): n = new node n. element ←x n. next ← top ← n size ← size + 1 push(5, S) 28

Push top size=3 5 1 5 push(x, S): n = new node n. element ←x n. next ← top ← n size ← size + 1 push(5, S) 29

Analysis • Bound the running time of an operation on the worst-case • As a function of the “size”, n, of the data structure • T(n) < 4 n+7 • Too detailed, we are just interested in the order of growth 30

Big-O cg(n) f(n) n 0 32

The running time of our stack and queue operations • Each operation takes O(1) time 34

Stacks via extendable arrays • We do not want our implementation using arrays to be limited to only N elements • When the array is full we will double its size 35

Push t A 12 1 A[0] A[1] A[N-1] push(x, S): if size(S) = N then {allocate a new array of size 2 N copy the old array to the new one; N ← 2 N } t ←t+1 A[t] ← x 36

Push t A 12 1 3 A[0] A[1] 3 4 5 7 3 2 8 1 A[N-1] push(x, S): if size(S) = N then {allocate a new array of size 2 N copy the old array to the new one; N ← 2 N } t ←t+1 A[t] ← x 37 push(5, S)

Push t A 12 1 3 A[0] A[1] 3 4 5 7 3 2 8 1 push(x, S): if size(S) = N then {allocate a new array of size 2 N copy the old array to the new one; N ← 2 N } t ←t+1 A[t] ← x 38 push(5, S)

Push 12 1 3 3 4 5 7 3 2 8 1 t A 12 1 3 A[0] A[1] 3 4 5 7 3 2 8 1 push(x, S): if size(S) = N then {allocate a new array of size 2 N copy the old array to the new one; N ← 2 N } t ←t+1 A[t] ← x 39 push(5, S)

Push t A 12 1 3 A[0] A[1] 3 4 5 7 3 2 8 1 A[2 N-1] push(x, S): if size(S) = N then {allocate a new array of size 2 N copy the old array to the new one; N ← 2 N } t ←t+1 A[t] ← x 40 push(5, S)

Push t A 12 1 3 A[0] A[1] 3 4 5 7 3 2 8 1 5 A[2 N-1] push(x, S): if size(S) = N then {allocate a new array of size 2 N copy the old array to the new one; N ← 2 N } t ←t+1 A[t] ← x 41 push(5, S)

Analysis • An operation may take O(n) worst case time ! • But that cannot happen often. . 42

Amortized Analysis • How long it takes to do m operations in the worst case ? • Well, O(nm) • Yes, but can it really take that long ? 43

44

x 45

x x 46

x x x 47

x x x 48

x x x 49

x x x 50

x x x x 51

x x x x 52

x x x x x x 53

x x x x x x 54

x x x x x x x 55

x x N x x x x x Let N be the size of the array we just copied (half the size of the new array) After N-1 pushes that cost 1 we will have a push that costs 2 N+1 Total time 3 N for N pushes, 3 per push 56

We proved: Theorem: A sequence of m operations on a stack takes O(m) time 58

Amortized Analysis (The bank’s view) • You come to do an operation with a bunch of tokens (amortized cost of the operation) 59

Operation 60

You have to pay for each unit of work by a token Operation 61

Operation If you have more tokens than work you put the remaining tokens in the bank 63

Operation 64

Operation If we have more work than tokens we pay with token from the bank 65

Operation If we have more work than tokens we pay with tokens from the bank 66

Amortized Analysis (The bank’s view) • Suppose we prove that the bank is never in a deficit • The total work is no larger than the total number of tokens • 8*m in our example if m is the number of operations 67

“Easy” push • Each push comes with 3 tokens • If it’s an easy push then we pay with one token for the single unit of work, and put two in the bank. t A 12 1 3 A[0] A[1] 3 4 5 7 3 2 8 1 A[N-1] 69

“Easy” push • Imagine that the tokens in the bank are placed on the items of the stack • We put one token on the item just inserted another token on an arbitrary item without a token t A 12 1 3 A[0] A[1] 3 4 5 7 3 2 8 1 5 A[N-1] 70

t A 12 1 3 A[0] A[1] 3 4 5 7 3 2 8 1 5 6 A[N-1] 71

t A 12 1 3 A[0] A[1] 3 4 5 7 3 2 8 1 5 6 6 7 1 10 4 67 2 5 7 A[N-1] 72

“hard” push • Tokens from the bank “pay” for copying the array into the larger array t A 12 1 3 A[0] A[1] 3 4 5 7 3 2 8 1 5 6 6 7 1 10 4 67 2 5 7 A[N-1] 73

“hard” push • Then you pay a token and put two in the bank as an “easy” push t A 12 1 3 3 4 5 7 3 2 8 1 5 6 6 7 1 10 4 67 2 5 7 t 12 1 3 3 4 5 7 3 2 8 1 5 6 6 7 1 10 4 67 2 5 7 4 74

Need to prove • The balance is never negative: • When we get to an expensive push there are enough tokens to pay for copying • By induction: prove that after the i-th push following a copying there are 2 i tokens in the bank 75

How many tokens we spent ? • Each operation spent 3 tokens 76

Summary • So the total # of tokens is 3 m THM: A sequence of m operations takes O(m) time !! (we finished the proof) • Each operation takes O(1) amortized time 77

Theorem: A sequence of m operations on a stack takes O(m) time proof (3). • We formalize the bank as a potential function 78

Let be a potential function Define Amortized(op) = actual(op) + 79

Amortized(op 1) = actual(op 1) + 1 - 0 Amortized(op 2) = actual(op 2) + 2 - 1 … … + Amortized(opm) = actual(opm) + m- (m-1) i. Amortized(opi) = iactual(opi) + m- 0 i. Amortized(opi) iactual(opi) if m- 0 0 We need to find that gives a nontrivial statement 80

Example: Our extendable arrays Define: the potential of the stack 81

Without copying: Amortized(push) = actual(push) + = 1 + 2(t’+1) – N - (2(t+1) – N) = 1+2(t’-t) = 3 With copying: Amortized(push) = N + 1 + (2 - N) = 3 Amortized(pop) = 1 + 2(t-1) – N - (2 t – N) = -1 82

Amortized(op 1) = actual(op 1) + 1 - 0 Amortized(op 2) = actual(op 2) + 2 - 1 … … + Amortized(opn) = actual(opn) + n- (n-1) i. Amortized(opi) = iactual(opi) + n- 0 3 m ≥ i. Amortized(opi) iactual(opi) if n- 0 0 83

Summary • So the total # of tokens is 3 m THM: A sequence of m operations starting from an empty stack takes O(m) time !! • Each operations take O(1) amortized time 84

Queue • • • Inject(x, Q) : Insert last element x into Q Pop(Q) : Delete the first element in Q Empty? (Q): Return yes if Q is empty Front(Q): Return the first element in Q Size(Q) Make-queue() 85

The Queue Data Abstraction inject 86

The Queue Data Abstraction inject pop inject First in, First out (FIFO). inject 87

Using an array t A 12 1 4 A[0] A[1] A[2] 2 5 A[N-1] pop(Q) 88

Using an array t A 1 4 2 A[0] A[1] A[2] 5 A[N-1] pop(Q) 89

Using an array t A 1 4 2 A[0] A[1] A[2] 5 A[N-1] This would be inefficient if we insist that elements span a prefix of the array 90

Using an array r f A 12 1 4 2 5 A[0] A[1] A[2] A[N-1] f r A A[0] A[1] A[2] Empty queue f=r 91

Using an array r f A 12 1 4 A[0] A[1] A[2] 2 5 A[N-1] pop(Q) 92

Using an array r f A 1 4 A[0] A[1] A[2] 2 5 A[N-1] pop(Q) inject(5, Q) 93

Using an array r f A 1 4 A[0] A[1] A[2] 2 5 5 A[N-1] pop(Q) inject(5, Q) 94

Using an array r f A 1 4 A[0] A[1] A[2] 2 5 5 5 A[N-1] pop(Q) inject(5, Q) pop(Q) 95

Using an array r f A 2 5 5 5 A[0] A[1] A[2] pop(Q) inject(5, Q) pop(Q), inject(5, Q), ………. A[N-1] 96

Using an array r f A 5 5 A[0] A[1] A[2] pop(Q) inject(5, Q) pop(Q), inject(5, Q), ………. A[N-1] 97

Make the array “circular” r A 5 5 f 5 A[0] A[1] A[2] 5 A[N-1] Pop(Q), inject(5, Q), pop(Q), inject(5, Q), ………. 98

Operations r f A 1 4 2 5 A[0] A[1] A[2] A[N-1] empty? (Q): return (f = r) top(Q): if empty? (Q) then error else return A[f] 99

Operations r f A 1 4 2 5 A[0] A[1] A[2] A[N-1] size(Q): if (r >= f) then return (r-f) else return N-(f-r) 100

Operations r A 5 5 A[0] A[1] A[2] f 5 5 A[N-1] size(Q): if (r >= f) then return (r-f) else return N-(f-r) 101

Pop r f A 1 4 2 5 A[0] A[1] A[2] pop(Q): if empty? (Q) then error else e ←A[f] f ← (f + 1) mod N return (e) pop(Q) 102

Pop r f A 1 4 2 5 A[0] A[1] A[2] pop(Q): if empty? (Q) then error else e ←A[f] f ← (f + 1) mod N return (e) pop(Q) 103

Inject r f A 4 2 5 A[0] A[1] A[2] inject(x, Q): if size(Q) = N-1 then error else A[r] ← x r ← (r+1) mod N inject(5, Q) 104

Inject r f A 4 2 5 5 A[0] A[1] A[2] inject(x, Q): if size(Q) = N-1 then error else A[r] ← x r ← (r+1) mod N inject(5, Q) 105

Implementation with lists head size=3 12 1 5 tail inject(4, Q) 106

Implementation with lists head size=3 12 1 5 4 tail inject(4, Q) 107

Implementation with lists head size=3 12 1 5 4 tail inject(4, Q) Complete the details by yourself 108

Double ended queue (deque) • • Push(x, D) : Insert x as the first in D Pop(D) : Delete the first element of D Inject(x, D): Insert x as the last in D Eject(D): Delete the last element of D Size(D) Empty? (D) Make-deque() 109

Implementation with doubly linked lists head tail size=2 13 5 x x. next x. prev x. element 110

Empty list head tail size=0 We use two sentinels here to make the code simpler 111

Push head size=1 tail 5 push(x, D): n = new node n. element ←x n. next ← head. next (head. next). prev ← n head. next ← n n. prev← head size ← size + 1 112

4 head size=1 tail 5 push(x, D): n = new node n. element ←x n. next ← head. next (head. next). prev ← n head. next ← n n. prev← head size ← size + 1 push(4, D) 113

4 head size=1 tail 5 push(x, D): n = new node n. element ←x n. next ← head. next (head. next). prev ← n head. next ← n n. prev← head size ← size + 1 push(4, D) 114

4 head size=1 tail 5 push(x, D): n = new node n. element ←x n. next ← head. next (head. next). prev ← n head. next ← n n. prev← head size ← size + 1 push(4, D) 115

4 head size=2 tail 5 push(x, D): n = new node n. element ←x n. next ← head. next (head. next). prev ← n head. next ← n n. prev← head size ← size + 1 push(4, D) 116

Implementations of the other operations are similar • Try by yourself 117