Data Structures for Shaping and Scheduling Heap Calendar

Data Structures for Shaping and Scheduling Heap Calendar Queue Timing Wheel ECE 1545

Data Structures Data structures used to perform scheduling and shaping: – Priority queues – Calendar queues – Timing wheel 2

Priority Queue • A priority queue is an abstract data type that stores a collection of data with a priority • Priority queue operations – insert: insert an element with a priority – delete. Min: remove the element with the lowest priority • Property of priority queue: – If two elements x and y are in the queue, if x has a lower priority than y, x will be removed before y

Implementing a Priority Queue: Candidates • Linked list – O(N) worst-case time on either insert() or delete. Min() • Binary Search Tree – O(log N) average time on insert() and delete. Min() – Overkill: all elements are sorted • However, we only need the minimum element • Heap – O(log N) worst case for both insertion and delete. Min operation 4

Heap A heap is a binary tree which satisfies the heap property: “The key of every node is less than or equal to the key of any of its children. ” Properties: – The smallest element in a heap is the root – No conclusion can be drawn about the order of children – The tree is completely filled on all levels except possibly the lowest root 4 5 3 7 6 5

Vector Representation • Use a vector A to store a heap • Store elements in level-order – Parent (k) : – Left (k) : – Right (k): A: 1 R 2 l 3 r root R �k/2 l � 2 k ll 2 k + 1 lr 4 ll 5 lr 6 rl rr 7 rr • Heap property: A[Parent(k)] ≤ A[k] 6

7 insert(x) To insert an element with value x: • Create a new leaf node • If heap property is violated, swap node with parent node • Repeat until parent node is less than x

14 Example from slides by A. Srinivasan (Florida State) 8 insert(14) 14 14

9 delete. Min() • Replace root with the last leaf (last vector element) – This maintains the complete binary tree property but may violate the partially ordered tree property – Start at root: If heap property is violated swap position with smaller child 31 31

31 Example from slides by A. Srinivasan (Florida State) 1 0 delete. Min() 31 31

Calendar Queue • A calendar queue is a priority queue with where the average enqueue and dequeue complexity is O(1) • Implemented as an array of sorted linked lists – Element of the array is called a ”bucket” – Each bucket is associated with a time interval (“width”) – Elements of linked list are sorted buckets linked lists current time pointer … 11

Calendar Queue Inspired by a desk calendar: • one page (bucket) per day of year • items of same day are listed in sorted order • listed items can be for subsequent years – items not in current year are ignored linked lists buckets current time pointer March 18 3/18/2019, 3 pm 2019 3/18/2019, 6: 30 pm 3/18/2020 8 am … 12

Calendar Queue • Calendar queue with n buckets, where each bucket has width w – Value x is added to bucket x/w mod n • Important: Number and width of buckets must be adjusted – Buckets should not have many items in list otherwise enqueue into sorted list is too long – Not too many buckets should be empty otherwise dequeue must search too many buckets – Bucket width should not be too large otherwise data is clustered in few buckets and most other buckets are empty – Bucket width should not be too small otherwise most data is not in “current year” 13

Calendar Queue: Adjusting number of buckets Target: One item for each bucket Rules of thumb: • Double the number of buckets when number of stored entries is twice the number of buckets • Halve the number of buckets when number of stored entries is half the number of buckets • When bucket number is adjusted, copy items to a new calendar queue 14

Calendar Queue: Adjusting the bucket width Target: Set Bucket width equal to the average separation of adjacent elements minimizes enqueue and dequeue times Rules of thumb: • Recompute bucket width when copying to a new calendar queue • Select samples around place of current time pointer and estimate average separation of adjacent elements • Set bucket width to estimate of average separation 15

Calendar Queue: Tightly clustered items 1000 items, 512 buckets • 500 items uniformly in [0, 0. 1] • 500 items uniformly in [5. 6, 5. 7] bucket width = 0. 002 buckets current time pointer linked lists [0, 0. 002) 0 5. 6 [0. 002, 0. 004) 0. 002 5. 6002 [0. 004, 0. 006) 0. 004 5. 6004 0. 006 5. 6006 [0. 006, 0. 008) … [1. 022, 1. 024) • After removing elements in [0, 0. 01], current time pointer has to cycle through 54 “years” for next items • Solution: After running through an “empty year”, make global search for smallest item 16

Calendar Queue: Evaluation • Evaluation from 1988 (on 5 Mhz PC) • Exponential separation between adjacent elements • Comparison: – Linear linked list – Splay tree • Supports claim of constant enqueue and dequeue times 17

Calendar Queue: Evaluation • Different distributions of separation: 18

Calendar Queue: Applications • Main application for calendar queue is the management of events in discrete event simulations • Suitable to compute timers (for timeout values) • Suitable for shaping functions that compute the release time (eligibility time) of packets • Less suitable for a sorted scheduler queues (e. g. , EDF, WFQ) – Due to potential search for smallest element 19

Timing Wheel • O(1) data structure for managing timers • Inspired by timing wheel in logic simulators 20

Simple Timing Wheel 7 0 1 2 6 5 4 3 • Enqueue and dequeue is O(1) • Drawback: required wheel size can be large • Assumes that timers are set to at most MAX value into the future (Here: MAX=8), i. e. , within a rotation from the current position • A cursor in the timing wheel moves one location every time unit • A list for each location • If cursor is at position i and if timer is set for i+j, add timer to position (i+j) mod MAX 21

Hashed Timing Wheel # of rounds remaining 0 7 0 1 2 6 5 2 1 4 4 3 0 1 2 • Works for arbitrary values of the timer • Record the number of remaining rounds with the stored entry • If cursor is at position i and if timer is set for i+j, – add timer to position (i+j) mod MAX – with remaining rounds (i+j) div MAX • Lists for a position can be sorted or unsorted • In the worst-case, enqueue is O(n) with sorted lists • If # entries is less than wheel size, average enqueue should be O(1) 22

Hierarchical Timing Wheel 10: 25: 30 hours wheel 0 1 23 2 … … 11 10 15 20 minutes wheel 0 1 59 2 … 3 25 … • Seconds pointer advanced per clock tick • If seconds pointer wraps, move the minutes pointer • If minutes pointer wraps, move the hour pointer • Example: Current time: 10 h 25 m seconds wheel 0 1 59 2 … 3 … 4 30 s Timer value:

Hierarchical Timing Wheel 11: 00 hours wheel 0 1 23 2 … … 11 10 15 20 minutes wheel 0 1 59 2 … 3 15 … 0 1 20 seconds wheel 59 2 … 3 … 4 • At 11 h 0 m 0 s, remove all entries from 11 h slot and move to minutes wheel, into position of minute value • In minutes wheel, store the seconds

Hierarchical Timing Wheel 11: 15: 00 hour wheel 0 1 23 2 … … • At 11 h 15 m 0 s, remove all entries from 15 m slot and move to seconds wheel, into position of seconds value 11 10 minute wheel 0 1 59 2 … 3 15 … 0 1 20 seconds wheel 59 2 … 3 20 4 • At 11 h 15 m 20 s, the seconds wheel moves to 20 s and finds the entry

Timing Wheels: Summary • Application areas for timing wheels similar to calendar queues: – Timers – Simulations – Shapers • Hashed timing wheel is essentially a calendar queue • Main difference of timing wheel and calendar queue: Timing wheel does not resize 26