SelfAdjusting Computation Umut Acar umutcs cmu edu Carnegie

Self-Adjusting Computation Umut Acar umut@cs. cmu. edu Carnegie Mellon University Aladdin Project Review

Example: Static Convex Hulls o Convex hull = smallest enclosing polygon 9/17/2020 Aladdin Project Review 2

Example: Dynamic Convex Hulls o Insert/delete points 9/17/2020 Aladdin Project Review 3

Example: Dynamic Convex Hulls o Insert/delete points 9/17/2020 Aladdin Project Review 4

Kinetic Convex Hulls 9/17/2020 Aladdin Project Review 5

Algorithms Community o o Problem specific solutions Many papers n o Agarwal, Atallah, Basch, Bentley, Chan, Cohen, Demaine, Eppstein, Even, Frederickson, Galil, Guibas, Henzinger, Hershberger, King, Italiano, Janardan, Mehlhorn, Micciancio, Mulmuley, Overmars, Sleator, Tamassia, Tarjan, Thorup. . . Efficient algorithms but complex 9/17/2020 Aladdin Project Review 6

Programming-Languages Community o o o Incremental computation Static-to-dynamic transformation Many papers n o Most effective techniques n n o Alpern, Demers, Field, Hoover, Horwitz, Hudak, Liu, de Moor, Paige, Pugh, Reps, Ryder, Strom, Teitelbaum, Weiser, Yellin. . . Static dependence graphs Memoization Work well for a limited class of problems 9/17/2020 Aladdin Project Review 7

Self-Adjusting Computation: the Best of Both Worlds o o o Language techniques Algorithmic techniques Dynamize any persistent program n n o Persistent data structures Any single assignment or functional program Closely match problem-specific performance n n n 9/17/2020 Sorting Kinetic Data Structures Dynamic Trees Aladdin Project Review 8

Representing Executions with Function Call Trees o Control Dependences main g f h k 9/17/2020 m Aladdin Project Review 9

Dynamic Dependence Graph = Call Tree + Data Dependences o o Control Dependences Data Dependences main g f h m k 1 a 9/17/2020 2 b 3 c Aladdin Project Review 10

Change Propagation main g f h m k 1 a 9/17/2020 2 b 3 c Aladdin Project Review 11

Change Propagation main g f h m k 0 1 a 9/17/2020 2 b 3 c Aladdin Project Review 12

Change Propagation main g f h m k 0 1 a 9/17/2020 2 b 3 c Aladdin Project Review 13

Change Propagation main g f h k 0 a 9/17/2020 2 b 3 c Aladdin Project Review 14

Change Propagation main g f h k 0 a 9/17/2020 2 b 5 3 c Aladdin Project Review 15

Change Propagation main g f h k 0 a 9/17/2020 2 b 5 3 c Aladdin Project Review 16

Change Propagation main g f h k 0 a 9/17/2020 2 b 5 c Aladdin Project Review 17

Change Propagation main g f h k 0 a 9/17/2020 2 b 5 c Aladdin Project Review 18

Summary: Change Propagation main g f h m k 1 0 a 9/17/2020 2 b 3 c Aladdin Project Review 19

Summary: Change Propagation main g f h 1 0 a h m k 2 b g f k 3 c 0 a 2 b 5 c Acar, Blelloch, Harper: POPL 2002 9/17/2020 Aladdin Project Review 20

Trace = Call Trees + Values Read 3 5 4 5 8 1 9/17/2020 2 5 3 8 Aladdin Project Review 21

Trace = Call Trees + Values Read 3 3 5 4 8 1 9/17/2020 5 5 2 5 3 4 8 7 4 1 5 Aladdin Project Review 1 5 22

Edit Distance = Time for Red Calls 3 3 5 4 8 1 9/17/2020 5 5 2 5 3 4 8 7 4 1 5 Aladdin Project Review 1 5 23

Trace Stability: A Measure of Input Sensitivity 3 3 5 4 8 1 5 5 2 5 3 4 8 7 4 1 5 Acar, Blelloch, Harper, Vittes, Woo: SODA 2004 9/17/2020 Aladdin Project Review 24

Dynamic Dependence Graphs: Re-execute Functions from Scratch 9/17/2020 Aladdin Project Review 25

Combine DDGs & Memoization: Re-use Subtrees 9/17/2020 Aladdin Project Review 26

Adaptive Memoization: Re-use Approximate Subtrees Acar, Blelloch, Harper: 2004 9/17/2020 Aladdin Project Review 27

Adaptive Functional Programming [Acar, Blelloch, Harper: POPL ’ 02] o o o Algorithmic and language techniques for self -adjusting computation Efficient representation for dynamic DGs Selective dependence tracking n Changeable values o n Stable values o 9/17/2020 Values sensitive to input change Value insensitive to input change Aladdin Project Review 28

Modifiable References o Holds a changeable value n o Change and propagate Primitives: n mod: create a modifiable n read: read a modifiable n write: write to a modifiable 9/17/2020 Aladdin Project Review 29

Example with Modifiables a = mod (write 2) b = mod (read x=a in write(f(x))) o 2 a f(2) b read’s must write 9/17/2020 Aladdin Project Review 30

Modifiables and Dynamic Dependence Graphs read x=a in a = mod (write 2) 2 b = mod (read x=a in write(f(x))) a f f(2) b 9/17/2020 Aladdin Project Review 31

Control Dependences with Time Stamps a 2 4 -15 read x=a. . . o f(2) b Time-stamps created by read/write n o 1 Edges = Reads + Time Interval Dietz-Sleator data structure Overhead n n 9/17/2020 Time = O(1) Space = O(#reads + #mods) Aladdin Project Review read x=a. . . o 4 15 Execution 32

Standard Quicksort datatype int list = NIL | CONS (int * int list) fun qsort (l: int list) = fun qs (l, rest) = case l of NIL => rest | CONS(h, t) => (smaller, bigger) = partition(h, t) sbigger = qs (bigger, rest) qs (smaller, CONS(h, sbigger)) begin qs(l, NIL) end 9/17/2020 Aladdin Project Review 33

Self-Adjusting Quicksort datatype int mlist = NIL | CONS (int * (int mlist) mod) fun qsort (l: int mlist) = fun qs (l, rest) = read l’=l in case l’ of NIL => write (rest) | CONS(h, t) => (less, bigger) = partition(h, t) sbigger = mod (qs(bigger, rest))) qs(less, CONS(h, sbigger)) begin mod (qs(l, NIL)) end 9/17/2020 Aladdin Project Review 34

Execute, Change, and Propagate (* Step 1: Execute *) in = [4, 5, 1, 0, 2]; mod_in = mod. List. from. List(in); mod_out = qsort (mod_in); print(mod_out); (* [0, 1, 2, 4, 5] *) (* Step 2: Change Input *) insert (mod_in, 3, 5); (* Step 3: Update *) propagate(); print(mod_out); 9/17/2020 (* [0, 1, 2, 3, 4, 5] *) Aladdin Project Review 35

Performance of Quicksort o Self-adjusting Quicksort updates its output n n o o o in expected O(logn) time for changes at the end of the input in expected O(n) time for changes at the beginning of the input Merge Sort: Expected O(logn) any change Insertion Sort: Expected O(n) any change Sum/Min/Max on lists: Expected O(logn) 9/17/2020 Aladdin Project Review 36

Trace Stability and Performance [SODA ‘ 04] o Theorem: If program P is O(f(n))-stable for some class of input changes D then using Dynamic DGs output-update takes n o q(n) = time for the priority queue n n o O(q(n) £ f(n)) time. Worst case: q(n) = O(log(f(n)) Typical case: q(n) = O(1) Theorem reduces dynamic problems to static 9/17/2020 Aladdin Project Review 37

Kinetic Data Structures o o Introduced by Basch, Guibas, Hershberger ‘ 97 Compute a property of moving objects n o Example: Kinetic convex hulls Previous work: Hand-designed, hand-coded 9/17/2020 Aladdin Project Review 38

Our Work: Automatic Kinetization [Acar, Blelloch, Vittes, ‘ 04] o Comparisons = modifiables n o When a comparison fails n n o Compute the failure time of comparisons Flip its value (true ! false, false ! true) Change propagate Applications n n n 9/17/2020 Mergesort, Quicksort: expected O(1) stable Mergehull: expected O(logn) stable Quickhull, Chan’s: No meaningful bounds Aladdin Project Review 39

Static Quickhull fun find. Hull(line as (p 1, p 2), l, hull) = ll = filter l (Geo. lineside line) case ll of NIL => CONS(p 1, hull) | _ => pm = max (Geo. dist line) ll right = find. Hull((pm, p 2), ll, hull, dest) find. Hull((p 1, pm), ll, right) fun quick. Hull l = (mx, xx) = minmax (Geo. min. X, Geo. max. X) l find. Hull((mx, xx), l, CONS(xx, NIL) 9/17/2020 Aladdin Project Review 40

Self-Adjusting Quickhull fun find. Hull(line as (p 1, p 2), l, hull) = ll = filter l (Geo. lineside line) mod(read ll’=ll in case ll’ of NIL => write (CONS(p 1, hull)) | _ => pm = max (Geo. dist line) ll right = mod (find. Hull((pm, p 2), ll, hull)) find. Hull((p 1, pm), ll, right)) fun quick. Hull l = (mx, xx) = minmax (Geo. min. X, Geo. max. X) l mod (find. Hull ((mx, xx), l, CONS(xx, NIL))) 9/17/2020 Aladdin Project Review 41

Kinetic Quickhull fun find. Hull(line as (p 1, p 2), l, hull) = ll = filter l (Kin. lineside line) mod(read ll’=ll in case ll’ of NIL => write (CONS(p 1, hull)) | _ => pm = max (Kin. dist line) ll right = mod (find. Hull((pm, p 2), ll, hull)) find. Hull((p 1, pm), ll, right)) fun quick. Hull l = (mx, xx) = minmax (Kin. min. X, Kin. max. X) l mod (find. Hull ((mx, xx), l, CONS(xx, NIL))) 9/17/2020 Aladdin Project Review 42

Kinetic Quickhull Demo 9/17/2020 Aladdin Project Review 43

Dynamic Trees Problem [Sleator, Tarjan, ’ 83] o Fundamental problem n o Given a forest of weighted trees support n n o Flow algorithms, dynamic graph algorithms Tree Changes: e. g. , insert/delete edges Queries: e. g. , heaviest edge on a path? Problem-specific solutions: n Link-Cut Trees [Sleator, Tarjan, ‘ 83, ‘ 85] n n 9/17/2020 Topology Trees [Frederickson, ‘ 93] Top Trees[Alstrup, Holm, Licht. , Thorup, ‘ 97] Aladdin Project Review 44

Self-Adjusting Tree Contraction [SODA ‘ 04] o Tree Contraction [Miller, Reif, ‘ 85] n n o o Randomization for symmetry breaking Expected O(logn)-stable An expected O(logn)-time solution to dynamic trees problem Simple, general, and efficient 9/17/2020 Aladdin Project Review 45

microseconds Experimental Results: LC-Trees & SA-Tree Contraction Path Query 9/17/2020 Weight Change Insert & Delete Edges Aladdin Project Review 46

Other Applications and Queries o Link-Cut Trees ü O O 9/17/2020 Path queries Subtree queries Non-local queries O Centers O Medians O Diameters o SA-Tree Contraction ü Path queries ü Subtree queries ü Non-local queries ü Centers ü Medians ü Diameters Aladdin Project Review 47

Conclusion o Foundations n n n o Applications n n n o Dynamic Dependence Graphs AFL language Trace Stability Sorting, min/max, . . . Kinetic Data Structures Dynamic trees Future Work: Multiple write, more applications. . 9/17/2020 Aladdin Project Review 48

Credits o o o o 9/17/2020 Guy Blelloch Robert Harper Srinath Sridhar Robert Tarjan Jorge Vittes Virgina Vassilevska Renato Werneck Maverick Woo Aladdin Project Review 49