NONCLAIRVOYANT PRECEDENCE CONSTRAINED SCHEDULING July 9 th 2019

NON-CLAIRVOYANT PRECEDENCE CONSTRAINED SCHEDULING July 9 th, 2019 SAHIL SINGLA PRINCETON UNIVERSITY AND INSTITUTE FOR ADVANCED STUDY JOINT WORK WITH NAVEEN GARG, ANUPAM GUPTA, AND AMIT KUMAR

WHAT IS SCHEDULING? time At any moment execute • at most 1 job per machine • at most 1 machine per job Preemption & Migration allowed 2 / 17

NON-CLAIRVOYANT SCHEDULING Preemption & Migration allowed 3 / 17

PRECEDENCE CONSTRAINTS Graham’ 66 Grandl et al. ’ 16 Apache Tez and Apache Spark 4 / 17

WHAT IS KNOWN WITH PRECEDENCE? • Offline: • Wtd Completion: Constant factor algorithms Hall-Schulz-Shmoys-Wein’ 97 • Wtd Flow: Speedup necessary & constant factor with constant speedup Kulkarni-Li’ 18 • Online/Non-clairvoyant: • Same release date • Only one value for the entire “hyper-job” • Not much is known • Some works assume each DAG is a “hyper-job”: Greedy works Agrawal-Li-Lu-Moseley’ 16 Robert-Schabanel’ 08 CAN WE DO ONLINE WITH PRECEDENCE? 5 / 17

MAIN RESULTS All jobs in a DAG released together 6 / 17

OUTLINE • INTRODUCTION: PRECEDENCE CONSTRAINED SCHEDULING • OUR ALGORITHM: INDEPENDENCE GRAPH &NASH SOCIAL WELFARE • PRIMAL-DUAL ANALYSIS • OTHER RESULTS AND OPEN PROBLEMS 7 / 17

WEIGHTED ROUND-ROBIN ALGORITHM Easy Chains Medium Trees 8 / 17

USE A DEPENDENCY GRAPH Inverted star Machines 9 / 17

MAXIMIZE NASH SOCIAL WELFARE (NSW) • Want a “Fair” Solution • NSW used previously by Munagala et al. (without precedence) • Convex Program Real Rate Virtual Rate Machines • Generalizes round-robin • Similar to Eisenberg-Gale for Fisher markets • Combinatorial Water-Filling algo 10 / 17

OUTLINE • INTRODUCTION: PRECEDENCE CONSTRAINED SCHEDULING • OUR ALGORITHM: INDEPENDENCE GRAPH &NASH SOCIAL WELFARE • PRIMAL-DUAL ANALYSIS • OTHER RESULTS AND OPEN PROBLEMS 11 / 17

PROOF APPROACH 12 / 17

PRIMAL-DUAL LP Jobs finished Bounded Speed Precedence 13 / 17

SETTING DUALS AND LP FEASIBILITY Jobs that help 14 / 17

OUTLINE • INTRODUCTION: PRECEDENCE CONSTRAINED SCHEDULING • OUR ALGORITHM: INDEPENDENCE GRAPH &NASH SOCIAL WELFARE • PRIMAL-DUAL ANALYSIS • OTHER RESULTS AND OPEN PROBLEMS 15 / 17

WEIGHTED FLOW TIME All jobs in a DAG released together 16 / 17

CONCLUSION QUESTIONS? 17 / 17

FURTHER SLIDES 18 / 17

RELATIONSHIP TO FISHER MARKETS 19 / 17

PRIMAL-DUAL LP Jobs finished Bounded Speed Precedence Alternately, 20 / 17

DUAL FEASIBILITY (IDEA) 21 / 17

RELATED MACHINES 22 / 17