Performance Tuning in Computer Systems with Structured Bayesian

Performance Tuning in Computer Systems with Structured Bayesian Optimisation and Reinforcement Learning Eiko Yoneki (University of Cambridge) E-mail: eiko. yoneki@cl. cam. ac. uk with Valentin Dalibard and Michael Schaarschmidt Problem: Machine Learning is successful, but it requires human experts to: Ø Blackbox Optimisation o o o § Optimise performance of training time § Optimise selection/engineering features § Optimise hyperparameters § Too slow for high-dimensional parameter space Complex and large parameter space!. . and parameter space may be dynamic and/or combinatorial… Grid search Random search Evolutionary methods Hill climbing… Bayesian optimisation Structured Bayesian Optimisation • Better priors encoded in probabilistic model give you large improvements Ø Deep Reinforcement Learning § • • Lack of good abstractions and reference implementations Unlike supervised training, no dominant execution paradigm Algorithms highly sensitive to hyperparameters Training performance: strong impact by opaque heuristics Reference implementations give good results but difficult to retool different execution modes RLGraph: Programing Model with Modular Computation Graph Structured Bayesian Optimisation RLgraph: Modular Computation Graphs for Deep Reinforcement Learning • • High level programming model: to design and execute RL algorithms across deep learning frameworks and distributed execution engines Generates robust, incrementally testable codes through a strict build mechanism, and by separating algorithm logic from execution semantics