Online Prediction of the Running Time of Tasks

Online Prediction of the Running Time of Tasks Department of Computer Science Northwestern University pdinda@cs. northwestern. edu Peter A. Dinda • Study of Prediction-based Best-effort Real-time scheduling for distributed interactive applications [In review] • Full randomized over traces • Reliable, introduces appropriate randomness • RTSA library (to be released) • Predict task running time from host load predictions and nominal time (size) of task • Currently compute-bound tasks • Running time reported as a confidence interval (CI) • CI computed using prediction error covariance matrix • Load is “discounted” to deal with I/O priority boosts • Study of such predictions (full paper in [HPDC 2001]) • Metrics: Coverage and Span • Fully randomized over traces • Accurate confidence intervals in most cases • Appropriate coverage with small spans Running Time Advisor API Requested confidence level Nominal time of task (size) Returned confidence level Expected running time CI lower bound Expected running time CI upper bond • Study of Host Load [Scientific Programming, 3: 4, 1999] • Digital Unix 5 second load average, 1 Hz • 39 week-long traces taken at different times of the year • Self-similarity, epochal behavior, correlation • http: //www. cs. northwestern. edu/~pdinda/Load. Traces • Host Load Playback [LCR 2000] • Reconstruct workload using load trace • http: //www. cs. northwestern. edu/~pdinda/Load. Traces/playload • Study of Host Load Prediction [Cluster Computing, 3: 4, 2000] • Fully randomized study on traces • MEAN, LAST, AR, MA, ARIMA, ARFIMA models • AR(16) models most appropriate • RPS Toolkit [CMU-CS-99 -138] • Extensible toolkit for building resource prediction systems • http: //www. cs. northwestern. edu/~pdinda/RPS. html Typical Performance