Recent Results in Resource Signal Measurement Dissemination and

Recent Results in Resource Signal Measurement, Dissemination, and Prediction RPS: The Resource Prediction System Toolkit Peter A. Dinda Prescience Lab Department of Computer Science Northwestern University. cs. northwestern. edu • RPS: Measure, Predict, and Disseminate information about dynamic resource supply • Ultimate goal: provide advice to adaptive applications • Publicly available, Extensible, Portable, Easy buy-in • Online predictive modeling • Simple models (MEAN, BESTMEDIAN, LAST…) • Box/Jenkins Models (AR, MA, ARIMA, …) • Fractional ARIMAs • Nonlinear modeling (TARs, Wavelet-decompositions) • [HPDC 99, Cluster 00, Cluster 02, SIGMETRICS 01, IPDPS 02, SC 01, SHAMAN 02] • http: //www. cs. northwestern. edu/~RPS Multiscale Prediction of Network Bandwidth (with Yi Qiao, Jason Skicewicz) • Large study of predictability of binned packet traces • Offline RPS predictors (linear models) • Different resolutions • Both power-of-two binning and low-pass via D 8 wavelets • Over 200 NLANR and other traces • Mostly WANs • All long period traces available at time of trace • Random selection of short-term traces • [NWU-CS-02 -12, NWU-CS-02 -13] • http: //www. cs. northwestern. edu/~plab/Clairvoyance Sensor Fine-grain measurement Resourceappropriate measurement Grid App Resource Signal (periodic sampling) Example: host load Course-grain measurement • Tension between different application needs • Application and sensor needs coupled • Inefficient bandwidth usage, especially in unicast Application Sensor Multicast Diffusion: Zero Cost Information Dissemination Stream Interval (with Brian Cornell, Jack Lange, NSF REU) Level 0 Sensor App Transport Network Header Editing Data Extraction Data Link Physical Sensor Data Ethernet IP TCP Data Level 0 Consumer • Sensor data piggybacked on existing application packets • Number and size of packets unchanged • Hierarchical classification of traces • Short period, Long Period, Bellcore • Predictability using linear models highly variable • Many traces unpredictabile white noise • Predictability varies with resolutiion • Sweet Spot: Predictability often maximized at particular resolution (with Jason Skicewicz) … • Resource signals: Discrete-time signals strongly correlated with resource supply • Host load • Windows performance counters (using Watch. Tower) • Network flow bandwidth and latency (using Remos) • Any text-based source Tsunami: Wavelet-based Approaches Video App Padding • Overwrite unused or redundant fields with sensor data • Shannon entropy of packet headers: 4. 8 bits per byte • In practice: 17. . 32 bits at IP and TCP, padding varies • Implementations: Minet, Linux Kernel Modules • [LCR 2002, NWU-CS-02 -12] • http: //www. cs. northwestern. edu/~plab/Diffusion Wavelet Transform Level M-1 Level L Inverse Wavelet Transform Level M • Sensor sends all levels appropriate to sampling rate • Each application receives levels based on its needs • Applications and sensors decoupled • Level rates decrease logarithmically. • Limited proof of concept implemented in RPS: Works • Wavelet toolbox in next RPS release • Biggest issue: The Delay Problem • Transforms introduce sample delay • Exponential in the number of levels • Affects both streaming and block transforms • Current efforts to overcome the delay problem • Exploit prediction (limited success so far) • Exploit “wavelet-like” decompositions that can trade-off between reconstruction accuracy and delay • [HPDC 2001] • http: //www. cs. northwestern. edu/~plab/Tsunami