Controlbased Quality Adaptation in Data Stream Management Systems

Control-based Quality Adaptation in Data Stream Management Systems (DSMS)

Data Stream Management System (DSMS) • Continuous data, discarded after being processed • Continuously answers queries • Applications – – – Financial analysis Mobile services Sensor networks Network monitoring More …

DSMS architecture • Network of query operators (1 – 12) • Each operator has its own queue • Scheduler decides which operator to execute • Query results pushed to clients • For our purposes, DSMS can be viewed as a blackbox

Load Shedding • Eliminating excessive load by dropping data items less Qo. S violations • Basic algorithm (Tatbul et al. , 2003): • Key questions – When? – How much? – Where?

What’s missing? • Current solutions focus on steady-state performance – Open-loop control ? • Assuming there inputs reach steady states • However, arrivals are bursty in practice – always in transient state • The solution: closed-loop control

Why Closed-Loop Control • Reduce the effects of modeling error, input and output disturbances • Improve dynamic response • Stabilize unstable systems

Identification of Database System

Control of Database System • Output is delay time • Incoming flow rate fluctuates and unknown • Uncertainties in cost factor

Experiments • Implemented a controller in a real DSMS – Borealis • With bursty synthetic and real data