Adaptive Cleaning for RFID Data Streams Shawn Jeffery

Adaptive Cleaning for RFID Data Streams Shawn Jeffery Minos Garofalakis Michael Franklin UC Berkeley Intel Research Berkeley UC Berkeley 1/14/05 VLDB 2006 Shawn Jeffery, Hi. Fi Project, UCB EECS 9/12/06

RFID: Radio Frequency IDentification Shawn Jeffery Hi. Fi Project UC Berkeley EECS

RFID data is dirty A simple experiment: • 2 RFID-enabled shelves • 10 static tags • 5 mobile tags Shawn Jeffery Hi. Fi Project UC Berkeley EECS

RFID Data Cleaning • RFID data has many dropped readings • Typically, use a smoothing filter to interpolate SELECT distinct tag_id But, RFID_stream how to set the‘ 5 size FROM [RANGE sec’] GROUP BY tag_id of the window? Smoothed output Smoothing Filter Raw readings Time Shawn Jeffery Hi. Fi Project UC Berkeley EECS

Window Size for RFID Smoothing Fido moving Fido resting Reality Raw readings Small window Large window Need to balance completeness vs. capturing tag movement Shawn Jeffery Hi. Fi Project UC Berkeley EECS

Truly Declarative Smoothing • Problem: window size non-declarative • Application wants a clean stream of data • Window size is how to get it • Solution: adapt the window size in response to data Shawn Jeffery Hi. Fi Project UC Berkeley EECS

Itinerary • Introduction: RFID data cleaning • A statistical sampling perspective • SMURF • Per-tag cleaning • Multi-tag cleaning • Ongoing work • Conclusions Shawn Jeffery Hi. Fi Project UC Berkeley EECS

A Statistical Sampling Perspective • Key Insight: RFID data random sample of present tags • Map RFID smoothing to a sampling experiment Shawn Jeffery Hi. Fi Project UC Berkeley EECS

RFID’s Gory Details Antenna & reader Tags Read Cycle (Epoch) E 0 E 1 E 2 E 3 E 4 E 5 E 6 E 7 E 8 E 9 Tag List Tag 1 Epoch Tag. ID Read. Rate Tag 2 0 1 . 9 0 2 . 6 0 3 . 3 Tag 4 (For Alien readers) Shawn Jeffery Hi. Fi Project UC Berkeley EECS

RFID Smoothing to Sampling RFID Read cycle (epoch) Sampling Sample trial Reading Single sample Smoothing window Repeated trials Read rate Probability of inclusion (pi) Now use sampling theory to drive adaptation! Shawn Jeffery Hi. Fi Project UC Berkeley EECS

SMURF • Statistical Smoothing for Unreliable RFID Data • Adapts window based on statistical properties • Mechanisms for: • Per-tag and multi-tag cleaning Shawn Jeffery Hi. Fi Project UC Berkeley EECS

Per-Tag Smoothing: Model and Background • Use a binomial sampling model 1 Si pi piavg (Read rate of tag i) 0 E 1 E 2 E 3 E 4 E 5 E 6 E 7 E 8 E 9 Smoothing Window wi Bernoulli trials Shawn Jeffery Hi. Fi Project UC Berkeley EECS Time (epochs)

Per-Tag Smoothing: Completeness • If the tag is there, read it with high probability Want a large window 1 pi 0 E 0 Reading with a low pi E 1 E 2 E 3 E 4 E 5 E 6 E 7 E 8 E 9 Time (epochs) Expand the window Shawn Jeffery Hi. Fi Project UC Berkeley EECS

Per-Tag Smoothing: Completeness Desired window size for tag i With probability Expected epochs needed to read 1 - Shawn Jeffery Hi. Fi Project UC Berkeley EECS

Per-Tag Smoothing: Transitions • Detect transitions as statistically significant changes in the data The tag has likely left by this point 1 pi 0 E 1 E 2 E 3 E 4 Statistically significant difference E 5 E 6 Shawn Jeffery Hi. Fi Project UC Berkeley EECS E 7 E 8 E 9 Time (epochs) Flag a transition and shrink the window

Per-Tag Smoothing: Transitions # observed # expected readings Is the difference “statistically significant”? Shawn Jeffery Hi. Fi Project UC Berkeley EECS

SMURF in Action Fido moving Fido resting SMURF Experiments with real and simulated data show similar results Shawn Jeffery Hi. Fi Project UC Berkeley EECS

Multi-tag Cleaning • Some applications only need aggregates • E. g. , count of items on each shelf Don’t need to track each tag! • Use statistical mechanisms for both: • Aggregate computation • Window adaptation Shawn Jeffery Hi. Fi Project UC Berkeley EECS

Aggregate Computation • –estimators (Horvitz-Thompson) • Count: • P[tag i seen in a window of size w]: Use small windows to capture movement Use the estimator to compensate for lost readings Shawn Jeffery Hi. Fi Project UC Berkeley EECS

Window Adaptation • Upper bound window similar to per-tag • “Transition” based on variance within subwindows Nw Count N w’ E 0 E 1 E 2 E 3 E 4 E 5 E 6 Shawn Jeffery Hi. Fi Project UC Berkeley EECS E 7 E 8 E 9 Time (epochs)

Multi-tag Scenario Shawn Jeffery Hi. Fi Project UC Berkeley EECS

Ongoing Work: Spatial Smoothing • With multiple readers, more complicated Two rooms, two readers per room C A B Reinforcement D Arbitration A? addressed B? A U B? by A statistical B? A? C? All are framework! U Shawn Jeffery Hi. Fi Project UC Berkeley EECS

Beyond RFID Other sensor data • -estimator for other aggregates Use SMURF for sensor networks Other streaming data • Use SMURF in general streaming systems (e. g. , Telegraph. CQ) Remove RANGE clause from CQL Shawn Jeffery Hi. Fi Project UC Berkeley EECS

Related Work • Commercial RFID middleware • Smoothing filters: need to set smoothing window • RFID-related work • • Rao et al. , Stream. Clean: complementary Intel Seattle, Hi. Fi, ESP: static window size • BBQ, Mauve. DB • • Heavyweight, model-based SMURF is non-parametric, sampling-based • Statistical filters (digital signal processing) • Non-linear digital filters inspired SMURF design Shawn Jeffery Hi. Fi Project UC Berkeley EECS

Conclusions • Current smoothing filters not adequate • Not declarative! • SMURF: Declarative smoothing filter • Uses statistical sampling to adapt window size Shawn Jeffery Hi. Fi Project UC Berkeley EECS

Thanks! Questions? jeffery@cs. berkeley. edu Shawn Jeffery Hi. Fi Project UC Berkeley EECS