Tiny Networking David Culler University of California Berkeley

Tiny Networking David Culler University of California, Berkeley Intel Research Berkeley http: //webs. cs. berkeley. edu 11/14/2002 NEC Networking

Vast Networks of Tiny Devices • Past 25 years of internet technology built up around powerful dedicated devices that are carefully configured and very stable – local high-power wireless subnets at the edges – 1 -1 communication between named computers • Here, . . . • • • every little node is potentially a router work together in application specific ways collections of data defined by attributes connectivity is highly variable must self-organize to manage topology, routing, etc and for power savings, radios may be off 99% of the time 11/14/2002 NEC Networking

Directed Diffusion Concept • Nodes express ‘interest’ in data with certain attributes (sinks) • Establishes gradient from sources 11/14/2002 Estrin, Govindan, Heideman NEC Networking

Directed Diffusion Concept • Nodes express ‘interest’ in data with certain attributes (sinks) • Establishes gradient from sources • Sources generate data • Useful paths reinforced, others suppressed • in-network aggregation • nested queries 11/14/2002 NEC Networking

Huge Design Space: Application • Traffic – any-to-any, all-to-one, one-to-all, collection of sub-groups, . . . – steady low-BW steam of readings/findings, bursts of data, periodic logs, . . . • Duration – years (hazard alarm), months (field season), when the big red button is pressed • Available infrastructure – power, base-stations, location • Mobility, Stationary – all fixed, all mobile, mixture, changing environment • Placement / Physical Topology / Scale – arbitrary, controlled, unknown • Network Services – time synchronization, localization, proximity, . . . => Best way to get a networking solution now is to define your application context 11/14/2002 NEC Networking

Design Space: Underlying Technology • Link (Radio) Technology – range, control of signal strength, noise tolerance – channel capacity, coding, error rates – single channel, multi-channel, tunable • Device Density, Failures over time • MAC (media access control) – channel sensing, back-off, protocols, collision behavior, link-level acks – time synch, energy aware • Power management & Energy Constraints – scheduling, functional allocation • Transmission Rate Control • Topology Formation – hierarchical spine, geographic, static, dynamic • Routing – single path, multipath, passive participation – explicit vs implicit nbhd detection 11/14/2002 NEC Networking

Losing the forest for the trees • Systems side – clever fix to one particular aspect, but only looking at it in sliver of the design space • Theory side – assuming cell coverage is a disk of radius r with sharp boundary – within radius Pconnect = 1, outside Pconnect = 0 – eg. , unit disk-graphs for routing, maximal independent sets, min. dominating sets, leader election • Wireless communication between small devices is inherently noisy, unpredictable, & non-deterministic – – – 11/14/2002 Radio signal fades with distance Interference Multipath (reflections) Collisions Mobility or not NEC Networking

Surge Demo 11/14/2002 NEC Networking

Local Operations => Global Behavior • Nodes ‘sense’ network environment – uncertain, partial information • Packets directed to a “parent” neighbor – all other neighbors “hear” too – carry additional organizational information • Each nodes builds estimate of neighborhood – adjusted with every packet and with time • Interactively selects parent • Routes traffic upward Þ Collectively they build and maintain a stable spanning tree Þ takes energy to maintain structure Predictable global behavior built from local operations on uncertain data node # dept h 17 1 6 3 . . . 11/14/2002 NEC Networking child? yes parent? % link goodnes s yes 90 . 7 75 . 6

Wireless “Connectivity” • Controlled study on 13 x 13 array of Rene nodes • Single transmitter • Record fraction of packets received at each node • Many packets at each of several transmit power levels Þcomplex fall-off over substantial range Þ‘range’ defined by CEP Contours of probability of reception from center node for range of transmit power strengths with Ganesan, Woo, Krishnamacheti, Estrin 11/14/2002 NEC Networking

Alternative Perspective • Develop algorithms that are built fundamentally upon a probabilistic connectivity structure – Embrace noise, rather than fight it – the network is really another sensor (and actuator) • Utilize simple, local rules, rather than complex protocols • Challenge: obtain predictable global behavior • Challenge: interfaces for imperfect operation 11/14/2002 NEC Networking

Simple Epidemic Broadcast Schema Local Rule: if (new mcast) then take local action retransmit modified request • Should forms roughly breadth-first spanning tree • Examples: Network wakeup, command propagation – Build spanning tree » record parent • Naturally adapts to available connectivity • Minimal state and protocol overhead => surprising complexity in this simple mechanism 11/14/2002 NEC Networking

Network Discovery: Radio Cells 11/14/2002 NEC Networking

Network Discovery 11/14/2002 NEC Networking

Behavior at Scale • Variations in connectivity • With many nodes, likely that one far away will hear • Long links tend to be asymmetric • Structure dominated by contention 11/14/2002 NEC Networking

Final Tree 11/14/2002 NEC Networking

Power Laws ? • Most nodes have very small degree (ave =. 92) • Some have degree = 15% of the population • Few large clusters account for most of the edges 11/14/2002 NEC Networking

Open Territory => Many Children • Example: Level 1 11/14/2002 NEC Networking

Open Territory => Many Children • Example: Level 2 – variation in distance 11/14/2002 NEC Networking

Open Territory => Many Children • Example: Level 3 – long links 11/14/2002 NEC Networking

Importance of Asymmetric Links • Asymmetric Link: – >65% successful reception in one direction – <25% successful reception in the other direction • 10%-25% of links are asymmetric • Many long links are asymmetric – in large field it is likely that someone far away can hear you – what does this mean for protocol design? 11/14/2002 NEC Networking

Collisions are primary factor • Nodes out of range may have overlapping cells – hidden terminal effect • Collisions => these nodes hear neither ‘parent’ – become stragglers • As the tree propagates – folds back on itself – rebounds from the edge – picking up these stragglers. • Seen in many experiments • Mathematically complex because behavior is not independent beyond singe cell 11/14/2002 NEC Networking

Probabilistic Connectivity Model • Radio signal fades with distance in complex manner depending on environment • Radio receiver has complex behavior to extract signal • What matters to algorithms is whether packets are delivered or not Þ work directly with probabilistic communication model Þ but which one? • Calculate comm. rates for numerous transmit/receive pairs at range of distances 11/14/2002 NEC Networking

Fall-off of Probability of Comm. Low Power 11/14/2002 High Power NEC Networking

Drive Simulation from Empirical Stochastic behavior clear 11/14/2002 transition silent NEC Networking

Example Cell Coverage from Model feet 11/14/2002 feet NEC Networking

Reception Model with Collisions • Second experiment with two nodes sending at once, record which nodes hear which one – follows P success closely • When does a second sender collide? – Clear comm. region => YES – silent region => NO – transition region => collides if would have communicated • Reception Model – Assume pij is the probability of success (i->j), Probability for B to receive A’s message = pab* 11/14/2002 i=collider(1 -pib) NEC Networking

Common Special Case: Data Gathering • Collection of nodes take periodic samples • Stream data towards a root node • Root announces interest – depth = 0 • Nodes listen to neighbors • When hear neighbor with smaller depth – start transmitting data to good neighbor with smallest depth – set own depth to one greater and include with data • Data transmission continuously reinforces / adjusts routes 11/14/2002 NEC Networking

Use Case Assumptions • Application – N-to-1 all data, no aggregation – Each node generates small packets at regular interval » Appln phase shift on collision – Routed to a specific node, e. g, base station or root of request – data rate below saturation – 7 x 7 grid, 10 ft spacing => several hops • Underlying – link-level acknowledgement (may be used for retransmission) – CSMA with simple channel sensing, fixed backoff, initial random delay – Receivers always attentive » may use sampled listening • Broadcast: choose parent = first contact • Shortest Path: choose parent = closer (which one) • Min path loss: choose parent = next step on min loss path 11/14/2002 NEC Networking

Max-Path-Reliability routing • each node maintains estimate of loss rate over entire path to root • select nbr on the minimum loss path as parent – Pi[to root] through j = P[link i, j] * Pj[to root] – assuming loss rate along path is independent of how packets enter the path • Subtleties – – – 11/14/2002 P=0. 86 95 65 estimating link rates transient error in link rate may lead to cycles rate of updates stability vs responsiveness 60 congestion warm-up phase NEC Networking 95 70 95 99

Broadcast • overall success rate: 18. 9% 11/14/2002 NEC Networking

SP 50 Path reliability • overall success rate: 44. 8% 11/14/2002 NEC Networking

MPR path reliability • overall success rate: 54. 7% 11/14/2002 NEC Networking

Routing Distance Distribution SD 50 MPR 11/14/2002 NEC Networking

Distribution of success rates MPR 11/14/2002 SP 50 NEC Networking

Observations • Better success rate with more, better hops • Partial information causes temporary loss – cycles introduced when link error estimates are poor – tracks std. dev. of link error rates – if difference between candidates not statistically significant, choose based on hop count – especially important during warm-up phase » also after topology changes • Shortest Path will tend to use most marginal neighbors – although range is highly variable, for particular pairs of nodes at particular distances connectivity is very bimodal 11/14/2002 NEC Networking

SP 75 Path Reliability • overall success rate: 52. 7% (vs 54. 7% for MPR) 11/14/2002 NEC Networking

Multihop Path-Rate Contours Min Path Loss 11/14/2002 Min Hop (75% nbr) NEC Networking

Hop-by-hop retransmission • Decent neighbors become good neighbors if you are willing to chat for a moment • may choose parent that is on “path of least transmissions” – expected number of transmissions = 1 / P(success) • Distributed computation is estimate of SUM of retransmissions • A few retransmissions make large difference in success rate 11/14/2002 NEC Networking

Overall Results Success # Rate to BS Retrans / App packet Ave. Hop Count MRP 83. 94% 1. 03 8. 36 6. 12 Shortest Path 85. 79% 0. 82 7. 13 6. 18 5. 76 13. 13 4. 92 Broadcast 11/14/2002 Total Msg. Overhead/ pack. Recv. By BS 42. 96% NEC Networking

Fairness SP 75 MRP Broadcast 11/14/2002 NEC Networking

Stability • How often does the routing algorithm change? • # of parent changes per unit time MRP 11/14/2002 SP 75 NEC Networking

Stability (Broadcast) 11/14/2002 NEC Networking

Key Building Block: Link estimator • Nodes assess quality of link from packets they hear – directed to them or snoop – sequence number => infer losses between packets • For each new packet (or empty window) revise estimate of link probability • classic EWMA – P i+1 = a. P i + (1 -a) X i , where X i = 0, 1 if loss, success 11/14/2002 NEC Networking

Link Probability Changes Dynamically 11/14/2002 NEC Networking

Revisit Classic Estimator • Want estimator that is responsive to change, but stable with low error • Candidates – EWMA stable, agile, flip-flop – Moving Average –. . . • Best ended up being the own that we eyeballed – EWMA cascaded with average over a time window 11/14/2002 NEC Networking

Estimator driven from prob. generator 11/14/2002 NEC Networking

Empirical Trace 11/14/2002 NEC Networking

Bottom Line • Can only estimate link rate to within ~10% • Takes about 100 packets to settle • Design distributed algorithms with this kind of approximation of an inherently noisy world 11/14/2002 NEC Networking

So how about that demo? 11/14/2002 NEC Networking

Higher Level Network Services • Time Synchronization – many nodes with drift and offset variation – classic: » pairwise timesynch limited to half the variance in RTT » NTP hierarchy to establish rough global time – local timesynch » deep integration into network stack (Hill) » multiple receivers, not send-rcv (Ellison, Girod) – global propagation: huge open distributed algorithm question • Ranging & Localization • Naming • Data distribution, Aggregation, Storage • We’ve just begun to tackle the full problem 11/14/2002 NEC Networking

Conclusion • The networking part of the “future sensor networks” very different the power-hungry, sparse IP world – – severe constraints, noisy, dynamic, partial information Low power means you hardly ever listen self-organized and inherently distributed opportunity to work across layers of abstraction • Presents a rich set of open problems – Probabilistic connectivity model is usable and powerful – On-line distributed approximations to global optimization – a whole new internet to invent • Applications define valid assumption sets for network design – guide the front of progress in this huge space 11/14/2002 NEC Networking

Engineering Perspective • Low-hanging fruit – enough base stations that only need single hop • Mid-range fruit – stationary, well-powered multi-hop backbone • Research centroid – unstructured, ad hoc, multihop routing in energy starved environment 11/14/2002 NEC Networking