Jet Stream Achieving Predictable Gossip Dissemination by Leveraging

Jet. Stream: Achieving Predictable Gossip Dissemination by Leveraging Social Network Principles Jay A. Patel 1, Indranil Gupta 1, and Noshir Contractor 2 1 Dept. of Computer Science 2 Dept. of Speech Communication University of Illinois at Urbana-Champaign

“Flat” Gossip a h b i h c g d e f • Network of n nodes • A node desires to multicast a message m • Each “infected” node gossips to l other randomly selected nodes (i. e. , targets) • Message reaches all w. h. p if l = log(n) – [Kermarrec: TPDS: 03] 2

Random Overlay • Selecting l random targets out of n nodes – Membership protocols • • SCAMP [Ganesh: TOC: 03] SWIM [Das: DSN: 02] CYCLON [Voulgaris: JNSM: 05] Others 3

Non-uniform In-degree Distribution High Variance • Constant out-degree: Gaussian distribution for in-degree 4

Uneven Workload • In-degree distribution leads to uneven workload 5

Gossip Summary • Decentralized process + Resilient: no single point of failure + Balanced: everyone contributes + Fast: parallel transmission • Area of improvements – Uneven workload – Cost: total message overhead is n*l – Speed: may be improved? 6

Social Network Theories • Reciprocity – “Mutual Interest” – Reduce messages – Even workload Different from previous work - [Marti: IPTPS: 03] - [Bernstein: IPTPS: 03] • Structural Holes – “Complimentary Interest” – Improve speed 7

Utilitarian Model • Utility is a strictly “local” concept • Calculate utility based on current target set • xij is a boolean value – Represents a link from node i to node j • Reciprocity • Structural Holes • Net Utility Maximum utility: l * (l - 1) 2 Recall: l is the out-degree (or gossip fan out) 8

Jet. Stream Algorithm: “Global” Node a’s new target set: {e, f, i} Iterate through membership list: {b, c, e, g, h, j} • Node a’s target set: {d, f, i} a j b i h c g d e f Randomly selected de-link node: Node d Replacement candidate list: {e, g} Start with random overlay • Calculate node’s utility • De-link random node • Iterate through membership list – Replacement candidates improve or maintain utility • Once per time period – Gradual “evolution” Node a’s newlocalutility: 2 3 9

Jet. Stream Overlay “Evolution” • Overlay converges after certain time – Converges implies no more target set changes – Emergent behavior • Global reciprocity: No variance in in-degree • Structural holes satisfied 10

From Randomized to Deterministic • n=100, l=5 • Overlay converges – Each node achieves (close to) max utility – “Globally optimal” state through local, greedy decisions – No variance in indegree – Note: n*l must be even 11

Localized Implemenation • Global doesn’t scale in large networks – O(n*l) memory and O(n*l 2) computational overhead • Localized: limited knowledge – Candidate list (replacement candidates): s • • Superset of target set Complete information As few as s=2*l Timeout mechanism: Candidate list node removed after tout – Network node list: lazy discovery – Overheads -- computation: O(2*l 3), memory: O(2*l 2) 12

Localized Implementation • n=5000, l=10 • Overlay stabilizes rapidly – does not “converge” – close to convergence – 90+% nodes optimal (i. e. , max utility) • “Suboptimal” nodes also close to max utility 13

Jet. Stream: Gossip Workload • n=5000, l=10 • Fairer Workload Jet. Stream: Low Variance – Much smaller range for workload – Node with highest workload • Jet. Stream: 16 • Random: 35 • Chord: 55 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25+ 14

Jet. Stream Macro Efficiency • Jet. Stream is 25% faster, 40+% fewer total messages 15

Why “Jet. Stream”? Continuous Gossip • Background traffic to maintain target sets – Stable “noise” – Grows logarithmically • For n=5000, l=10 – Approx. 0. 4 packets per iteration – 24 bytes/sec (at 60 bytes/packet) • Continuous Ithresh amount of gossip – Ithresh = 4. 8 bytes/sec – Lower net traffic 16

Conclusion • Based on simple social network principles – Social network principles “uniformizes” gossip • “Fairer” Workload: net reduction by over 40% • Faster: over 25% speedier dissemination • Feasible for real systems – Local, greedy approach is sufficient – Churn adaptable, resilient, low overhead 17