SmallScale PeertoPeer PublishSubscribe Vinod Muthusamy HansArno Jacobsen University

Small-Scale Peer-to-Peer Publish/Subscribe Vinod Muthusamy, Hans-Arno Jacobsen University of Toronto July 17, 2005 P 2 PKM 2005 San Diego, CA

Motivation n Many distributed hash table (DHT)-based peer-to-peer applications n n n Typically designed for and evaluated on a large-scale Small-scale performance is important n n n Storage (Ocean. Store, Past) Database (PIER) Naming service (Chord-DNS) Publish/subscribe (Scribe, P 2 P-To. PSS, Meghdoot) Even large systems start small Benefits even on small scale Small-scale case study of a DHT-based pub/sub protocol July 17, 2005 (P 2 PKM ’ 05) Small-Scale P 2 P Publish/Subscribe 2

Agenda n Background n n n Publish/subscribe model Distributed hash tables Small-scale benefits Publish/subscribe on DHT Evaluation Conclusions July 17, 2005 (P 2 PKM ’ 05) Small-Scale P 2 P Publish/Subscribe 3

Publish/subscribe model TSX Stock markets NASDAQ NYSE Publisher AMG N=5 8 12 ORCL= HON=24 MSFT =27 Publisher =84 IBM JNJ= 58 19 = C T Publications IN Broker Network Notification Subscriptions Subscriber n n n Simple interface Decoupled interaction Network efficiency Subscriber Subscriptions: IBM > 85 ORCL < 10 JNJ > 60 July 17, 2005 (P 2 PKM ’ 05) Small-Scale P 2 P Publish/Subscribe 4

Distributed hash tables n n Example: Pastry Distributed version of a hash table data structure n n Store (key, value) pairs Each node receives at most K/N keys Each node has about O(log. N) routing state Lookups resolved with O(log. N) hops July 17, 2005 (P 2 PKM ’ 05) Small-Scale P 2 P Publish/Subscribe 5

Small-scale benefits Distributed pub/sub Large-scale benefits (traditional focus) Small-scale benefits (untapped) (also largescale benefits) July 17, 2005 (P 2 PKM ’ 05) • • Multicast Distributed matching DHT • • • Decoupled interaction Declarative bindings Incremental service deployment Late binding • • Organic (infrastructureless) scaling Self-organizing Effective use of commodity components (fault tolerance, load balancing) Incremental scalability Small-Scale P 2 P Publish/Subscribe 6

Service oriented architecture (SOA) n An illustration of small-scale benefits n SOA: a distributed architecture n n Loosely coupled reusable components Enterprise service bus (ESB) comm. fabric n n n Can be realized as a pub/sub broker network Pub/sub support in IBM Websphere, Sonic ESBs Example n n Three-tier Web server Communication between components through pubs and subs n n n Application Server Enterprise Service Bus Decoupled interaction Declarative bindings Simple to add a Monitor component n n Incremental service deployment Late binding July 17, 2005 (P 2 PKM ’ 05) User Database Small-Scale P 2 P Publish/Subscribe Products Database Monitor 7

Pub/sub case study n P 2 P-To. PSS n n n A pub/sub system built over DHT Large-scale performance previously studied [under submission] Evaluate small-scale benefits (of the same algorithm) now n Subscription load balance July 17, 2005 (P 2 PKM ’ 05) Small-Scale P 2 P Publish/Subscribe 8

Distributed multidimensional matching (DMM) 1) Pub/sub domain: Subscription price < 50 weight > 50 3) Network domain: Publication price = 10 weight = 70 0 2) Spatial domain: weight 100 01 11 0 1 00 10 00 0 0 price July 17, 2005 (P 2 PKM ’ 05) 100 1 01 10 11 DHT network used to map from tree nodes to peers Small-Scale P 2 P Publish/Subscribe 9

Distributing the tree n n Let region r have z-code z, and tree node n(r) Store n(r) at peer p(r), where p(r) is where DHT would store key z Each peer performs mapping locally Hash function ensures even distribution Subscription price = … weight = … name/code p_w_1101 July 17, 2005 (P 2 PKM ’ 05) Network Region weight price hash z-code 1101 peer-id AF 3483 BQ 1 Small-Scale P 2 P Publish/Subscribe p/w 10

Load balancing n Peers can be overloaded due to n n n Excessive subscriptions storage Excessive publication traffic Overloaded peer delegates subscriptions to children in the DMM tree n Network 0 00 01 p/w A local decision July 17, 2005 (P 2 PKM ’ 05) Small-Scale P 2 P Publish/Subscribe 11

Evaluation n Sim. Pastry simulator (Microsoft Research) n n n Nodes connected in a LAN n n Pastry DHT Simulate network latencies, message costs 1 ms latency between every pair of nodes Subscriptions n Attribute name + range n n E. g. : “ 20 < weight < 40” Metric n Storage load of subscriptions July 17, 2005 (P 2 PKM ’ 05) Small-Scale P 2 P Publish/Subscribe 12

DHT performance n n 1000 random keys DHT successfully balances load July 17, 2005 (P 2 PKM ’ 05) Small-Scale P 2 P Publish/Subscribe 13

n n weight Pub/sub performance price 1000 fine-grained subscriptions Load balancing distributes storage load July 17, 2005 (P 2 PKM ’ 05) Small-Scale P 2 P Publish/Subscribe 14

n n weight Pub/sub performance price 1000 coarse-grained subscriptions Load balancing distributes but also increases storage load July 17, 2005 (P 2 PKM ’ 05) Small-Scale P 2 P Publish/Subscribe 15

Observation n Load balancing technique assumes available resources somewhere in the network n n Leads to runaway subscription delegation Common assumption in DHT applications n n CAN DHT’s active caching and replication Meghdoot’s hotspot circumvention In DHT, no knowledge of aggregate available resources Fundamental assumption of DHT applications not valid in small-scale networks July 17, 2005 (P 2 PKM ’ 05) Small-Scale P 2 P Publish/Subscribe 16

Conclusions n Pub/sub and DHTs n n n Common assumption on DHT applications n n n Proven benefits in large-scale network Untapped benefits in small-scale networks Sufficient available resources in the network Small-scale networks break this assumption Future work n n n Implement algorithms to estimate global resource availability Curb runaway subscription delegation with this information Evaluate other DHT applications in small-scale networks July 17, 2005 (P 2 PKM ’ 05) Small-Scale P 2 P Publish/Subscribe 17

Q&A Small-Scale Peer-to-Peer Publish/Subscribe www. msrg. toronto. edu July 17, 2005 (P 2 PKM ’ 05) Small-Scale P 2 P Publish/Subscribe 18