A Common API for Structured Peerto Peer Overlays

A Common API for Structured Peer-to. Peer Overlays Frank Dabek, Ben Y. Zhao, Peter Druschel, Ion Stoica

Structured Peer-to-Peer Overlay n They are: ¨ Scalable, self-organizing overlay networks ¨ Provide routing to location-independent names ¨ Examples: CAN, Chord, Pastry, Tapestry, … n Basic operation: ¨ Large sparse namespace N (integers: 0– 2128 or 0– 2160) ¨ Nodes in overlay network have node. Ids N ¨ Given k N, a deterministic function maps k to its root node (a live node in the network) ¨ route(msg, k) delivers msg to root(k) Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Current Progress n Lots of applications built on top File systems, archival backup ¨ Application level multicast ¨ Routing for anonymity, attack resilience ¨ n But do we really understand them? What is the core functionality that applications leverage from them? ¨ What are the strengths and weaknesses of each protocol? How can they be exploited by applications? ¨ How can we build new protocols customized to our future needs? ¨ Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Our Goals n Protocol comparison ¨ Compare and contrast protocol semantics ¨ Identify basic commonalities ¨ Isolate and understand differences n Towards a common API ¨ Easily supportable by old and new protocols ¨ Enables application portability between protocols ¨ Enables common benchmarks ¨ Provides a framework for reusable components Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Talk Outline n Motivation n DHTs and DOLRs n A Flexible Routing API n Usage Examples Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Decomposing Functional Layers n Distributed Hash Tables (DHT) put(key, data), value = get(key) ¨ Hashtable layered across network ¨ Handles replication; distributes replicas randomly ¨ Routes queries towards replicas by name ¨ n Decentralized Object Location and Routing (DOLR) publish(object. Id), route(msg, node. Id), route. Obj(msg, object. Id, n) ¨ Application controls replication and placement ¨ Cache location pointers to replicas; queries quickly intersect pointers and redirect to nearby replica(s) ¨ Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

DHT Illustrated Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

DOLR Illustrated Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Architecture CFS PAST CAN, Chord+DHash DHT Split. Stream Multicast i 3 Ocean. Store DOLR Bayeux Tier 2 Tapestry Pastry+Scribe Tier 1 Replication Routing Mesh Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu Tier 0 January 14, 2003

Talk Outline n Motivation n DHTs and DOLRs n A Flexible Routing API n Usage Examples Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Flexible API for Routing n Goal ¨ Consistent API for leveraging routing mesh ¨ Flexible enough to build higher abstractions n n n Openness promotes new abstractions Allow competitive selection to determine right abstractions Three main components ¨ Invoking routing functionality ¨ Accessing namespace mapping properties ¨ Open, flexible upcall interface Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

API (routing) Data types n Key, node. Id = 160 bit integer n Node = Address (IP + port #), node. Id n Msg: application-specific msg of arbitrary size Invoking routing functionality n Route(key, msg, [node]) route message to node currently responsible for key ¨ Non-blocking, best effort – message may be lost or duplicated. ¨ node: transport address of the node last associated with key (proposed first hop, optional) ¨ Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

API (namespace properties) n next. Hop. Set = local_lookup(key, num, safe) Returns a set of at most num nodes from the local routing table that are possible next hops towards the key. ¨ Safe: whether choice of nodes is randomly chosen ¨ n nodehandle[ ] = neighbor. Set(max_rank) ¨ ¨ n Returns unordered set of nodes as neighbors of the current node. Neighbor of rank i is responsible for keys on this node should all neighbors of rank < i fail nodehandle[ ] = replica. Set(key, num) Returns ordered set of up to num nodes on which replicas of the object with key can be stored. ¨ Result is subset of neighbor. Set plus local node ¨ n boolean = range(node, rank, lkey, rkey) ¨ Returns whether current node would be responsible for the range specified by lkey and rkey, should the previous rank-1 nodes fail. Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

API (upcalls) Application forward deliver msg n Routing Layer msg Delivers an incoming message to the application. One application per node. Demultiplexing done by including demux key in msg. Forward(&key, &msg, &next. Hop. Node) ¨ ¨ ¨ n msg Deliver(key, msg) ¨ n Routing Layer Synchronous upcall invoked at each node along route On return, will forward msg to next. Hop. Node App may modify key, msg, next. Hop. Node, or terminate by setting next. Hop. Node to NULL. Update(node, boolean joined) ¨ Upcall invoked to inform app of a change in the local node’s neighbor. Set, either a new node joining or an old node leaving. Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Talk Outline n Motivation n DHTs and DOLRs n A Flexible Routing API n Usage Examples Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

DHT Implementation n Interface ¨ put (key, value) ¨ value = get (key) n Implementation (source S, root R) ¨ Put: route(key, [PUT, value, S], NULL) Reply: route(NULL, [PUT-ACK, key], S) ¨ Get: route(key, [GET, S], NULL) Reply: route(NULL, [value, R], S) Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

DOLR Implementation n Interface Route. Node(msg, node. Id) ¨ Publish(object. Id) ¨ Route. Obj(msg, object. Id, n) ¨ n Implementation (server S, client C, object O) Route. Node: route(node. Id, msg, NULL) ¨ Publish: route(object. Id, [“publish”, O, S], NULL) Upcall: add. Local([O, S]) ¨ Route. Obj: route(node. Id, [n, msg], NULL) Upcall: server. Set[] = get. Local(O); if (|server. Set|<n), route(node. Id, [n-|server. Set|, msg], NULL) for first n entries in server. Set, route(server. Set[i], msg, NULL) ¨ Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Conclusion n Very much ongoing work ¨ Feedback n valuable and appreciated Ongoing Work ¨ Implementations will support routing API ¨ Working towards higher level abstractions Distributed Hash Table API DOLR publish/route API n For more information, see IPTPS 2003 n Thank you… Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Backup Slides Follow… Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Storage API: Overview linsert(key, value); n value = lget(key); n Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Storage API n linsert(key, value): store the tuple <key, value> into local storage. If a tuple with key already exists, it is replaced. The insertion is atomic wrt to failures of the local node. n value = lget(key): retrieves the value associated with key from local storage. Returns null if no tuple with key exists. Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

To Do Following slides contain functions that we haven’t decided on yet… Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Basic DHT API: Overview insert(key, value, lease); n value = get(key); n release(key); n Upcalls: n insert. Data(key, value, lease); Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Basic DHT API n insert(key, value, lease): inserts the tuple <key, value> into the DHT. The tuple is guaranteed to be stored in the DHT only for “lease” time. “value” also includes the type of operations to be performed on insertion. Default operation types include ¨ REPLACE: replace value associated with the same key ¨ APPEND: append value to the existing key ¨ UPCALL: generate an upcall to application before inserting ¨… Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Basic DHT API n value = get(key): retrieves the value associated with key. Returns null if no tuple with key exists in the DHT. Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Basic DHT API n Release(key): releases any tuples with key from the DHT. After this operations completes, tuples with key are no longer guaranteed to exist in the DHT. Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Basic DHT API: Open questions n Semantics? ¨ Verification/Access control/multiple DHTs? ¨ Caching? ¨ Replication? n Should we have leases? It makes us dependent on secure time sync. Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Replicating DHT API n Insert(key, value, num. Replicas); adds a num. Replicas argument to insert. Ensures resilience of the tuple to up to num. Replicas-1 “simultaneous” node failures. Open questions: n consistency Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Caching DHT API Same as basic DHT API. Implementation uses dynamic caching to balance query load. Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Resilient DHT API Same as replicating DHT API. Implementation uses dynamic caching to balance query load. Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Publish API: Overview Publish(key, object); n object = Lookup(key); n Remove(key, object): n Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Publish API n Publish(key, object): ensures that the locally stored object can be located using the key. Multiple instances of the object may be published under the same key from different locations. n object = Lookup(key): locates the nearest instance of the object associated with key. Returns null if no such object exists. Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003

Publish API n Remove(key, object): after this operation completes, the local instance of object can no longer be located using key. Ocean. Store / Sahara Retreat ravenben@eecs. berkeley. edu January 14, 2003