Open DHT A Public DHT Service Sean C

Open. DHT: A Public DHT Service Sean C. Rhea UC Berkeley June 2, 2005 Joint work with: Brighten Godfrey, Brad Karp, John Kubiatowicz, Sylvia Ratnasamy, Scott Shenker, Ion Stoica, and Harlan Yu

Peer-to-Peer File Sharing • Very simple insight – Most computers unused most of the time • Idea: harness this spare capacity to – Quickly download music files [Napster, Gnutella] – Search for aliens [SETI@Home] – Make free long-distance phone calls [Skype] • Question: how to find desired resource(s)? – Early approaches: scoped flooding – Downsides: scalability, accuracy Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

A Better Search Facility: The Distributed Hash Table (DHT) • Same interface as a programmatic hash table, – put(key, value) stores value under key – get(key) returns the value(s) stored under key • But shared across many machines • Implemented via an overlay network Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

A Better Search Facility: The Distributed Hash Table (DHT) stores k 1, v 1 K V K V k 1, v 1 k 1 K V k 1, v 1 K V v 1 k 1, v 1 K V Sean C. Rhea K V v 1 K V put(k 1, v 1) k 1 K V get(k 1) Open. DHT: A Public DHT Service June 2, 2005

DHTs and File Sharing: DHT Stores Pointers to Files pointer to file K V K V put(file, IP) K V K V Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

DHTs and File Sharing: DHT Stores Pointers to Files pointer to file K V K V K V IP K V get(file) xfer over HTTP K V K V Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

DHTs and Spam Detection: Detecting Similar Messages K V put(hash(msg), IP) K V K V K V “I love you!” Sean C. Rhea K V Open. DHT: A Public DHT Service June 2, 2005

DHTs and Spam Detection: Detecting Similar Messages K V put(hash(msg), IP) K V K V K V “I love you!” Sean C. Rhea K V “I love you!” Open. DHT: A Public DHT Service June 2, 2005

DHTs and Spam Detection: Detecting Similar Messages Something’s fishy! K V K V “I love you!” Sean C. Rhea K V “I love you!” Open. DHT: A Public DHT Service June 2, 2005

DHTs and Spam Detection: Detecting Similar Messages K V put(hash(msg), IP) K V K V K V “I love you!” Sean C. Rhea K V “I love you!” Open. DHT: A Public DHT Service K V “I love you!” June 2, 2005

DHTs and Spam Detection: Detecting Similar Messages K V Something’s fishy! K V K V K V “I love you!” Sean C. Rhea K V “I love you!” Open. DHT: A Public DHT Service K V “I love you!” June 2, 2005

More DHT Applications • Distributed Storage Systems – CFS, Hive. Cache, PAST, Pastiche – Ocean. Store / Pond • Content Distribution Networks / Web Caches – Bslash, Coral, Squirrel • Indexing / Naming Systems – Chord-DNS, Co. Do. NS, DOA, SFR • Internet Query Processors – Catalogs, PIER • Communication Systems – Bayeux, i 3, MCAN, Split. Stream Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Some Areas of DHT Research • Better routing protocols – One-hop, degree-optimal • Load balancing – Non-uniform key distributions • Security – Byzantine fault-tolerant routing • Data redundancy and fault tolerance – Replication, erasure-coding • Stronger semantics – Supporting read-modify-write Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

How Many DHTs Will There Be? File Sharing K V Spam Detection K V K V Can’t Share Files Company Machine: K V Owns Stock in Spam Company K V K V Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

How Many DHTs Will There Be? File Sharing K V Spam Detection K V K V Redundant Link K V K V Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

How Many DHTs Will There Be? File Sharing K V Spam Detection K V K V K V Unshared Links K V K V Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Benefits of Sharing a DHT • Amortizes costs across applications – Maintenance bandwidth, connection state, etc. • Facilitates “bootstrapping” of new applications – Working infrastructure already in place • Allows for statistical multiplexing of resources – Takes advantage of spare storage and bandwidth • Facilitates upgrading existing applications – “Share” DHT between application versions Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Challenges in Sharing a DHT • Robustness – Must be available 24/7 • Shared Interface Design – Should be general, yet easy to use • Resource Allocation – Must protect against malicious/over-eager users • Economics – What incentives are there to provide resources? Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Challenges in Sharing a DHT • Robustness – Must be available 24/7 • Shared Interface Design – Should be general, yet easy to use • Resource Allocation – Must protect against malicious/over-eager users • Economics – What incentives are there to provide resources? Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

The DHT as a Service K V K V Sean C. Rhea K V Open. DHT: A Public DHT Service K V June 2, 2005

The DHT as a Service K V K V K V Open. DHT K V Sean C. Rhea K V Open. DHT: A Public DHT Service K V June 2, 2005

The DHT as a Service Open. DHT Clients Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

The DHT as a Service Open. DHT Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

The DHT as a Service What is this interface? Open. DHT Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

The Traditional Interface: lookup 11 12 1 10 2 9 3 8 4 7 Sean C. Rhea 6 5 Open. DHT: A Public DHT Service June 2, 2005

The Traditional Interface: lookup(k) On reaching the successor of k, message passed to an “upcall” Sean C. Rhea k Open. DHT: A Public DHT Service June 2, 2005

DHTs and Spam Detection: Upcall: I’ve seen this Detecting Similar Messages message before! K V put(hash(msg), IP) K V K V K V “I love you!” Sean C. Rhea K V “I love you!” Open. DHT: A Public DHT Service June 2, 2005

DHTs and Spam Detection: Detecting Similar Messages Something’s fishy! K V K V “I love you!” Sean C. Rhea K V “I love you!” Open. DHT: A Public DHT Service June 2, 2005

Upcall Challenges • Distribution – How do we get new upcall code to all nodes? Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Upcall Challenges lookup(k) How did the upcall code get here? Sean C. Rhea k Open. DHT: A Public DHT Service June 2, 2005

Upcall Challenges • Distribution – How do we get new upcall code to all nodes? – Active networking experience is a warning… Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Upcall Challenges • Distribution – How do we get new upcall code to all nodes? – Active networking experience is a warning… • Security – How do we safely run untrusted clients’ upcalls? Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

What about Put/Get? • Works great for some applications – File sharing, for example Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

DHTs and File Sharing: DHT Stores Pointers to Files pointer to file K V K V put(file, IP) K V IP K V get(file) xfer over HTTP K V K V Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

What about Put/Get? • Works great for some applications – File sharing, for example • What about applications with upcalls? – Our spam detection application, for example Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

What about Put/Get? • Works great for some applications – File sharing, for example • What about applications with upcalls? – Our spam detection application, for example • Idea: let application nodes run the upcalls – Each node only runs upcalls for the applications that it’s participating in Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Upcall Example File Sharing Spam Detection I can handle spam detection messages put/get I can handle spam detection messages Open. DHT I can handle spam detection messages put/get Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Upcall Example File Sharing Spam Detection put/get Who’s handling hash(message)? Open. DHT put/get “I love you!” Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Upcall Example File Sharing Spam Detection put/get Open. DHT Who’s handling hash(message)? put/get “I love you!” Sean C. Rhea “I love you!” Open. DHT: A Public DHT Service June 2, 2005

Upcall Example File Sharing Something’s fishy! Spam Detection put/get DHT keeps track of which nodes support. Open. DHT which upcalls via Recursive Distributed Rendezvous (Re. Di. R) put/get “I love you!” Sean C. Rhea “I love you!” Open. DHT: A Public DHT Service June 2, 2005

Re. Di. R • Goal: Implement two functions using put/get: – join(namespace, node) – node = lookup(namespace, identifier) H(A) H(B) H(namespace) A L 0 A L 1 A L 2 Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Re. Di. R • Goal: Implement two functions using put/get: – join(namespace, node) – node = lookup(namespace, identifier) H(A) H(B) H(C) A L 0 A L 1 A, B C L 2 Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Re. Di. R • Goal: Implement two functions using put/get: – join(namespace, node) – node = lookup(namespace, identifier) H(A) H(B) H(C) H(D) A L 0 A, C D L 1 A, B C D L 2 Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Re. Di. R • Goal: Implement two functions using put/get: – join(namespace, node) – node = lookup(namespace, identifier) H(A) H(B) H(C) H(D) H(E) A, D L 0 A, C D L 1 A, B C D E L 2 Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Re. Di. R • Goal: Implement two functions using put/get: – join(namespace, node) – node = lookup(namespace, identifier) H(A) H(B) H(C) H(D) H(E) A, D L 0 A, C D, E L 1 A, B C D E L 2 Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Re. Di. R • Join cost: – Worst case: O(log n) puts and gets – Average case: O(1) puts and gets H(A) H(B) H(C) H(D) H(E) A, D L 0 A, C D, E L 1 A, B C D E L 2 Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Re. Di. R • Goal: Implement two functions using put/get: – join(namespace, node) – node = lookup(namespace, identifier) H(k 1) A, D L 0 A, C D, E L 1 successor A, B C L 2 H(A) H(B) H(C) Sean C. Rhea D E H(D) H(E) Open. DHT: A Public DHT Service June 2, 2005

Re. Di. R • Goal: Implement two functions using put/get: – join(namespace, node) – node = lookup(namespace, identifier) H(k 2) A, D L 0 L 1 successor A, C no successor A, B C L 2 H(A) H(B) H(C) Sean C. Rhea D, E D E H(D) H(E) Open. DHT: A Public DHT Service June 2, 2005

Re. Di. R • Goal: Implement two functions using put/get: – join(namespace, node) – node = lookup(namespace, identifier) successor A, D no successor D, E H(k 3) L 0 A, C L 1 A, B C no successor D E L 2 H(A) H(B) Sean C. Rhea H(C) H(D) H(E) Open. DHT: A Public DHT Service June 2, 2005

Re. Di. R • Lookup cost: – Worst case: O(log n) gets – Average case: O(1) gets A, D L 0 A, C D, E L 1 A, B C D E L 2 H(A) H(B) Sean C. Rhea H(C) H(D) H(E) Open. DHT: A Public DHT Service June 2, 2005

Re. Di. R Performance (On Planet. Lab) Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Open. DHT Design Summary • Open. DHT is a common infrastructure for – Storage of values, pointers, etc. – Organizing clients that handle application upcalls • Benefits: – Amortizes maintenance costs across applications – Facilitates “bootstrapping” of new applications – Allows for statistical multiplexing of resources Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Impact Application Uses Open. DHT for Interface Croquet Media Manager replica location put/get DOA HIP indexing name resolution put/get host mobility range queries mcast tree constr. indexing storage rendezvous storage redirection put/get Re. Di. R put/get Re. Di. R Tetherless Computing Place Lab QStream VPN Index Free. DB Instant Messaging CFS i 3 Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Future Work • Open. DHT makes a great common substrate for: – Soft-state storage – Naming and rendezvous • Many P 2 P applications also need to: – Traverse NATs – Redirect packets within the infrastructure (as in i 3) – Refresh puts while intermittently connected • All of these can be implemented with upcalls – Who provides the machines that run the upcalls? Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

Future Work • We don’t want to add upcalls to the core DHT – Keep the main service simple, fast, and robust • Can we build a separate upcall service? – Some other set of machines organized with Re. Di. R – Security: can only accept incoming connections, can’t write to local storage, etc. • This should be enough to implement – NAT traversal, reput service – Some (most? ) packet redirection • What about more expressive security policies? Sean C. Rhea Open. DHT: A Public DHT Service June 2, 2005

For more information, see http: //opendht. org/