Chord slides by Nick Feamster Chord Overview What
Chord (slides by Nick Feamster)
Chord: Overview • What is Chord? – A scalable, distributed “lookup service” – Lookup service: A service that maps keys to nodes – Key technology: Consistent hashing • Major benefits of Chord over other lookup services – Provable correctness – Provable “performance” 2
Chord: Primary Motivation Scalable location of data in a large distributed system Publisher Key=“Let. It. Be” Value=MP 3 file N 2 N 1 N 4 N 3 N 5 Key Problem: Lookup Client Lookup(“Let. It. Be”) 3
Chord: Design Goals • Load balance: Chord acts as a distributed hash function, spreading keys evenly over the nodes. • Decentralization: Chord is fully distributed: no node is more important than any other. • Scalability: The cost of a Chord lookup grows as the log of the number of nodes, so even very large systems are feasible. • Availability: Chord automatically adjusts internal tables to reflect newly joined nodes as well as node failures, ensuring that, the node responsible for a key can always be found. 4
Hashing • Hashing: assigns “values” to “buckets” – – In our case the bucket is the node, the value is the file key = f(value) mod N, where N is the number of buckets place value in the key-th bucket Achieves load balance if values are randomly distributed under f • Consistent Hashing: assign values such that addition or removal of buckets does not modify all values to buckets assignments • Chord challenges – How to perform hashing in a distributed fashion? – What happens when nodes join and leave? Consistent hashing addresses these problems 5
Consistent Hashing • Main idea: – map both keys and nodes to the same (metric) identifier space – find a “rule” how to assign keys to nodes Ring is one option. 6
Consistent Hashing • The consistent hash function assigns each node and key an m-bit identifier using SHA-1 as a base hash function • Node identifier: SHA-1 hash of IP address • Key identifier: SHA-1 hash of key 7
Chord Identifiers • m bit identifier space for both keys and nodes • Key identifier: SHA-1(key) Key=“Let. It. Be” SHA-1 ID=60 • Node identifier: SHA-1(IP address) IP=“ 198. 10. 1” SHA-1 ID=123 • How to map key IDs to node IDs? 8
Consistent Hashing in Chord Rule: A key is stored at its successor: node with next higher or equal ID 0 K 5 IP=“ 198. 10. 1” N 123 K 101 N 90 K 20 Circular 7 -bit ID space K 60 N 32 Key=“Let. It. Be” 9
Consistent Hashing Properties • Load balance: all nodes receive roughly the same number of keys • For N nodes and K keys, with high probability – each node holds at most (1+ )K/N keys 10
Lookups in Chord • Every node knows of every other node – requires global information • • Routing tables are large: O(N) Lookups are fast: O(1) 0 N 10 Where is “Let. It. Be”? Hash(“Let. It. Be”) = K 60 N 123 N 32 “N 90 has K 60” K 60 N 90 N 55 11
Lookups in Chord • Every node knows its successor in the ring • Requires O(N) lookups 0 N 123 N 10 Where is “Let. It. Be”? Hash(“Let. It. Be”) = K 60 N 32 “N 90 has K 60” K 60 N 90 N 55 12
Reducing Lookups: Finger Tables • Each node knows m other nodes in the ring (it has m fingers) • Increase distance exponentially • Finger i points to successor of n+2 i-1 i=1. . m N 120 N 112 80 + 25 N 16 80 + 26 N 96 80 + 24 80 + 23 80 + 22 80 + 21 80 + 20 N 80 13
N 32 finger table Faster Lookups • Lookups are O(log N) hops F 0 points to successor(32+20) = 60 F 1 points to successor(32+21) = 60 F 2 points to successor(32+22) = 60 F 3 points to successor(32+23) = 60 F 4 points to successor(32+24) = 60 F 5 points to successor(32+25) = 80 F 6 points to successor(32+26) = 99 N 5 N 14 N 110 N 20 K 19 N 99 N 32 Lookup(K 19) N 80 N 60 Look for a node identifier in the finger table that is less then the key identifier and closest in the ID space to the key identifier 14
Summary of Performance Results • Efficient: O(log N) messages per lookup • Scalable: O(log N) state per node • Robust: survives massive membership changes 15
Joining the Ring • Three step process – Initialize all fingers of new node – Update fingers of existing nodes – Transfer keys from successor to new node • Two invariants to maintain – Each node’s successor is correctly maintained – successor(k) is responsible for k 16
Join: Initialize New Node’s Finger Table • Locate any node p in the ring • Ask node p to lookup fingers of new node N 5 N 20 N 36 N 99 1. Lookup(37, 38, 40, …, 100, 164) N 40 N 80 N 60 17
Join: Update Fingers of Existing Nodes • New node calls update function on existing nodes n becomes the ith fingerprint of node p if p precedes n by at least 2 i-1 and ith finger of node p succeeds n. N 5 N 20 N 99 Update in O(log 2 N) N 36 N 40 N 80 N 60 18
Join: Transfer Keys • Only keys in the range are transferred N 5 N 20 N 99 N 36 K 30 N 40 K 38 K 30 N 80 Copy keys 21. . 36 from N 40 to N 36 K 38 N 60 19
Handling Failures • Problem: Failures could cause incorrect lookup • Solution: Fallback: keep track of a list of immediate successors N 120 N 102 N 85 Lookup(85) N 80 20
Handling Failures • Use successor list – Each node knows r immediate successors – After failure, will know first live successor – Correct successors guarantee correct lookups • Guarantee with some probability – Can choose r to make probability of lookup failure arbitrarily small 21
Joining/Leaving overhead • When a node joins (or leaves) the network, only a fraction of the keys are moved to a different location. • For N nodes and K keys, with high probability – when node N+1 joins or leaves, O(K/N) keys change hands, and only to/from node N+1 22
Structured vs. Unstructured Overlays • Structured overlays have provable properties – Guarantees on storage, lookup, performance • Maintaining structure under churn has proven to be difficult – Lots of state that needs to be maintained when conditions change • Deployed overlays are typically unstructured 23
- Slides: 23