Consistent Hashing Load Balancing in a Changing World

Consistent Hashing: Load Balancing in a Changing World David Karger, Eric Lehman, Tom Leighton, Matt Levine, Daniel Lewin, Rina Panigrahy MIT

Caches can Load Balance Server l l Items distributed among caches l l Users get items from caches MIT l Numerous items in central server. Requests can swamp server. Distribute items among caches. Clients get items from caches. Server gets only 1 request per item

Who Caches What? l Each cache should hold few items » else cache gets swamped by clients l Each item should be in few caches » else server gets swamped by caches » and cache invalidations/updates expensive l Browser must know right cache » fast, local computation MIT

A Solution: Hashing Server l items assigned to caches by hash function. l Users use hash to compute cache for item. MIT l Example: y = ax+b (mod n) Intuition: Assigns items to “random” caches » few items per cache Easy to compute which cache holds an item

Problem: Adding Caches Suppose a new cache arrives. l How work it into hash function? l Natural change: l y=ax+b (mod n+1) l Problem: changes bucket for every item » every cache will be flushed » servers get swamped with new requests Goal: when add bucket, few items move MIT

Problem: Inconsistent Views Each client knows about a different set of caches: its view l View affects choice of cache for item l » Same item may hash to many places: caches swamp server with request for item » Many items may hash to same place: clients swamp cache l MIT Goal: despite views, items evenly distributed into a few caches each

Solution: Consistent Hashing l l Use standard hash function to map caches and items to points in unit interval. » “random” points spread uniformly Item assigned to nearest cache in view item Cache (Bucket) Computation easy as standard hash function MIT

Properties All buckets get roughly same number of items (like standard hashing). l When kth bucket is added only a 1/k fraction of items move. l » and only from a few caches When a cache is added, minimal reshuffling of cached items is required. MIT

Multiple View Properties l Despite multiple views, each cache gets few items » no cache overloaded l Despite multiple views, each item only in few caches. » server protected, cache updates easy System tolerates multiple, inconsistent views of caches (also fault tolerant). MIT

Load Balancing l Task: distribute items into buckets » Data to memory locations » Files to disks » Tasks to processors » Web pages to caches (our motivation) l MIT Goal: even distribution

Problem: No Synchronization l l l Each user knows about a different set of caches: a view View affects assignment of items to caches Problems when there are multiple views: l items View 1 View 3 View 2 View 4 l l The items assigned to a specific cache are different in each view. These sets could be essentially disjoint for standard hash functions. Over all views, cache is responsible for too many items assigned to one cache over 4 views MIT Cache not large enough to contain active set of items

Multiple Views: Cont. . l item l l View 3 View 1 View 4 View 2 item assigned to different caches in each of 4 views MIT Item may be assigned to different caches in different views. Standard hash function may assign item to a different cache in every view. Result: item requested from many caches » Server swamped with requests for copies of the item. » Hard to update cached copies

Problem: Adding Caches l l New cache means new hash function » natural change: y = ax+b (mod n+1) Standard hash functions completely redistribute items when the range of function changes: » Every cache will be flushed » server is swamped with requests since items are reshuffled between caches. » Need to broadcast the new hash function to all users at the same time – some kind of global synchronization? . . . MIT