Hiearchial Caching in Traffic Server Hiearchial Caching A

  • Slides: 18
Download presentation
Hiearchial Caching in Traffic Server

Hiearchial Caching in Traffic Server

Hiearchial Caching A set of techniques and mechanisms to increase the size and performance

Hiearchial Caching A set of techniques and mechanisms to increase the size and performance of network caches. Fundamentally having a set of independent caches act as single aggregate cache from the user agent point of view.

History and Motivation

History and Motivation

In the beginning User agent talks directly to application server.

In the beginning User agent talks directly to application server.

Scaling up Not so eventually central servers cannot handle the load due to computation

Scaling up Not so eventually central servers cannot handle the load due to computation or bandwith limits. Offload some of this work to caches between the user agent and the origin server.

It never ends Usage and requirements continue to scale up and a single layer

It never ends Usage and requirements continue to scale up and a single layer of caches is frequently insufficient. Naturally engineers say “add another layer!” But how do we do that?

Basic Techniques

Basic Techniques

Sharding Split incoming connections among a set of Traffic Server instances. Load balancer (connection

Sharding Split incoming connections among a set of Traffic Server instances. Load balancer (connection aware) or router (packet based). Each cache receives a portion of the requests and acts independently.

Clustering Share cache among a set of peers Peers communicate so that requests can

Clustering Share cache among a set of peers Peers communicate so that requests can be routed from one peer to another. Note: cluster is not well supported.

Parent Proxying Traffic Server instance gets content from another Traffic Server, the parent. This

Parent Proxying Traffic Server instance gets content from another Traffic Server, the parent. This create a cache hierarchy.

Consistent Hashing Selection of parent proxies should be the same across instances Loss of

Consistent Hashing Selection of parent proxies should be the same across instances Loss of parent should spread load across remaining ones

Tiered Parents User agents hit ingress ATS (which is usually a sharded pod). Ingress

Tiered Parents User agents hit ingress ATS (which is usually a sharded pod). Ingress pulls content from larger cache pod which is split for redundancy. Consistent hashing is used to keep the ingress instances running due to parent failures. User agents reroute to different ingress on ingress fail.

Hiearchial Caching at Yahoo!

Hiearchial Caching at Yahoo!

Functional Structure User first connects to a proxy layer which does any request modification

Functional Structure User first connects to a proxy layer which does any request modification and layer 7 routing. Proxy layer fetches content from cache layer. Cache layer serves cached content if possible and if not fetches it from the data center.

Basic Physical Layout

Basic Physical Layout

Expanded View

Expanded View

Details Edge pod machines are identical across the pod. Ingress and cache peer are

Details Edge pod machines are identical across the pod. Ingress and cache peer are relative to a request, not the machine Consistent hashing is used to assign a specific cache peer to a cached object. If ingress and cache peer are the same machine it handles the request. On a cache miss it goes directly to next layer (central cache or data center). If the ingress and cache peer are different Ingress checks its own cache for a promoted (locally cached) object. On a cache miss ingress goes to cache peer for the object. Cache peer, on a cache miss, goes to next layer.

Ingress vs. Peer Functional role of edge machine determined by request. Each machine has

Ingress vs. Peer Functional role of edge machine determined by request. Each machine has a dedicated proxy port for cache only requests. Ingress, if not the cache peer, forwards the request to the cache proxy port. All plugin processing is done by ingress. Cache peer disables plugins for the transaction. That is, for transactions arriving on the cache proxy port. Cache proxy port should NOT be accessible outside the pod Or even the local network segment.