Next Hop Progress Layer 7 Routing ATS Summit

Next. Hop Progress Layer 7 Routing ATS Summit Fall 2018 Aaron Canary - Oath John Rushford - Comcast Vijay Mamidi - Apple

Design Legacy Extensible Host Resolution Alan Carroll Apache. Con 2014

Why Caching Proxy Fast Extensible Proven CDN (Stable, Scalable, Supported)

Why Caching Proxy Efficient: ● Runtime ● Deployment ● Development

How Layer 7 Framework Goal: Robustly provide an Upstream Connection

What jjrushford vmamidi Config Routing Architecture Parent. Selection Host. Status Up. Stream Generator Modular Routing Logic Shared Data Storage DBTable Extendible Acid. Ptr acanary solidwallofcode

Shared Data Storage Currently host state is stored by system Host. DB Http. Connection. Count Host. Status CARP/Host Parent. Host Health. Check. Plugin Each new system requires ● new storage container ● reimplement thread safety ● indexing and hashing ● performance optimization

Shared Data Storage Systems extend existing data Host. DB Http. Connection. Count Host. Status CARP/Host Parent. Host Health. Check. Plugin store Host. Ext. Data Addr. Ext. Data 8

Shared Data Storage - Properties High Performance Volatile ● Many read operations ● Few write operations ● Prefer fast operations ● Minimize blocking ● In memory only ● Transient data

Shared Data Storage - Properties Not A. C. I. D. Compliant DB ACID DB Fast, Simple Atomic Consistent Inconsistent - Snapshot Isolated Sequential Write Durable Volatile

Shared Data Storage - Expectations Fast Reduce Allocations Precache Efficiency Reduce Mutexes Extensible Supports common use Scales for custom use Proven Code Reuse & Sharing Thread Safe Stable Data 11

Shared Data Storage DBTable Design ● Atomic Items ● Contiguous Objects ● Fields are RT-static ● Concurrent ● Use scoped blocking ● Objects are stable Extendible Field idx Object idx Item

Shared Data Storage Type bool int char[], struct Atomic Item Bit. Ref std: : atomic<T> Snapshot. Ptr

Shared Data Storage Bit. Ref - a reference to a bit → xxxxx 0 xx Bit. Ref(uint 8* store, uint 8 mask) bool operator=(bool) bool operator==(bool) const explicit operator const bool() const bool compare_exchange(bool) bool test_and_set()

Shared Data Storage Snapshot. Ptr<T> shared_ptr<const T> get. Ptr() void commit(T*) start. Commit() Commit. Ptr<T> : unique_ptr<T> void abort()

Shared Data Storage DBTable Design ● Atomic Items ● Contiguous Objects ● Fields are RT-static ● Concurrent ● Use scoped blocking ● Objects are stable Extendible Field idx Object idx Item

Shared Data Storage Txn. Args Allocation Compile Time Static Tracking Index Reservation Number of Args Compile Time Const Typed Storage No, 64 bits Named Yes to. YAML Extendible Runtime Const Memory Schema Runtime Const Any size, Any Type, Packed booleans, Constructor/Destructor Yes

Shared Data Storage - Allocation Txn. Args compile time size Extendible CORE OBJECT arg 0 arg 1 arg 2 Reserve Register arg 0 allocated size

Shared Data Storage - Inheritance Txn. Args compile time size Extendible CORE OBJECT arg 0 Derived Reserve OBJECT arg 1 arg 2 Derived OBJECT Register arg 0 allocated size

Shared Data Storage - Inheritance Txn. Args compile time size Extendible CORE OBJECT arg 0 Derived arg 1 arg 2 Derived OBJECT arg_d 0 OBJECT arg 0 arg_d 0 allocated size

Shared Data Storage - API Txn. Args Extendible C Obj. Txn. Arg. Index. Reserve(name, *idx) Obj. Ext. Add(name, size, *idx) Obj. Txn. Arg. Index. Lookup(name, *idx) Obj. Ext. Find(name, *idx, *size) void* Obj. Txn. Arg. Get(obj, idx) void* Obj. Ext. Get(obj, idx, *size) Obj. Txn. Arg. Set(obj, idx, *arg) Obj. Ext. Set(obj, idx, *arg)

Shared Data Storage - API Txn. Args Extendible C++ Ext. Field. Id<Obj, T> idx; Obj. Txn. Arg. Index. Reserve(name, *idx) Ext. Add(name, *idx) Obj. Txn. Arg. Index. Lookup(name, *idx) Ext. Find(name, *idx) void* Obj. Txn. Arg. Get(obj, idx) T const& Ext. Get(obj, idx) Obj. Txn. Arg. Set(obj, idx, *arg) T& Ext. Set(obj, idx)

Shared Data Storage Extendible is a “runtime static” dictionary. 1. 2. 3. 4. Plugin Init: define fields Fields are Frozen System computes memory alignment and size Plugin Loaded: find defined fields a. Assert same types 5. Plugin Hooks: efficiently access allocations 1. Plugin. Init(Reload): find defined fields a. Assert same types

Shared Data Storage DBTable Design ● Atomic Items ● Contiguous Objects ● Fields are RT-static ● Concurrent ● Use scoped blocking ● Objects are stable Extendible Field idx Object idx Item

Shared Data Storage DBTable<K, V>(N) ● vector<part>(N) ○ Scope lock shared_mutex ○ unordered_map<K, share_ptr<V>>

Shared Data Storage DBTable<K, V>(N) ● share_ptr<V> obtain(K) ● operator[], find, pop, clear ● visit(function<bool(K const &, V &)>)

Shared Data Storage DBTable Design ● Atomic Items ● Contiguous Object ● Fields are RT-static ● Concurrent ● Use scoped blocking ● Objects are stable Extendible Field idx Object idx Item

Shared Data Storage Forgot Simplify Internals

How jjrushford vmamidi Config Routing Architecture Parent. Selection Host. Status Up. Stream Generator Modular Routing Logic Shared Data Storage DBTable Extendible Acid. Ptr acanary solidwallofcode

Modular Routing Logic Async Update Logic Producer Shared Data Per Txn Logic Consumer

Modular Routing Logic Remap Parent. Selection CARP Host. DB Session. Mgr Connection. Count Http. SM Send Request Upstream Failover RR Latch Consist. Hash Health DNScache Reuse Ssn No. Loop Open Ssn

Up. Stream Generator Remap Get. Session Strategy Config Failover RR Latch Consist. Hash Health DNScache Reuse Ssn Send Request Upstream No. Loop Open Ssn

Up. Stream Generator Remap Txn. Set. Target. Addr Get. Session Strategy Config CDN Routing Logic Failover RR Latch Consist. Hash Connection Logic Health Send Request Upstream DNScache Reuse Ssn No. Loop Open Ssn

Up. Stream Generator Remap Get. Session Strategy Config (Override) Send Request Upstream 404

Up. Stream Generator

What jjrushford vmamidi Config Routing Architecture Parent. Selection Host. Status Up. Stream Generator Modular Routing Logic Shared Data Storage DBTable Extendible Acid. Ptr acanary solidwallofcode

Host Status • • • Hosts are configured in remap. config and parent. config Status of hosts are managed passively, marked unavailable, due to transaction timeouts or connection errors. Hosts are managed by removing them from config files for maintenance or mitigation of connectivity issues.

Parent selection Host Status changes (completed) • Added Host. Status class along with changes to Parent selection and traffic_ctl to mark parent proxies offline. # traffic_ctl host down parent-proxy 01. cdn. com • Marks the parent unavailable for use, next available parent is used. # traffic_ctl host up parent-proxy 01. cdn. com

Origin host status changes • Changes to Http. Transact. cc to mark origins offline using traffic_ctl # traffic_ctl host down origin-01. cdn. com • Objects served stale from cache with “Warning: 111” Revalidation Failed due to inability to reach the origin server). proxy. config. http. cache. max_stale_age • HTTP 503 “Origin server congested” is sent to client if object is not in cache or is too stale.

Next steps • Health checking tools coming soon from Oath & Apple • Convert parent. config to yaml format • Combine parent. config functionality with remap. config.

Example parents. yaml hosts: p 1: &p 1 host: p 1 -cache-cdn. com protocol: - http: 80 - https: 443 healthcheck: url: tcp: //192. 168. 1. 80 p 2: &p 2 host: p 2 -cache-cdn. com protocol: - http: 80 - https: 443 healthcheck: url: tcp: //192. 168. 1. 81 groups: - g 1: &g 1 - *p 2

Example strategy. yaml # #include parents. yaml # strategy: policy: consistent_hash # ( Enum of 'consistent_hash' or 'first_live' or 'rr_strict' or 'rr_ip' or 'latched') hash_key: path+query groups: - *g 1 - *g 2 protocol: http failover: ring_mode: exhaust_rings response_codes: - 404 - 503 health_check: - passive