Apache Traffic Server Spring Summit 2017 SSL Lazy

Problem Statement • Need to load tens of thousands of SSL Certs • Certs

Design Goals • Lazy load cert on demand • First request for https •

Design Features • Loaded cert TTL • • "Evicts" cert from memory, freeing memory

Version 2 • Straight to redis every time • Uses local redis slave with

Future Features • Command line messaging into plugin (Thanks, amc!) • Remove from blacklist

Slides: 6

Download presentation

Apache Traffic Server Spring Summit 2017 SSL Lazy Loader Plugin Design Steven Feltner reveller – IRC sfeltner@godaddy. com steven. feltner@gmail. com Spring, 2017 Copyright© 2017 Go. Daddy Inc. All Rights Reserved.

Problem Statement • Need to load tens of thousands of SSL Certs • Certs get updated every few minutes • Customers purchase new certs, update certs, cancel accounts or revoke certs • "Statically" configuring through ssl_multicert. config • Requires minutes to parse and reload • Often got stuck in a reload circle • Lengthy startup times • With such long start up times, upgrades required pulling nodes from rotation 2 Copyright© 2017 Go. Daddy Inc. All Rights Reserved.

Design Goals • Lazy load cert on demand • First request for https • Lookup in a domain map to make sure we even need to pursue the effort • Load cert from redis • Subsequent requests for same domain go straight to lookup table without the need for redis lookup • Support for redis sentinels • Provides high availability for redis clustering • Provides configuration data about cluster 3 Copyright© 2017 Go. Daddy Inc. All Rights Reserved.

Design Features • Loaded cert TTL • • "Evicts" cert from memory, freeing memory usage Offloads infrequently used certs Refreshes certs already loaded into memory in case of updates/revoked certs Difficult to balance configuration • Config time too short - it is not effective • Config time too long - changes to certs lag behind • Cert look up blacklist • Prevents possible DDo. S attempts by immediately returning without looking up a cert in redis • Configurable as time-based or LRU • Customers can be a PITA when they purchase/renew and then immediately ping their site • Cust who waited for 3 weeks after his cert expired to renew 4 Copyright© 2017 Go. Daddy Inc. All Rights Reserved.

Version 2 • Straight to redis every time • Uses local redis slave with priority of 0 • Fast sync time • 200 k certs were synced into a new instance in less than 2 secs from main cluster • Stays current with main cluster but cannot be promoted to master • Eliminates single point of failure • If local redis is not available, gets config from redis Sentinel for a read-only node in main cluster • Once local instance comes back online, all traffic routed to on-box redis • Reduces overall network traffic by localizing accesses to local redis • Reload every cert on every request • • 5 No need for cert TTL Certs are never stored in memory so better memory utilization Certs never go stale; updates are immediate Revoked certs are removed from redis immediately Copyright© 2017 Go. Daddy Inc. All Rights Reserved.