Building Scalable Web Architectures Aaron Bannert aaronapache org

Building Scalable Web Architectures Aaron Bannert aaron@apache. org / aaron@codemass. com

Goal To build a reliable, scalable, cheap, flexible, extendable internet application.

The Age of LAMP What does a LAMP architecture give us?

Scalability l Grows l Stays l Can in small steps up when it counts grow with your traffic l Room for the future

Reliability l High Quality of Service l Minimal Downtime l Stability l Redundancy l Resilience

Low Cost l Little or no software licensing costs l Minimal hardware requirements l Abundance l Reduced of talent maintenance costs

Flexible l Modular l Public l Open Components APIs Architecture ¡ Vendor ¡ Many Neutral options at all levels

Extendable l Free/Open Source Licensing ¡ Right to Use ¡ Right to Inspect ¡ Right to Improve l Plugins ¡ Some Free ¡ Some Commercial ¡ Can always customize

Free as in Beer? Price Speed Quality Pick any two.

LAMP-like Architectures

The Big Picture

External Caching Tier

Web Serving Tier

Application Server Tier

Internal Cache Tier

Database Tier

Misc. Services (DNS, Mail, etc…)

The Glue • Routers • Switches • Firewalls • Load Balancers

Software Choices Building LAMP Software

External Caching Tier

External Caching Tier l What is this? ¡ Squid ¡ Apache’s mod_proxy ¡ Commercial HTTP Accelerator

External Caching Tier l What does it do? ¡ Caches l Images, CSS, XML, HTML, etc… ¡ Flushes l outbound HTTP objects Connections Useful for modem users, frees up web tier ¡ Denial of Service Defense

External Caching Tier l Hardware Requirements ¡ Lots of Memory ¡ Moderate to little CPU ¡ Fast Network ¡ Moderate Disk Capacity l ¡ l Room for cache, logs, etc… (disks are cheap) One slow disk is OK Two Cheapies > One Expensive

External Caching Tier l Other Questions ¡ What ¡ How to cache? much to cache? ¡ Where to cache (internal vs. external)?

Web Serving Tier

Web Serving Tier l What is this? ¡ Apache ¡ thttpd ¡ Tux Web Server ¡ IIS ¡ Netscape

Web Serving Tier l What does it do? ¡ HTTP, ¡ Serves HTTPS Static Content from disk ¡ Generates l Dynamic Content CGI/PHP/Python/mod_perl/etc… ¡ Dispatches l requests to the App Server Tier Tomcat, Weblogic, Websphere, JRun, etc…

Web Serving Tier l Hardware Requirements ¡ Lots and lots of Memory l Memory is main bottleneck in web serving l Memory determines max number of users ¡ Fast Network ¡ CPU depends on usage ¡ l Dynamic content needs CPU l Static file serving requires very little CPU Cheap slow disk, enough to hold your content

Web Serving Tier: Zero-copy l Performance ¡ Dedicated l Hint static content servers Modern web servers are very good at serving static content such as • HTML • CSS • Images • Zip/GZ/Tar files

Web Serving Tier l Performance ¡ Stateless Hint Sessions l Each connection is a fresh start l Server remembers nothing ¡ Benefits? l Allows Better Caching l Scales Horizontally

Web Serving Tier l Choices ¡ How much dynamic content? ¡ When to offload dynamic processing? ¡ When to offload database operations? ¡ When to add more web servers?

Application Server Tier

Application Server Tier l What does it do? ¡ Dynamic Page Processing JSP l Servlets l Standalone mod_perl/PHP/Python engines l ¡ Internal l Services Eg. Search, Shopping Cart, Credit Card Processing

Application Server Tier 1. How does it work? 1. Web Tier generates the request using l l l 2. HTTP (aka “REST”, sortof) RPC/Corba Java RMI XMLRPC/Soap (or something homebrewed) App Server processes request and responds

Application Server Tier l Caveats ¡ Decoupling of services is GOOD l Manage Complexity using well-defined APIs ¡ Don’t decouple for scaling, change your algorithms! ¡ Remote Calling overhead can be expensive l Marshaling of data l Sockets, net latency, throughput constraints… l XML, Soap, XMLRPC, yuck (don’t scale well) l Better to use Java’s RMI, good old RPC or even Corba

Application Server Tier l More Caveats ¡ Remote Calling can introduce new failure scenarios l Classic Distributed Problems • How to detect remote failures? • How long to wait until deciding it’s failed? How to react to remote failures? What do we do when all app servers have failed?

Application Server Tier l Hardware Requirements ¡ Lots and Lots of Memory l App Servers are very memory hungry l Java was hungry to being with l Consider going to 64 bit for larger memory-space ¡ Disk depends on application, typically minimal needed ¡ FAST CPU required, and lots of them ¡ (This will be an expensive machine. )

Database Tier

Database Tier l Available DB Products ¡ Free/Open Source DBs l l ¡ Postgre. SQL GNU DBM Ingres SQLite Commercial l l Oracle MS SQL IBM DB 2 Sybase Sleepy. Cat l l My. SQLite m. SQL Berkeley DB

Database Tier l What does it do? ¡ Data Storage and Retrieval ¡ Data Aggregation and Computation ¡ Sorting ¡ Filtering ¡ ACID properties l (Atomic, Consistent, Isolated, Durable)

Database Tier l Choices ¡ How much logic to place inside the DB? ¡ Use Connection Pooling? ¡ Data Partitioning? l Spreading a dataset across multiple logical database “slices” in order to achieve better performance.

Database Tier l Hardware Requirements ¡ ¡ ¡ Entirely dependent upon application. Likely to be your most expensive machine(s). Tons of Memory Spindles galore RAID is useful (in software or hardware) l Reliability usually trumps Speed • RAID levels 0, 5, 1+0, and 5+0 are useful ¡ ¡ ¡ CPU also important Dual power supplies Dual Network

Internal Cache Tier

Internal Cache Tier l What is this? ¡ Object l What Cache Applications? ¡ Memcache ¡ Local l Lookup Tables BDB, GDBM, SQL-based ¡ Application-local Caching (eg. LRU tables) ¡ Homebrew Caching (disk or memory)

Internal Cache Tier l What does it do? ¡ Caches objects closer to the Application or Web Tiers ¡ Tuned for your application ¡ Very Fast Access ¡ Scales Horizontally

Internal Cache Tier l Hardware ¡ Lots l Requirements of Memory Note that 32 bit processes are typically limited to 2 GB of RAM ¡ Little or no disk ¡ Moderate to low CPU ¡ Fast Network

Misc. Services (DNS, Mail, etc…)

Misc. Services (DNS, Mail, etc…) l Why mention these? ¡ Every LAMP system has them ¡ Crucial but often overlooked ¡ Source of hidden problems

Misc. Services: DNS l Important Points ¡ Always have an offsite NS slave ¡ Always have an onsite NS slave ¡ Minimize l network latency Don’t use NAT, load balancers, etc…

Misc. Services: Time Synchronization l Synchronize the clocks on your systems! l Hints: ¡ Use NTPDATE at boot time to set clock ¡ Use NTPD to stay in synch ¡ Don’t ever change the clock on a running system!

Misc. Services: Monitoring l System Health Monitoring ¡ Nagios ¡ Big Brother ¡ Orcalator ¡ Ganglia l Fault Notification

The Glue • Routers • Switches • Firewalls • Load Balancers

Routers and Switches l Expensive l Complex l Crucial Piece of the System l Hints ¡ Use Gig. E if you can l Jumbo Frames are GOOD ¡ VLans to manage complexity ¡ LACP (802. 3 ad) for failover/redundancy

Load Balancers l Hardware ¡ Software vs. Software is complex to set up, but cheaper ¡ Hardware ¡ IMHO: is expensive, but dedicated Use SW at first, graduate to HW

Load Balancers l What services to balance? ¡ HTTP Caches and Servers, App Servers, DB Slaves l What NOT to balance? ¡ DNS ¡ LDAP ¡ NIS ¡ Memcache ¡ Spread ¡ Anything with it’s own built-in balancing

Message Busses l What is out there? ¡ Spread ¡ JMS ¡ MQSeries ¡ Tibco Rendezvous l What does it do? ¡ Various forms of distributed message delivery. l Guaranteed Delivery, Broadcasting, etc… ¡ Useful for heterogeneous distributed systems

What about the OS? Operating System Selection

Lots of OS choices l Linux l Free. BSD l Net. BSD l Open. Solaris l Commercial Unix

What’s Important? l Maintainability ¡ Upgrade Path ¡ Security Updates ¡ Bug Fixes l Usability ¡ Do your engineers like it? l Cost ¡ Hardware Requirements ¡ (you don’t need a commercial Unix anymore)

Features to look for l Multi-processor l 64 bit Support Capable l Mature Thread Support l Vibrant User Community l Support for your devices

Hardware Choices Building LAMP Hardware

Commodity Hardware Discussion l Consistency ¡ Consistency vs. Specialization reduces maintenance costs Less Burn-in testing l Fewer drivers to support l Fewer OS variants l Fewer types of security updates, upgrades l In Sort: “Don’t throw hardware at the problem. ” l ¡ However, l specialization may improve ROI Put the money where best needed

Commodity Hardware Discussion l What I do when planning for growth: ¡ Specialize in the beginning When cost is more important l And designs aren’t yet mature l ¡ Design for horizontal scalability Plan on machine-sized pieces l Want to grow by just adding more boxes l ¡ Eventually settle on two or three machine types

In-House vs. Colocation l Almost no reason to stay in-house these days l Colos keep getting cheaper l Leased lines are still expensive

Beige-Box vs. Name Brand l Determine ¡ Talk l How your Req’s ahead of time to your engineers First important is a support plan? ¡ Hardware l Name Brand usually has fewer options ¡ Works l Seek will break, plan on it well if they have exactly what you need a neutral technical advisor l In the end it should come down to cost

Disk Drive Technologies l SCSI ¡ ¡ l Expensive Big (300 GB) Fast Reliable IDE ¡ ¡ Cheap Huge (500 GB!) Slow On-board support, often w/ RAID 0/1 Use SCSI for Performance l Use IDE for cluster nodes l ¡ IDE w/ RAID for cheap speed

Disk Drive Technologies l PATA ¡ Immature drivers l l Particularly w/ OSS • Linux has poor support ¡ ¡ SATA ¡ ¡ Tried and Tested Obsolete Prices coming down Unnecessary addons l Hot Swap not often needed, costs more l SATA is not SCSI ¡ l The fast SATAs cost as much as SCSI SATA not quite there for servers

Disk Drive Technology: Spindles l Number ¡ More of Spindles spindles can give Higher Throughput l Higher Concurrency l • Concurrency is crucial for Databases l Reliability • Failover drives, mirrors

Memory Technologies l ECC ¡ l Expensive Use only for keystone machines l Non-ECC ¡ l Cheap, Fast Use for cluster nodes

Processor Technologies: SMP l Multiple Processors ¡ Significantly higher cost l l ¡ Less Reliable l ¡ More parts to break Requires MP-capable OS l l ECC, Dual Power Expensive Chassis, Motherboard Good in Linux 2. 6, Solaris, Free. BSD 5. x Dual CPU systems cost more than double ¡ Possible exception: Dual-Core CPUs

Processor Technologies: 64 bit Most 32 bit OSes limit each process to 2 GB l Some 32 bit BIOSes are limited to 3. 6 GB RAM l 64 bit chips are still expensive l 64 bit OSes are becoming quite mature l ¡ ¡ l Solaris 10 (AMD 64) Linux 2. 6 (x 86_64) Programs work but not yet tuned ¡ ¡ Java looks good My. SQL not so good

Summary

Design for Horizontal Scalability l Design Stateless Systems l Decouple Internal Services l Write well-defined APIs

Use Commodity Parts l Standardize Hardware l Use Commodity Software (Open Source!) l Avoid Fads

THE END Thank You