Apache Performance Tuning Part 2 Scaling Out Sander

Apache Performance Tuning Part 2: Scaling Out Sander Temme <sander@temme. net>

Agenda l l l Introduction Redundancy in Hardware Building Out: Separate Tiers Building Out: Load Balancing Putting it All Together Conclusion

Introduction l Why do This? l l l Scalability (Oh my gosh, I’m so popular!) Reliability (We need five nines!) Why NOT do This? l It costs money

Redundancy in Hardware l Moving Parts Break l l l Hard Disks Power Supplies Hard Disks Fans Hard Disks Buy High Quality Disks l l l Refurbished, OEM, Brand Name Which has longer warranty? Which is more reliable?

Server Configuration l Mirror those Disks l l Install the RAID utility Have it warn you RAID is no good if you don’t learn of failures! Redundant Power Supplies l On different circuits

Scaling Vertically

Scaling Vertically l l Move Services to Other Hosts Pros: l l Less resource contention Specialized hardware Scale out tiers individually Cons: l l Development/Deployment harder More hosts to manage

Load Balancing Schemes l l DNS Tricks Peer Server Pools l l l Network Load Balancing (Win 2 k 3) Whackamole Load Balancing Appliance l l Box from F 5, Juniper, Cisco, Foundry, … Linux Virtual Server

DNS Round-Robin l l l Easy! Multiple A Records in DNS Zone File Not Smart: l l l DNS Lookups are cached Load on Server Outage

Example Zone File scalingout. org. 86400 2006051401 86400 7200 86400 ) scalingout. org. gw bagheera IN SOA ns. scalingout. org. sctemme. scalingout. org. ( ; Serial ; refresh (1 day) ; retry (2 hours) ; expire (10 days) ; minimum (1 day) IN NS bagheera. scalingout. org. IN IN A A 10. 11. 0. 1 10. 11. 0. 2 mail ns IN IN CNAME bagheera www IN IN IN A A A 10. 113 10. 114 10. 115 ; . . .

Peer-based: NLB l l Windows 2000 Server Enterprise Ed. , Windows Server 2003 Up to 32 hosts in cluster All hosts assume cluster IP, MAC NLB makes LB decision l l Only one host gets to answer TCP handshake Should be application independent

Load Balancing Device Client Internet

Load Balancing l l l One Load Balancer Many Web Servers Choice of Balancing Schemes l l Reliability l l Round-robin, Least Used, … Heartbeats, unavailable servers don’t receive requests Feature War

Example: Linux Virtual Server l l l Free, Open Source, etc. IP Virtual Server module in kernel Lots of auxiliary modules l l l Like a box of Legos May come with Your Distribution Do It Yourself http: //www. linuxvirtualserver. org/

Example: mod_proxy_balancer l l l New in Apache HTTP Server 2. 2 Part of mod_proxy Two Load Balancing Methods l l l By number of requests By number of bytes Detects failed backends

Apache Configuration Listen 80 Log. Level debug Transfer. Log logs/access_log Load. Module proxy_modules/mod_proxy. so Load. Module proxy_http_modules/mod_proxy_http. so Load. Module proxy_balancer_modules/mod_proxy_balancer. so Proxy. Pass / balancer: //mycluster/ Proxy. Pass. Reverse / http: //1. 2. 3. 4: 80 Proxy. Pass. Reverse / http: //1. 2. 3. 5: 80 <Proxy balancer: //mycluster> Balancer. Member http: //1. 2. 3. 4: 80 Balancer. Member http: //1. 2. 3. 5: 80 </Proxy>

Example: Tomcat and mod_jk l l l Apache + mod_jk Multiple Tomcat servers Balancer Worker

Apache Configuration Load. Module jk_module /Volumes/Files/asf/httpd-r 415210 w/modules/mod_jk. so JKMount /servlets-examples/* loadbalancer JKMount /*. jsp loadbalancer Jk. Mount /jkmanager/* jkstatus JKLog. File logs/jk_log JKLog. Level debug JKWorker. Property worker. list=loadbalancer, jkstatus JKWorker. Property worker. tc 1. port=15109 JKWorker. Property worker. tc 1. host=localhost JKWorker. Property worker. tc 1. type=ajp 13 JKWorker. Property worker. tc 1. lbfactor=1 JKWorker. Property worker. tc 2. port=15209 JKWorker. Property worker. tc 2. host=localhost JKWorker. Property worker. tc 2. type=ajp 13 JKWorker. Property worker. tc 2. lbfactor=1 JKWorker. Property worker. tc 3. port=15309 JKWorker. Property worker. tc 3. host=localhost JKWorker. Property worker. tc 3. type=ajp 13 JKWorker. Property worker. tc 3. lbfactor=1 JKWorker. Property worker. loadbalancer. type=lb JKWorker. Property worker. loadbalancer. balance_workers=tc 1, tc 2, tc 3 JKWorker. Property worker. jkstatus. type=status

Tomcat Configuration l Put same content on all Tomcats Edit conf/server. xml: l jvm. Route must match jk worker name! l <Engine name="Catalina" default. Host="localhost" jvm. Route="tc 1">

Tomcat Session Replication l l Share Http. Session objects across instances One instance dies, session lives on Apache will route requests to other instance Uses IP Multicast

Session Replication Configuration l l Uncomment <Cluster> element in server. xml Put empty <distributable /> element in <webapp> element in web. xml

Tuning the Database Tier l l Not my area (sorry) Give Money to Oracle Consultants l l (or My. SQL) (or …) Tip: Separate Read and Write Operations l l Replicate from Write db to Read db data slightly stale l Does it matter?

Putting it All Together Read-only Internet Client Write-only

Q&A

Conference Roadmap l l l T 01 Scalable Internet Architectures T 12 Open. Solaris Zones: Web server and application consolidation TU 8 PHP and My. SQL Best Practices WE 26 The Ins and Outs of Clustered Logging TH 27 Configuring Apache Derby for Performance and Durability FR 22 Clustering Apache Geronimo

Current Version http: //people. apache. org/~sctemme/Apcon. US 2006/WE 23/

Thank You