pJigsaw A Clusterbased Web Server with Cooperative Caching

p-Jigsaw: A Cluster-based Web Server with Cooperative Caching Supports Ge Chen, Cho-Li Wang, Francis C. M. Lau (Presented by Cho-Li Wang) The Systems Research Group Department of Computer Science and Information Systems The University of Hong Kong

What’s a cluster ? ÜA cluster is a type of parallel or distributed processing system, which consists of a collection of interconnected standalone/complete computers cooperatively working together as a single, integrated computing 2

Rich Man’s Cluster Ü Computational Plant (C- Plant cluster) Ü Rank: 30 at TOP 500 (11/2001) Ü 1536 Compaq DS 10 L 1 U servers (466 MHz Alpha 21264 (EV 6) microprocessor, 256 MB 48 cabinets, each of which contains 32 nodes ECC SDRAM) (48 x 32=1536) Ü Each node contains a 64 bit, 33 MHz Myrinet network interface card (1. 28 Gbps/s) connected to a 64 -port 3

Poor Man’s Cluster HKU Linux Cluster Ü 32 733 MHz Pentium III PCs, 392 MB Memory Ü Hierarchical Ethernetbased network : four 24 port Fast Ethernet switches + one 8 -port Gigabit Ethernet backbone switch) Ü Additional 80 -port Cisco Catalyst 2980 G Fast Ethernet Switch Ü 4

Cluster Computer Architecture Programming Environment (Java, C, MPI, HPF) Web Windows User Interface Other Subsystems (Database, Web server OLTP, etc. ) Single System Image Infrastructure Availability Infrastructure OS OS Node High-Speed LAN (Fast/Gigabit Ethernet, SCI, Myrinet) 6

Talk Outline Ü Motivation -- The Need for Speed Ü Cluster-based Solutions Ü System Architecture of p-Jigsaw Ü Performance Evaluation Ü Conclusion and Future Work Ü Other SRG Projects 7

The Challenges Ü Netscape Web site in November 1996: 120 million hits per day Ü Microsoft Corp. Web site received more than 100 M hits per day. (1, 200 hits per second) Ü Olympic Winter Games 1998 (Japan): 634. 7 M (16 days), peak day 57 M, peak minute 110 K Ü Winbledon July 1999, 942 M hits (14 days), peak day 125 M, peak minute 430 K Ü Olympic Games 2000 : peak day 502. 6 M, peak minute 600 K hits. (10 K hits per second) 8

The Need for Speed Ü Internet user popularity is growing very fast According to United States Internet Council’s report, regular Internet user has increased from less then 9 M in 1993 to more than 300 M in the summer of 2000, and is still growing fast Ü Broadband becomes popular According IDG’s report, 57% of the workers in U. S access Internet via broadband in office. The figure will be more than 90% by 2005. Home broadband user will also increase from less than 9 M now to over 55 M by 2005 Ü HTTP requests account for larger portion of Internet traffic now One study shows that HTTP activity has grown to 9

The Need for Speed Ü The Need for Speed Growing user number Faster last-mile connection speed Increasing portion of HTTP requests accounts for all Internet traffic Ü Require a more powerful Web server architecture 11

Cluster-Based Solution Cluster -- A Low-cost yet efficient parallel computing architecture

Dispatcher-based A network component of the Web-server system acts as a dispatcher which routes the requests to one of the Web servers to fulfill the load balancing. Each Web server works individually. n Layer 4 switching with level 2 address translation : One-IP, IBM e. Network, Web. Mux, LVS in DR mode n Layer 4 switching with level 3 address translation : Cisco Local. Director, Alteon ACEDirector, F 5 Big/IP, LVS in NAT mode. n Layer 7 switching (Contentbased): LARD, IBM Web Accelerator, Zeus Load Balancer (ZLB) client Internet Dispatcher 15

p-Jigsaw -- Goals Ü High Efficiency: Explore aggregate power of cluster resources (CPU, memory, disk, network bandwidth). Explore in-memory Web caching on cluster-based Web servers Ü High Scalability Maintain high cache hit rate and high throughput as cluster size grows Eliminate potential bottleneck in the overall design Ü High Portability Multi-platform support Heterogeneous cluster 17

Main Features of p-Jigsaw Web servers Ü Global Object Space (GOS) Ü Hot Objects Caching Ü Cooperative Object Caching Ü Distributed Cache Replacement Algorithms 18

Global Object Space (All Web objects in system are visible and accessible to every node through GOS) Global Object Space Hot Object Cache Memory Cache p-Jigsaw JVM JVM OS OS Server Node High-Speed LAN Server Node 19

HN AGAC CCNN LAC Incoming Request: http: //p-Jigsaw. org/node 4/dir 2/pic 1. jpg : Home Node : Approximated Global Access Counter : Cache Copy Node Number : Local Access Counter Global Object Table for Node 1 Object URL HN /node 4/dir 2 4 … /node 1/dir 12/fig 121. bmp Search on GOT 1 (Hashing) 3 6 1 Miss! Object URL AGAC LAC HN /node 3/dir 31/fig 311. jpg 15000 100 3 /node 1/dir 12/fig 121 11000 50 1 … … Cached in node 1 CCNN … … … 1 11000 2 -3 -4 4 Object URL HN AGAC CCNN … … /node 4/dir 12/pic 1. jpg 4 15000 2 -3 -4 5 Cached copy is forwarded from node 2, 3, or 4, depends on the server workload Hot Object Cache Hard disk Node 2 Node 3 Hard disk Node 1 Redirect the request to node 4 (Home node of the requested page) Global Object Table for Node 4 Local Object Table for Node 1 2 AGAC Node 4 23

Distributed Cache Replacement Ü Two LFU-based Algorithms are Implemented: LFU-Aging: AGAC/ 2 ; every △ t Weighted-LFU: AGAC/ (file size) Global LRU (GLRU) is implemented for comparison Ü Try to cache the “hottest objects” in global object space Ü Cached object’s life time is set according to HTTP timestamp. Cache consistency is maintained by invalidation scheme. 24

Update of Access Counters GOT for Node 1 Object URL/Partial URL AGA C GOT for Node 3 HN 45+200 AGAC = 245 HN /node 3/dir 31/doc 3. html /node 3/dir 31/doc 3. ht ml 3 /node 4/dir 41 4 /node 1/dir 11/doc 1. ht 105 ml for Node 1 LOT … Object URL 3 /node 3/dir 32/pic 3. jpg 50 245 200 … … … 3 1 LAC …AGAC… HN /node 3/dir 32/pic 3. jp g 45 0 200 245 3 /node 1/dir 11/doc 1. h tml 24 105 1 … … … HOC HOC 1 2 3 4 GOS LAC is periodically sent back to objects’ HN to maintain an approximate global access counter for every cached object 25

Experiment Setup Ü Ü Ü 32 -node PC cluster. Each node consists of a 733 MHz Pentium III PC running Linux 2. 2. 4. The nodes are connected with an 80 -port Cisco Catalyst 2980 G Fast Ethernet Switch. A NFS server (2 -way SMP) with Gigabit Ethernet link to the switch. 16 nodes acts as clients, and the rest as Web servers. Each of the server nodes has 392 MB physical memory installed 32 -node PC cluster 27

Experiment Results Effects of Scaling the Cluster Size 4. 56 2. 02 28

Experiment Results Effects of Scaling the Cache Size Aggregated Cache size for 16 nodes = 1. 8% (8 MB per node), 3. 6%, 7. 2%, and 14. 4% (64 MB per node) of the size of the data set 29

Analysis of Requests Handle Patterns Ü Local Cache Object (in local memory) The server that receives the request has the requested object in its local hot object cache. Ü Peer Node Cache Object (in remote memory) The server that receives the request does not have the requested object in its local hot object cache. The object is fetched from either the home node or a or other peer nodes. Ü Disk Object (local or remote disk) The requested object is not in the global object space, and has to be fetched from the file server. This has the longest serving time. 31

Analysis of Requests Handle Patterns n~60% LFU-based algorithms show high local cache hit rates. With 64 MB cache per node, the local cache hit rate is around 60% for both Weighted-LFU and LFUAging, 32

Analysis of Requests Handle Patterns ~50% ~35. 2% ~6. 7% ~25% With small cache size (8 MB), the cooperative cache can improve the global cache hit rate and reduce the costly file server disk access, which is a common bottleneck for a website. 33

Conclusions Ü Use of cluster wide physical memory as object cache can lead to improved performance and scalability of Web server systems Ü With relatively small amount of memory dedicated for object content caching, we are able to achieve a high hit rate with the cooperative caching Ü Favor replicating more hot objects rather than squeezing more different objects into the global object space. 36

Future Work Ü The HKU “Hub 2 World” Project Build a giant proxy cache server on a large PC cluster with HKU’s 300 -node Gideon cluster based on p-Jigsaw Cache hot objects on 150 GB in-memory cache (0. 5 GB x 300) + 12 Terabytes disk space (40 GB x 300) Design of new caching algorithms 37

Other SRG Projects Welcome to download our software packages and test them on your clusters. URL: http: //www. srg. csis. hku. hk/

Current SRG Clusters 39

JESSICA 2 – A Distributed JVM A Multithreaded Java Program Thread Migration JVM JVM JVM Global Object Space 40

JUMP Software DSM Allows programmers to assume a globally shared virtual memory, even they execute program on nodes that do not physically share memory Ü The DSM system will maintain the memory consistency among different machines. Data faulting, location, and movement are handled by the DSM. Ü Proc 1 Proc 2 Proc N-1 Proc N Mem 1 Mem 2 Mem N-1 Mem N Network Globally Shared Virtual Memory 41

HKU DP-II on Gigabit Ethernet Single-trip Latency Test (Min: 16. 3 µs) Bandwidth Test (Max: 79. 5 MB/s) RWCP Giga. E PM : 48. 3 us round-trip latency and 56. 7 MB/s on Essential Gigabit Ethernet NIC Pentium II 400 MHz. RWCP Giga. E PM II : 44. 6 us round trip time. 98. 2 MB/s bandwidth on Packet Engines G-NIC II for connecting Compaq XP-1000 (Alpha 21264 at 500 MHz. ), . 42

SPARKLE Project ÜA Dynamic Software Architecture for Pervasive Computing – “Computing in Small” Facets Application Won’t Fit Applications distributed as monolithic blocks Our component-based solution 43

SPARKLE Project Service Providers Facet Servers Intelligent proxies Co-operative Caching (User Mobility) Facet Retrieval Execution Servers Facet Query Delegation/ Mobiie code Computational Grid Clients (Linux + JVM) Peer-to-Peer Interaction Overview of the proposed software architecture 44

Cluster. Probe : Cluster Monitoring Tool 45

Q&A For more information, please visit http: //www. csis. hku. hk/~clwang