C Edward Chow Department of Computer Science University

C. Edward Chow Department of Computer Science University of Colorado at Colorado Springs Sponsored by Computer Comm. Lab/ITRI

Content Switch Topics • • What is a Content Switch? What Services it Can Provide Content Switch Example Related Technologies Content Switch Architecture and Basic Operations TCP Delay Binding and Related Improvement Content Switch Rule and Conflict Detection Conclusion Content Switch Edward Chow

Content Switch (CS) • Route packets based on high layer (Layer 5/7) headers and content. • Examples: – Direct Web traffic based on pattern of • URLs, cookies – URL Switching • XML Tag Value– Web Switching – Can Route incoming email based on email address; Connect POP/IMAP based on login • Web switches and Intel XML Director/accelerator are special cases of content switch. Content Switch Edward Chow

What Services It Can Provide • Enabling premium services for e-commerce, ISP, and Web hosting providers • Load Balancing and High Available Server Clusters: Web, E-commerce, Email, Computing, File, SAN • Policy-based networking, differential/Qo. S services. • Firewall, Strengthening Do. S protection, cache/firewall load-balancing • ‘Flash-crowd' management • Email Spam Protection, Virus Detection/Removal • Applet Authentication/Filtering Content Switch Edward Chow

F 5 VRM Solution Site II losangeles. domain. com Internet Site I newyork. domain. com Router 3 -DNS BIG-IP Local DNS GLOBAL-SITE Webmaster Site III tokyo. domain. com Server Array User london. domain. com Content Switch Edward Chow

Intel Netstructure XML Director 7280 • Example of Rule: Server 1: create */order. asp & //Amount[Value >= 10000] Content Switch Edward Chow

Phobos In-Switch • Only load balancing switch in a PCI card form factor • Plugs directly into any server PCI slot • Supports up to 8, 192 servers, ensuring availability and maximum performance • Six different algorithms are available for optimum performance: Round Robin, Weighted Percentage, Least Connections, Fastest Response Time, Adaptive and Fixed. • Provides failover to other servers for high-availability of the web site • U. S. Retail $1995. 00 Content Switch Edward Chow

E-Commerce Example: 1. Client submits via HTTP/Post (or SOAP) the following purchase in XML: <purchase> <customer. Name>CCL</customer. Name> <customer. ID>111222333</customer. ID> <item><product. ID>309121544</product. ID> <product. Name>IBM Thinkpad T 21</product. Name> <unit. Price>5000</unit. Price> <no. Of. Units>10</no. Of. Units> <sub. Total>50000</sub. Total> </item> <item><product. ID>309121538</product. ID> <product. Name>Intel wireless LAN PC Card</product. Name> <unit. Price>200</unit. Price> <no. Of. Units>10</no. Of. Units> <sub. Total>2000</sub. Total> </item> <total. Amount>52000</total. Amount> </purchase> Content Switch Edward Chow

E-Commerce Example: 2. Content Switch • Content switch receives the packet. • Recognize it is a http post request from http request line POST /purchase. cgi HTTP/1. 1 • Recognize it is an XML document from the meta header content-type: TEXT/XML • Parsing XML content • Extract values of tag sequences: 52000 purchase/total. Amount CCL purchase/customer. Name • Rule 1 is matched and packet is routed to one of high. Speed. Servers. Rule 1: if (xml. purchase/total. Amount > 5000) route. To(high. Speed. Servers); Rule 2: if (xml. purchase/customer. Name == CCL) route. To(special. Customer. Servers); Content Switch Edward Chow

No Free Lunch: Penalty of Having Content Switch • Increased packet processing time. • For XML Director/Accelerator, it needs to parse XML document and match tag sequences. 1 -3? order of processing time Content Switch Edward Chow

Related Technologies • Application level solution: Proxy server; Apache/Tomcat/Servlet; Microsoft NLB • Kernel level layer 4 load balancing solution: http: //www. linuxvirtualserver. org/ – Joseph Mark’s presentation – LVS-NAT(Network Address Translation) web page – LVS-IP Tunnel web page – LVS-DR (Direct Routing) web page • Hardware solution: Cisco 11000, F 5 (Big IP), Alteon Web Systems, Foundry Networks (Server. Iron), Excellent information on: Foundry Server. Iron Installation and Configuration Guide, May 2000. • Routing table lookup: Longest prefix (Gupta/Mc. Keown) Content Switch Edward Chow

Basic Operations of Content Switching CS: Content Switching Incoming Packets CS Rule Editor Packet Classification Header Content Extraction CS Rule Matching Algorithm Network Path Info Server Load Status Content Switch Edward Chow Packet Routing (Load Balancing) Forward Packet To Servers

Content Switch Architecture Apostolopoulos Infocom 2000 Content Switch Edward Chow

Content Switch Architecture Case A: Controller finds there is an entry in its Hash Table, Route request to “sticky connection” outgoing port Hash Table Content Switch Client Edward Chow Real Server 1

Content Switch Architecture Case B: Step 1. Controller finds there is no entry in Hash Table, Route request to content switch processor Hash Table Content Switch Client Edward Chow Real Server 1

Content Switch Architecture CS Rules Step 2. CS processor a. Extract content/Match CS rules b. Route request c. Setup Sequence# modification on server side port Case B: Step 1. Controller finds there is no entry in Hash Table, Route request to content switch processor Hash Table Client Content Switch Edward Chow pkt Modification info Real Server 1

Content Switch Architecture CS Rules Case B: Step 1. Controller finds there is no entry in Hash Table, Route request to content switch processor pkt Modification info Real Server 1 Step 3. At server side port, Return pkts are modified Sequence#/IP addr/Chksum Route back to client Hash Table Client Content Switch Step 2. CS processor a. Extract content/Match CS rules b. Route request c. Setup Sequence# modification on server side port Edward Chow

Efficient Software Architecture • Tasks: Million packets with thousand of rules to match and load balancing algorithms to run. • How to assign tasks to the (network) processors and threads? – Packet Extraction (Understand header formats, XML parsing) – Content Switching Rule Matching – Packet Routing (Load Balancing, Bandwidth Control) • How Much Packet Processing Should Controllers Do? • What a controller can do? • A Typical Parallel Processing Problem? Content Switch Edward Chow

TCP Delay Binding (Splicing) client server content switch SYN(CSEQ) step 1 SYN(DSEQ) ACK(CSEQ+1) ACK(DSEQ+1) DATA(CSEQ+1) ACK(DSEQ+1) step 5 step 6 step 7 DATA(DSEQ+1) ACK(CSEQ+Len. R+1) step 8 step 9 step 2 step 3 step 4 SYN(CSEQ) SYN(SSEQ) ACK(CSEQ+1) ACK(SSEQ+1) DATA(SSEQ+1) ACK(CSEQ+len. R+1) step 10 ACK(DSEQ+ len. D+1) DATA(? ) 2 nd request ACK(? ) Content Switch step 11 Edward Chow ACK(SSEQ+len. D+1) len. R: size of http request. . len. D: size of return document

Improve Content Switching • Setup CS-Real Server connections ahead of time (Persistent HTTP Connections). Net. Scale Reduce TCP 3 -way handshake time • Pre-allocate Server Scheme (Guess Real Server based on the TCP Sync) • Sequence# modification on every return pkt Need to recompute checksum also. • Filter Scheme (Offload Sequence# modification/rule matching to real servers). • Buffering/Pipeline (aggregate) Requests Content Switch Edward Chow

Pre-Allocate Server Scheme client content switch SYN(CSEQ) ACK(SSEQ + 1) step 1 SYN(SSEQ) ACK(CSEQ+1) DATA(CSEQ+1) ACK(SSEQ+1) DATA(SSEQ+1) ACK(CSEQ+Len. R+1) ACK(SSEQ+len. D+1) step 2 step 3 Pre-allocated server SYN(CSEQ) SYN(SSEQ) ACK(CSEQ+1) ACK(SSEQ+1) step 4 DATA(CSEQ+1) ACK(SSEQ+1) step 5 DATA(SSEQ+1) step 6 ACK(CSEQ+len. R+1) ACK(SSEQ+len. D+1) • Guess routing decision based on IP/Port#/History. • Advantage: • Faster than TCP delay binding. • Possible direct route between client and server • Reduce session processing overhead convert Content Switch no need to. Edward Chowserver sequence #

Degenerated to TCP Delayed Binding If Guess is Wrong Pre-allocated client SYN(CSEQ) content switch step 1 SYN(SSEQ)/ ACK(CSEQ+1) ACK(SSEQ + 1) DATA(CSEQ+1)/ ACK(SSEQ+1) Server sent HTTP 404 step 6 step 7 step 8 Sequence # step 9 conversion needed for right server now step 10 DATA(SSEQ+1)/ACK(CSEQ+Len. R+1) ACK(SSEQ+len. D+1 Content Switch step 11 Edward Chow SYN(CSEQ) server step 2 SYN(SSEQ)/ ACK(CSEQ+1) step 3 ACK(SSEQ+1) step 4 DATA(CSEQ+1)/ACK(SSEQ+1) step 5 DATA(SSEQ+1) FIN(CSEQ+len. R+1)) Right server SYN(CSEQ) SYN(RSEQ)/ ACK(CSEQ+1) ACK(RSEQ+1) DATA(CSEQ+1)/ACK(RSEQ+1) DATA(RSEQ+1)/ACK(CSEQ+len. R+1) step 12 ACK(RSEQ+len. D+1)

Filter Process Scheme client SYN(CSEQ) content switch step 1 SYN(DSEQ)/ACK(CSEQ+1) step 2 ACK(DSEQ+1) Filter Process run on server step 3 DATA(CSEQ+1)/ACK(DSEQ+1) step 5 a step 4 step 5 b SYN(CSEQ) Migrate (Data, CSEQ, DSEQ) SYN(SSEQ)/ ACK(CSEQ+1) step 6 step 7 step 8 DATA(DSEQ+1) ACK(CSEQ+Len. R+1) ACK(DSEQ+len. D+1) Content Switch server step 9 step 10 Edward Chow ACK(SSEQ+1) DATA(CSEQ+1)/ACK(SSEQ+1) DATA(SSEQ+1) ACK(CSEQ+len. R+1) ACK(SSEQ+len. D+1)

Pre-allocate performance plot Series 1 - Basic scheme with no rule matching module inserted, i. e. , using default IPVS. Series 2 - Basic scheme with the rule matching module inserted. Series 3 - Pre-allocate scheme with all hits, i. e. , where all preallocate guesses were correct. Series 4 - Pre-allocate scheme with all misses, i. e. , where all pre-allocate guesses were wrong. Content Switch Edward Chow

Handling multiple requests in a Keep-Alive connection • Determine when new request arrives – Verify that previous request has been completely received – Request data size is > 0 • Key assumption is only one outstanding request is sent at a time by client, i. e. , requests are not pipelined • Reuse connections – Store each connection control information in a hash table keyed by real server address, once it is established. Content Switch Edward Chow

Quiz • Web server keeps the TCP connection alive, expecting the browser to return for images and in-line media files. • How many keep-alive connections are setup on IE 5 and Netscape 4. 7 for web page with many. jpg/. gif images? • Can these image requests be pipelined from client browser to web server? Content Switch Edward Chow

Multiple HTTP Requests from One TCP Connection NAT approach if g. s server 1 cs. jpg server 2. . . c uc Content Switch client Index. htm ro ck y. m id server 9 • A keep alive TCP connection may include multiple HTTP “GET” requests. • Content Switch examines each “GET” request and makes new routing decision. • Content Switch establishes another connection with a different server based on the routing decision. • Those HTTP responses from different servers need to be interleaved and seen by the user as if from the same server. • Solutions: In order delivery (buffer requirement); Out of order delivery (seq# tracking)? • Problems: Should we throw away earlier html requests if receive later requests? Content Switch Edward Chow

Multiple HTTP Requests from One TCP Connection g server 1 . jp s c c u Content Switch client roc server 2. . . cs ky. m . gi • • id server 9 f Can servers return documents directly to client in keep-alive session case? Can equivalent VS-Tunnel or VS-DR be implemented using Content Switch? Content Switch Edward Chow

Content Switch Rule Survey shows that existing switches support • rules in basic (condition action) or (action condition) form • some define condition as class, then specify the action in separate statement or command • simple single conditional term • command line interface (to facilitate incremental update? ) • Actions can include reject, forward, put in queue (for bandwidth control, scheduling) Content Switch Edward Chow

Content Switch Rule Design • Rule syntax generic to support all Intended features. • Use simple C if statement syntax rule: if (condition) { action } – Easy to read – Allow optimization using c compiler • Condition consists of multiple terms of – variable relational_operator value e. g. xml. purchase/total. Amount > 50000 smtp. to == “chow@cs. uccs. edu” cookie. name == “servlet 1” bitmatch(64, 8, 0 xff) == 64 # above mean TTL=64 idea from netfilter universal filter – suffix(variable, string) e. g. suffix(url, “gif”) – regex(variable, pattern) e. g. regex(url, “/purchase”) • Action consists of reject, forward(server| queue) load. Balance(server. Group, load. Balancing. Algorihtm) Content Switch Edward Chow

Efficient CS Rule Matching • Brute force, strict priority: Rules are executed in sequential manner. • Efficient Rule Matching Method: – Organize Rules so that rules can be skipped based on existing content types. – Utilize compiler optimization technique. Content Switch Edward Chow

Simple CS Rule Editor GUI Content Switch Edward Chow

Conflict Detection on Content Switching Rules • Detect conflicts among rules or rule set. • Absolute conflict type: r 1: if (xml. purchase/customer. Name == “CCL”) {route. To(r 1)} r 2: if (xml. purchase/customer. Name == “CCL”) {route. To(r 2)} • Potential conflict type: r 1: if (xml. purchase/total. Amount > 5000) {route. To(quick. Servers)} r 2: if (xml. purchase/total. Amount >20000) {route. To(super. Servers)} • Algorithm: Build tree with the same variable, check operator and value to see if they are the same or lead to potential conflict, compare actions to decide conflict type or duplication. • Developed conflict detection algorithm for rules with multiple term condition. Can be applied to policy-based rules conflict detection. • Editor can build these trees while a user enters rules and warns about conflict right away. Content Switch Edward Chow

XML Tag Value Extraction • A xml. Content. Extract() is built to extract the tag values of a list of unique tag sequences. • It is based on clark cooper’s expat 1. 0 xmlparser. • Its argument include the pointer to an XML document, the pointer to the array of strings (unique xml tag squences we follow the xsl selector syntax), and the number of sequences. • It return the list of a structure node, with the tag sequence, its attribute, and its value. • Currently, it supports one attribute and tag sequece needs to be unique. Content Switch Edward Chow

Status of UCCS ACSD Project • • A Linux-based LVS content switch called LCS was developed Sponsored by CCL/ITRI. Based on Linux-2. 2. 16 -3, current release LCS 02. ip_forward. c, ip_masq. c, ip_vs. c are modified to implement basic TCP delay binding. ip_cs. c are added for most of the content switching functions with http header extraction and xml content extraction. A simple Java-based rule. Edit program was created for rule editing and conflict detection. Rule translate program to convert the rule set into a Linux kernel module and allow dynamic replacement of rule without restarting the system. LCS is being ported to Intel IXP 1200 network processor. Content Switch Edward Chow

LCS Demo • We set up viva. uccs. edu as a content switch and wait and ace as two real servers. • URL Switching demo: http: //viva. uccs. edu/~lcs 1/ route to ace. uccs. edu http: //viva. uccs. edu/~lcs 2/ route to wait. uccs. edu • XML Web Switching (E-commerce applications) http: //archie. uccs. edu/~acsd/lcs/xmldemo. html When the 2 nd subtotal tag >=50000, route to ace. When the 2 nd subtotal tag <50000, route to wait. • Let us know if you have problem accessing them. My students may be working on LCS extension. Content Switch Edward Chow

LCS Rule Example R 4: if (atoi(rule_fields[1]. value) >= 50000) { return route_to("ace", NON_STICKY, saddr); } R 5: if ((atoi(rule_fields[1]. value) > 0) && (atoi(rule_fields[1]. value) < 50000)){ IP_RULE_MSG("serevr=waitn"); return route_to("wait", NON_STICKY, saddr); } R 10: if (strstr(url, "lcs 1") != NULL) { IP_RULE_MSG("server=acen"); return route_to("ace", NON_STICKY, saddr); } R 11: if(strstr(url, "lcs 2") != NULL){ IP_RULE_MSG("server=waitn"); return route_to("wait", NON_STICKY, saddr); } Content Switch Edward Chow

Related Load Balancing Research Results • Modified Apache status module to report – Total bytes to be transferred by child processes – Average document transfer speed • Modified LB-DNS to receive server status and bandwidth probing results. • LB-DNS returns IP-address of the best server based a weight contributed by both server load and bandwidth. • Modified Web. Stone benchmark to test the performance of load balancing web server clusters. Content Switch Edward Chow

Load balancing Systems Bandwidth Probe Results Statistics Gathering Daemon Modified Web Server 1 Server Delay Server Ranking /tmp/Stat. File Modified Web Server n Content Switch LBA: Modified DNS Edward Chow Request for Web pages

Connection Rate: LBA vs. Round. Robin Round robin only run once Content Switch Edward Chow

Conclusion • Content Delivery Network improves internet content retrieval • LVS provides a low cost layer 4 switching service for cluster. • Linux Content Switch with generic rules can be easily configured for wide-variety of value-added services: – Premium services – Load balancing/High Available server farm. – Firewall – Bandwidth control/Traffic shaping • Require efficient SW/HW architecture and rule matching algorithms to reduce processing overhead. • Content rule design/conflict detection are important and challenging. • TCP delay binding can be improved. Content Switch Edward Chow

References • • • http: //www. linuxvirtualserver. org/ http: //www. akamai. com/ http: //cs. uccs. edu/~chow/pub/contentsw/talk/contentswitching. ppt [Aron 2000] Aron, Mohit, “Differential and predictable Qo. S in web server systems”, Ph. D dissertation Rice University, Oct. 2000. [Zhang 97] Lixia Zhang, Sally Floyd, and Van Jacobson, “Adaptive Web Caching, ” April 25, 1997. http: //www-nrg. ee. lbl. gov/floyd/web. html [Esi 2001] Edge Side Includes, http: //www. esi. org/. [Chow 2001 a] C. Edward Chow and Indira Semwal, “Web Load Balancing Through More Accurate Server Report, ” Proceeding of PDCAT 2001, Taipei, Taiwan. [Chow 2001 b] C. Edward Chow, Ganesh Godavari, and Jianhua Xie, “Content Switch Rules and their Conflict Detection, ” Proceeding of PDCAT 2001, Taipei, Taiwan. [Chow 2001 c] C. Edward Chow and Weihong Wang, “The Design and Implementation of Linux LVS-based Content Switch”, Proceeding of PDCAT 2001, Taipei, Taiwan. [Aversa 2000] Luis Aversa and Azer Bestavros, “Load Balancing a Cluster of Web Servers: Using Distributed Packet Rewriting, ” Proceedings of IPCCC 2000. [Cao 98] Pei. Cao, Jin Zhang and Kevin Beach, “Active Cache: Caching Dynamic Contents on the Web” http: //www. cs. wisc. edu/~cao/papers/active-cache. ps Content Switch Edward Chow