HTTPTester A web appliance testing tool Kiran Komaravolu

HTTP-Tester- A web appliance testing tool Kiran Komaravolu Dept of Computer Science and Engg Lehigh University

What is the need to test Web Appliances n Need to use best components. n Compare performance of different web devices. Verify if a device is performs as per specifications. n Choosing between different service providers. n

Related Work The Ethereal network analyser (www. ethereal. com). n TCPDUMP/PCAP (www. tcpdump. org). n Httpflow (Grevnin and Davison) n Simultaneous proxy evaluation. (Davison). n Medusa Proxy. (Koletsou and Voelker). n

Http-Tester Replays captured requests to an existing server to one that is being evaluated. n No active deployment needed. n Does not affect existing services. n Can be used to evaluate any Web appliance running HTTP. n

HTTP-Tester n Topology setup

HTTP-Tester n Components n The Sniffer. n Logs all packets onto a packet database. n Request Replicator. n Replays all requests to the “tested device” n The Post Processor. n Reads the packet logs, extracts HTTP requests and responses and correlates them.

HTTP n HTTP Message Structure n A command line. (GET, POST, OK etc. ) n GET http: //www. google. com / HTTP 1. 0 n Zero or more header lines. n Content-Type: text/html; charset=iso-8859 -1 Control Return + Line Feed. n Optional Message Body n n Response message

HTTP Transactions. n HTTP 1. 0 No persistent connections. n A new connection is opened up for every request made, and is closed down after response sent. n n HTTP 1. 1 Supports Persistent Connections. n Client may make a series of requests on a connection. Server returns responses in same order. n Supports chunk encoding. n

The Sniffer Blindly captures all relevant network packets and logs them. n The pcap packet capture library has been used to read packets off the wire. n Berkeley DB package has been used to log the packets into a ‘db’ database. n n The job of efficient memory and disk usage has been delegated to DB.

Request Replicator Replays all requests flowing into original device onto the tested device. n Works in parallel with the sniffer module. n Looks for TCP SYN packets and determines the beginning of a request flow. And opens up a new connection to the second device. n The payload every next packet sent to on this connection is sent to the second device. n

Request Replicator. This way the second server sees the same workload as the first one. n The replies sent by the server are ignored. They are flushed out as the socket buffers overflow. The sniffer module though logs the TCP packets. n

Post Processing. Extracts HTTP requests/responses from the packet logs. n Every request is mapped onto its response. The same request response pair on the second connection is also identified. n Httpflow routines used to extract HTTP requests from packet stream. n

Identifying Requests. We know which way requests flow. n Assumption: First TCP packet of every request has one of GET, HEAD, OPTIONS, DELETE, POST, PUT. n End of headers is determined by a CRLF. n content-length field is specified in case of PUT/POST requests. n

Identifying responses Response headers extrated by looking for a CRLF. n End of Response message determined in three ways: n Server sends a FIN or RST packet. n Response headers contain a “content-length” field. n Response is chunk encoded. Each chunk has its own length. Zero length chunk indicates end of stream. n

Correlating requests and responses n HTTP 1. 0