Grid Torrent Framework A Highperformance Data Transfer and

Grid. Torrent Framework: A High-performance Data Transfer and Data Sharing Framework for Scientific Computing 1

Outline l Introduction l Motivation l PTCP l Grid. Torrent Framework l Test Results ¡ LAN Test Results ¡ WAN Test Results ¡ Overhead l Questions 2

Introduction l Today’s computational science is dataintensive ¡Large Hadron Collider (LHC) experiment at CERN generates petabytes of data l Accessibility, replication and creation of the data are made very easy by ¡Internet ¡Computational Grid 3

Motivation l The data is geographically distributed l Users are dispersed l Collaboration environments are required l Resources should be used in efficient and effective way ¡CPU ¡Storage ¡Network 4

PTCP l TCP has a performance problem over WANs l It was developed to solve the above problem by using striping technique 5

Grid. Torrent Framework l It is aimed ¡ to provide collaboration environment for dispersed users ¡ to make data transfer, management, and sharing easy via content manager ¡ to use systems resources efficiently and effectively by harnessing P 2 P (Bittorrent) network structure 6

Collaboration & Content Manager l The Content Manager allows users to publish or share their files with selected access control rights l The Collaboration Manager permits users to build a virtual l ACL enforce access sharing environment control rights for a by managing working given content groups or friend list l Task Manager handles the users’ task list 7

Grid. Torrent Client l It is responsible for ¡ initiating actual data publishing ¡ data sharing with other GTF clients ¡ ensuring secure environment for the above activities 8

WS-Tracker l WS-Tracker is a WS enabled server l It assists in the communication between peers (Grid. Torrent clients) l It delivers task lists which is generated by users to Grid. Torrent clients l It supplies ACL of each shared file to Grid. Torrent clients 9

Experimental Results l PTCP and Grid. Torrent Framework tests cases were conducted both in LAN and WAN type of computer networks l Server and client machines’ specification and location table l File size is 300 MB Name Specification Network Interface Institution Location A Intel(R) Quad-Core Xeon(TM) 4 x 2. 33 GHz CPU with 8 GB of RAM on Red Hat Enterprise Linux 4. 0 Broadcom Net. Xtreme II BCM 5708 1000 Base -T Indiana University Bloomington, IN B Sun Fire V 880 8 x 1. 2 GHz Ultra. SPARC III processors with 16 GB of RAM on Solaris 9. It has 6 x 72 GB 10 K rpm internal HD Gigabit Ethernet and 10/100 -Base. T Ethernet Indiana University Bloomington, IN C Dual Pentium III 731 MHz CPU with 512 MB of RAM on GNU/Linux 2. 6. 20 -1. 2316. fc 5 Gigabit Ethernet and 10/100 -Base. T Ethernet Florida State University Tallahassee, FL 10

LAN Test Setup l Server is located at Bloomington, IN l Client is at Indianapolis, IN l The number of parallel TCP streams between server and client has increased from 1 to 16 (PTCP) l The number of seeders increased from 1 to 16 (GTF) Client and server configuration for PTCP Grid. Torrent test case configuration for LAN test. Regular Java sockets are used for data transfer. 11

LAN Test Result l There is no significant improvement in bandwidth for both PTCP and GTF l Experimental data transfer (80 -100 Mbps) rate is much lower theoretical (1000 Mbps) and measured data transfer rate (857 Mbps) 12

WAN Test-I Setup l Server is located at Bloomington, IN l Client is at Tallahassee, FL l The number of parallel TCP streams between server and client has increased from 1 to 16 (PTCP) l The number of seeders increased from 1 to 16 (GTF) Client and server layout for PTCP test case Grid. Torrent test case configuration for wide area network test. Regular Java sockets are used for data transfer. 13

WAN Test-I Result l Bandwidth usage is vastly improved in both GTF and PTCP l PTCP’s bandwidth utilization rate has risen steadily until fifteen streams ¡ its peak value is 118 Mbps l GTF’s bandwidth utilization rate has risen steadily until thirteen streams l Grid. Torrent is performing better than PTCP when parallel streams number is less than five 14

WAN Test-II Setup l Server is located at Bloomington, IN l Client is at Tallahassee, FL l The number of parallel TCP streams between server and client has increased from 1 to 16 (PTCP) l Besides Java socket, other data transfer protocols can be exploited in Grid. Torrent client l The number of seeders increased from 1 to 16 (GTF) ¡ Four parallel TCP sockets were used between peer and seeders Client and server layout for PTCP test case Grid. Torrent test case configuration for wide area network test. Grid. Torrent client uses four parallel TCP sockets in each connection for 15 every source

WAN Test-II Result l Using parallel TCP with Bittorrent algorithm demonstrates much better bandwidth usage than standalone Grid. Torrent and PTCP l The maximum attained bandwidth is around 145 Mbps which is %23 higher than PTCP’s result 16

Overhead l Both parallel TCP and Grid. Torrent have overhead due to nature of multiple parallel connections l PTCP’s communication channel overhead time can be compared to Grid. Torrent WS-Tracker client’s overhead time varying between 300 and 600 milliseconds l Another overhead of Grid. Torrent is that control messages exchanged between peers to ensure strictly enforced to all participating peers ¡ The total size of overhead messages is between 148 KB to 169 KB 17

Questions 18

Thanks to All. 19