A Multicast Transport Driver for Globus XIO Karl
A Multicast Transport Driver for Globus XIO Karl Jeacle karl. jeacle@cl. cam. ac. uk
Rationale 4 Would like to achieve high-speed bulk data delivery to multiple sites 4 Multicasting would make sense 4 Existing multicast research has focused on sending to a large number of receivers 4 But Grid is an applied example where sending to a moderate number of receivers would be extremely beneficial ETNGRID 2005
Multicast availability 4 Deployment is a problem! 4 Protocols have been defined and implemented 4 Valid concerns about scalability; much FUD 4“chicken & egg” means limited coverage 4 Clouds of native multicast 4 But can’t reach all destinations via multicast 4 So applications abandon in favour of unicast 4 What if we could multicast when possible… 4…but fall back to unicast when necessary? ETNGRID 2005
Multicast TCP? 4 TCP 4“single reliable stream between two hosts” 4 Multicast TCP 4“multiple reliable streams from one to n hosts” 4 May seem a little odd, but there is precedent… 4 TCP-XMO – Liang & Cheriton 4 M-TCP – Mysore & Varghese 4 M/TCP – Visoottiviseth et al 4 PRMP – Barcellos et al 4 SCE – Talpade & Ammar ETNGRID 2005
ACK implosion ETNGRID 2005
Building Multicast TCP 4 Want to test multicast/unicast TCP approach 4 But new protocol == kernel change 4 Widespread test deployment difficult 4 Build new TCP-like engine 4 Encapsulate packets in UDP 4 Run in userspace 4 Performance is sacrificed… 4…but widespread testing now possible ETNGRID 2005
TCP/IP/UDP/IP Sending Application Receiving Application TCP IP IP If natively implemented UDP IP ETNGRID 2005 UDP test deployment IP
TCP engine 4 Where does initial TCP come from? 4 Could use BSD or Linux 4 Extracting from kernel could be problematic 4 More compact alternative 4 lw. IP = Lightweight IP 4 Small but fully RFC-compliant TCP/IP stack 4 lw. IP + multicast extensions = “TCP-XM” ETNGRID 2005
TCP-XM overview 4 Primarily aimed at push applications 4 Sender initiated – advance knowledge of receivers 4 Opens sessions to n destination hosts simultaneously 4 Unicast is used when multicast not available 4 Options headers used to exchange multicast info 4 API changes 4 Sender incorporates multiple destination and group addresses 4 Receiver requires no changes 4 TCP friendly, by definition ETNGRID 2005
TCP SYNACK Sender Receiver DATA ACK FIN ACK ETNGRID 2005
TCP-XM Receiver 1 Sender Receiver 2 Receiver 3 ETNGRID 2005
TCP-XM connection 4 Connection 4 User connects to multiple unicast destinations 4 Multiple TCP PCBs created 4 Independent 3 -way handshakes take place 4 SSM or random ASM group address allocated 4(if not specified in advance by user/application) 4 Group address sent as TCP option 4 Ability to multicast depends on TCP option ETNGRID 2005
TCP-XM transmission 4 Data transfer 4 Data replicated/enqueued on all send queues 4 PCB variables dictate transmission mode 4 Data packets are multicast (if possible) 4 Retransmissions are unicast 4 Auto fall back/forward to unicast/multicast 4 Close 4 Connections closed as per unicast TCP ETNGRID 2005
Fall back / fall forward 4 TCP-XM principle 4“Multicast if possible, unicast when necessary” 4 Initial transmission mode is group unicast 4 Ensures successful initial data transfer 4 Fall forward to multicast on positive feedback 4 Typically after ~75 K unicast data 4 Fall back to unicast on repeated mcast failure ETNGRID 2005
TCP-XM reception 4 Receiver 4 No API-level changes 4 Normal TCP listen 4 Auto-IGMP join on TCP-XM connect 4 Accepts data on both unicast/multicast ports 4 tcp_input() accepts: 4 packets addressed to existing unicast destination… 4…but now also those addressed to multicast group 4 Tracks how last n segs received (u/m) ETNGRID 2005
Grid multicast? 4 How can multicast be used in Grid environment? 4 TCP-XM is new multicast-capable protocol 4 Globus is de-facto Grid middleware 4 Would like TCP-XM support in Globus… ETNGRID 2005
Globus XIO 4 e. Xtensible Input Output library 4 Allows “i/o plugins” to Globus 4 API 4 Single POSIX-like API / set of semantics 4 Simple open/close/read/write API 4 Driver abstraction 4 Hides protocol details / Allows for extensibility 4 Stack of 1 transport & n transform drivers 4 Drivers can be selected at runtime ETNGRID 2005
XIO architecture ETNGRID 2005
XIO implementation ETNGRID 2005
XIO/XM driver specifics 4 Two important XIO data structures 1. Handle 4 Returned to user when XIO framework ready 4 Used for all open/close/read/write calls 4 lw. IP netconnection structure used 2. Attribute 4 Used to set XIO driver-specific parameters… 4 … and TCP-XM protocol-specific options 4 List of destination addresses ETNGRID 2005
XIO code example // init stack globus_xio_stack_init(&stack, NULL); // load drivers onto stack globus_xio_driver_load("tcpxm", &txdriver); globus_xio_stack_push_driver(stack, txdriver); // init attributes globus_xio_attr_init(&attr); globus_xio_attr_cntl(attr, txdriver, GLOBUS_XIO_TCPXM_SET_REMOTE_HOSTS, hosts, numhosts); // create handle globus_xio_handle_create(&handle, stack); // send data globus_xio_open(&handle, NULL, target); globus_xio_write(handle, "hellon", 6, 1, &nbytes, NULL); globus_xio_close(handle, NULL); ETNGRID 2005
One-to-many issues 4 Stack assumes one-to-one connections 4 XIO user interface requires modification 4 Needs support for one-to-many protocols 4 Minimal user API changes 4 Framework changes more significant 4 GSI is one-to-one 4 Authentication with peer on connection setup 4 But cannot authenticate with n peers 4 Need some form of “GSI-M” ETNGRID 2005
LAN speed ETNGRID 2005
LAN efficiency ETNGRID 2005
WAN speed ETNGRID 2005
WAN efficiency ETNGRID 2005
Driver availability 4 Multicast transport driver for Globus XIO 4 Requires Globus 3. 2 or later 4 Source code online 4 Sample client 4 Sample server 4 Driver installation instructions 4 http: //www. cl. cam. ac. uk/~kj 234/xio/ ETNGRID 2005
mcp & mcpd 4 Multicast file transfer application using TCP-XM 4‘mcpd &’ on servers 4‘mcp file host 1 host 2… host. N’ on client 4 http: //www. cl. cam. ac. uk/~kj 234/mcp/ 4 Full source code online 4 Free. BSD, Linux, Solaris ETNGRID 2005
All done! 4 Thanks for listening! 4 Questions? ETNGRID 2005
- Slides: 29