CCB The Condor Connection Broker Dan Bradley danhep
CCB The Condor Connection Broker Dan Bradley dan@hep. wisc. edu Condor Project CS and Physics Departments University of Wisconsin-Madison
Condor Connections Central Manager Job Submit Point adv e s i ert ne got iate you ’ve Execute Node bee adv nm erti atc hed run this job transfer files www. cs. wisc. edu/Condor se
Execute Node Unreachable Central Manager Execute node is behind a firewall or is NATed. Job Submit Point adv e s i ert ne got iate you ’ve adv bee no go! run this job Execute Node nm erti atc hed transfer files www. cs. wisc. edu/Condor se
Submit Node Unreachable Central Manager Submit node is behind a firewall or is NATed. Job Submit Point adv e s i ert ne got iate you ’ve adv bee no go! run this job Execute Node nm erti atc hed transfer files www. cs. wisc. edu/Condor se
Common Scenarios › Why cross private network boundaries? h. Flocking h. Multi-site Condor pool h. Glidein www. cs. wisc. edu/Condor
CCB: Condor Connection Broker › Condor wants two-way connectivity › With CCB, one-way is good enough Execute Node Job Submit Point run this job I want to connect to the submit node CCB_ADDRESS=ccb. host. name transfer files reversed connection www. cs. wisc. edu/Condor
CCB: Condor Connection Broker › Works in the mirror case too Job Submit Point Execute Node I want to connect to the execute node run this job reversed connection transfer files CCB_ADDRESS=ccb. host. name www. cs. wisc. edu/Condor
Limitations of CCB 1. Doesn’t help with standard universe 2. Requires one-way connectivity Execute Node Job Submit Point no go! CCB_ADDRESS=ccb 1. host GCB or VPN can help CCB_ADDRESS=ccb 2. host www. cs. wisc. edu/Condor
Connecting to CCB Server CCB server must be reachable by both sides. Job Submit Point t c e nn o c B CC D ation A RE horiz aut l l eve CC Bl Execute Node iste DA E n aut MO hor N iza tion lev el CCB_ADDRESS=ccb. host www. cs. wisc. edu/Condor
CCB Server Behind Firewall CCB Server Must have an open port to connect to CCB Job Submit Point o c B CC t c e nn Execute Node CC Bl iste n open port here (default 9618) CCB_ADDRESS=ccb. host www. cs. wisc. edu/Condor
Security on Reversed Connection CCB Server Client and server security policies are enforced in logical direction Job Submit Point t c e nn o c B CC Execute Node CC Bl iste n run this job reversed connection daemon-side client-side CCB_ADDRESS=ccb. host www. cs. wisc. edu/Condor
GCB: Generic Connection Broker › GCB: Condor 6. 9. 13 h. Clever: mostly invisible to Condor code h. However, this makes some things difficult! › CCB: Condor 7. 3. 0 h. Inspired by GCB h. More tightly integrated into Condor h. Not a complete replacement www. cs. wisc. edu/Condor
Why CCB? › Secure hsupports full Condor security set › Robust hsupports reconnect, failover › Portable hsupports all Condor platforms, not just Linux www. cs. wisc. edu/Condor
Why CCB? › Dynamic h CCB clients and servers configurable without restart › Informative log messages h Connection errors are propagated h Names and local IP addresses reported (GCB replaces local IP with broker IP) › Easy to configure h automatically switches UDP to TCP in Condor protocols h CCB server only needs one open port www. cs. wisc. edu/Condor
Configuring CCB › The Server: h The collector is a CCB server h UNIX: MAX_FILE_DESCRIPTORS=10000 › The Client: 1. CCB_ADDRESS = $(COLLECTOR_HOST) 2. PRIVATE_NETWORK_NAME = your. domain (optimization: hosts with same network name don’t use CCB to connect to each other) www. cs. wisc. edu/Condor
Tests of CCB › Igor Sfiligoi’s Cross-Atlantic Mega Condor Glidein Test Pool for CMS hone machine with 70 CCB collectors hexecute nodes in private networks h. GSI authentication h 100, 000 registered Condor daemons h 200, 000 jobs/day with one schedd www. cs. wisc. edu/Condor
Summary › CCB makes Condor work if h You have one-way connectivity Fine Print: h. And using Condor 7. 3+ h. And the private side sets CCB_ADDRESS h. And the private side is authorized at the DAEMON authorization level by CCB h. And the public side can connect to CCB h. And the public side is authorized at the READ authorization level by CCB h. And not using “standard universe” www. cs. wisc. edu/Condor
- Slides: 17