The Roadmap to New Releases Todd Tannenbaum Department
The Roadmap to New Releases Todd Tannenbaum Department of Computer Sciences University of Wisconsin-Madison http: //www. cs. wisc. edu/condor tannenba@cs. wisc. edu www. cs. wisc. edu/condor 1
Stable vs. Development Series › Much like the Linux kernel, Condor provides two different releases at any time: h. Stable series h. Development series › Allows Condor to be both a research project and a production-ready system www. cs. wisc. edu/condor 2
Stable series › Series number in version is even (e. g. 6. 2. 0) › Releases are heavily tested › Only bug fixes and ports to new platforms are added on a stable series www. cs. wisc. edu/condor 3
Stable series (cont. ) › A given stable release is always compatible with other releases from the same series › Recommended for production pools www. cs. wisc. edu/condor 4
Development Series › Series number in the version is odd (e. g. 6. 1. 17, 6. 3. 0) › New features and new technology are added frequently › Versions from the same development series are not always compatible with each other www. cs. wisc. edu/condor 5
Development Series (cont. ) › Releases are not as heavily tested › Generally not recommended for production pools h… unless new features are required h… unless we recommend otherwise : ^) www. cs. wisc. edu/condor 6
Where is Condor Today? › New stable version 6. 2. 0 recently › › released Previous development series cycle, ver 6. 1. *, was far too long This was our first stable/development series: we're hopefully learning. . . www. cs. wisc. edu/condor 7
New Ports in 6. 2. 0 › Full support (with checkpointing and remote system calls): h All current versions of Linux (x 86) • Kernel: 2. 2. * and 2. 0. * • C Library: glibc-2. [01] and libc-5 • NOTE: libc 5 support dropped h. Solaris 2. 7 and 2. 8(Sparc and x 86) h. Irix 6. 5 www. cs. wisc. edu/condor 8
New Ports in 6. 2. 0 (cont. ) › ”Clipped" support (no checkpointing or remote system calls, but all other functionality is available) h. Windows NT h. Alpha Linux www. cs. wisc. edu/condor 9
What is New in 6. 2. 0? › Personal Condor and Grid Support h. Flocking h. Proof-of-concept support for • Globus Job Universe • Globus Glide-In › Full, integrated support for Symmetric Multi-Processor (SMP) machines www. cs. wisc. edu/condor 10
What's New in 6. 2. 0? (cont. ) › PVM support › DAGMan (for managing inter-job › dependencies) Use of machine Class. Ad attributes in submit file h. Example: Executable = foo. $$(ARCH). $$(OPSYS) www. cs. wisc. edu/condor 11
What's New in 6. 2. 0? (cont. ) › Greatly expanded I/O support for STANDARD universe jobs h. Condor can automatically buffer I/O requests from jobs h. Compression/decompression on-the-fly! h. Users get much more information about the kinds of I/O their jobs are performing h. Users can "remap" files to alternate locations www. cs. wisc. edu/condor 12
What's New in 6. 2. 0? (cont. ) › Condor. Version and Condor. Platform strings included in all binaries and libraries h. Helps identify and avoid problems with having the wrong version installed h. Different parts of the Condor protocol automatically check for version incompatibilities www. cs. wisc. edu/condor 13
What's New in 6. 2. 0? (cont. ) › Better accounting h. Historical usage logging in collector • Use condor_stats • web interface called Condor. View h. Accountant stores usage information per user › Better control over user priorities h"Priority factors" www. cs. wisc. edu/condor 14
What's New in 6. 2. 0? (cont. ) › More powerful administration tools h. Setting configuration values remotely h. Querying daemons directly for status › Other misc new tools hcondor_findhost, condor_hold/release, condor_qedit, condor_userlog, condor_run, … › Lots of performance and bug fixes › A complete list is online in the manual (www. cs. wisc. edu/condor/manual) www. cs. wisc. edu/condor 15
The 6. 3 Development Series › Major focus areas for ver 6. 3. x Series h. Future-aware scheduling for dedicated resources • MPI support h. Security enhancements – Secure network communication » Kerberos, PKI (GSI/Open. SSL) – Limitations on executable, remote syscall sandbox h. Condor-G production version • Integrate Bypass technology h. Enhanced Windows support www. cs. wisc. edu/condor 16
What else will be added in 6. 3. *? › And also several easy-to-add, uservisible features h”Master agents" - helper programs spawned by the condor_master to aid in administration • Retrieving remote log, history and/or configuration files • Remote "top", "ps" and other monitoring functions www. cs. wisc. edu/condor 17
What will be added in 6. 3. *? (cont. ) › Solution for Condor clusters on h. Private network h. Behind a firewall › New condor_starter and condor_shadow for the VANILLA universe on Unix h"NT version" will be used for Unix, too h. Lots of new features, like transferring files automatically for "vanilla" jobs (no need for a shared filesystem) www. cs. wisc. edu/condor 18
What will be added in 6. 3. *? (cont. ) › condor_startd will enforce resource › limits dynamically (like RAM usage) More logic put into the condor_shadow to h. Detect temporary problems with a job's execution, put the job on hold, and notify the user h. Enforce job submitting user’s policy › Better propagation of error messages www. cs. wisc. edu/condor 19
The 6. 5. * Development Series › 6. 5. * will be for adding fundamentally new technology to Condor Being developed in parallel with 6. 3. * Will ultimately be v 7. 0 › › › Will be incompatible with previous versions of Condor www. cs. wisc. edu/condor 20
New Technology in 6. 5. * › New version of Class. Ads h. Conditionals !! • if/then/else h. Aggregates (lists, nested classads) h. Built-in functions • String operations, pattern matching, time operators, unit conversions h. Clean implementations in C++ and Java h. Class. Ad collections www. cs. wisc. edu/condor 21
Other changes for 6. 5. * › Re-write of the condor_schedd h. Performance enhancements and lowered resource requirements (particularly RAM) › Re-write of the checkpoint server h. NEST technology infusion h. Enhanced support for multiple servers h. Will meta-data along with checkpoint files www. cs. wisc. edu/condor 22
Planned Future Ports › Full support h. Intel Linux 2. 4 kernel (Red. Hat 7. x) › Clipped support h. Windows 2000 / Whistler www. cs. wisc. edu/condor 23
Possible Future Ports h. Full Support (I. e. checkpointing, etc) • Windows 2000 • Alpha Linux • DUX 5. x h. Clipped-support • {free, open, net}BSD • Mac. OS X • Power. PC Linux • HPUX 11. 0 • AIX 4. 2 www. cs. wisc. edu/condor 24
Thank you for coming to Paradyn/Condor Week! www. cs. wisc. edu/condor 25
- Slides: 25