Partner Logo System Management Node Configuration Software Package

Partner Logo System Management: Node Configuration & Software Package Management German. Cancio@cern. ch German Cancio – WP 4 developments

System management Architecture u The pillars: n Central Configuration Database n Node Configuration Management n Base system installation n Software Package Management n Monitoring German Cancio – WP 4 developments - n° 2

Node Configuration Management German Cancio – WP 4 developments - n° 3

Node Configuration Management Component GUI Access API High Level API HLDL Low Level API DBM Client PAN Notification + Transfer XML Server German Cancio – WP 4 developments - n° 4

Node Configuration Management (NCM) u Client software running on the node which takes care of “implementing” what is in the configuration profile u Sits on top of the low-level Config access library (NVA-API) u Modules: n “Components” n Component support libraries n Framework German Cancio – WP 4 developments - n° 5

NCM: Components u “Components” (like SUE “features” or LCFG ‘objects’) are responsible for updating local config files, and notifying services if needed u Components register their interest in configuration entries or subtrees, and get invoked in case of changes u Components n n Usually, this implies regenerating and/or updating local config files (eg. /etc/sshd_config) One method only: Configure() u Use n n n do only configure the system standard system facilities (Sys. V scripts) for managing services Reuse the standard facilities: most services come with a Sys. V script. Components can notify services using Sys. V scripts when their configuration changes. Components are kept small&simple German Cancio – WP 4 developments - n° 6

Component example sub Configure { my ($self) = @_; # access configuration information my $config=NVA: : Config->new(); my $arch=$config->get. Value('/system/architecture’); # low-level API $self->Fail (“not supported") unless ($arch eq ‘i 386’); # (re)generate and/or update local config file(s) open (myconfig, ’/etc/myconfig’); … # notify affected (Sys. V) services if required if ($changed) { system(‘/sbin/service myservice reload’); … } } German Cancio – WP 4 developments - n° 7

Existing component taxonomy u Components components u Basic n can be classified into basic, service-specific and Grid components: Manage basic system configurations eg. network, NFS, printers, security, time… u Service specific components: For batch nodes, servers (provided by service managers) n Eg. LSF, PBS, Castor, accounting, root mail, … u Grid n components (provided by Grid middleware WP’s) Eg. Globus, GDMP, LCAS, Resource Broker… u Existing components are available both from SUE and LCFG n SUE features: Basic and Service specific n LCFG components: Basic and Grid components è Need complete classification (which components to port, which ones to rewrite) è Functionality -> component German Cancio – WP 4 developments - n° 8

NCM: Component support libraries for recurring tasks: u Logging and error reporting u Template u SUE n processor (for fast config file generation) sysmgt libraries Eg. check/edit files, system information, regexps, crontab, (x)inetd… u Monitoring integration German Cancio – WP 4 developments - n° 9

NCM: framework A light weight framework (cdisp) glues the components to the Configuration Client. Overall functionality: 1. Register which components are interested in which configuration entries/subtrees 2. Upon a config update, look up the components which need to be invoked 3. Ordering of component invocations according to dependencies 4. Invocation of components German Cancio – WP 4 developments - n° 10

NCM Architecture XML server client DBM Cache registration & notification NVA API CCConfig cdisp Invocation Configure() Components “Low-level” API “High-level” API Component libs SUE sysmgt Logging Template processor Monitoring interface Node Config Client Node Config Mgmt German Cancio – WP 4 developments - n° 11

Base System Installation German Cancio – WP 4 developments - n° 12

Base System Installation u Nodes n are installed using the default system installer. Use standard installation infrastructure (DHCP, install servers, repositories) u Configuration n n information stored in the CDB, generate ks or js files A NCM component generates ks/js files out of node profiles. Template ks/js files used for substituting partition, network, timezone, and other miscellaneous information Keep the installation simple: installation server should not need any tweaking excepting adding the NCM required packages. Can be used in combination with existing environment, eg. AIMS for updating NFS & DHCP servers Site-specific databases should not directly be accessed, only CDB (instead, update CDB with LANDB and CCDB information) u Hook into Node Configuration Management afterwards (‘post-install’) n Start the NCM which downloads the latest node profile n It will bring the machine to its state as reflected in the CDB. n (Monitoring can be activated at this moment). German Cancio – WP 4 developments - n° 13

Software Package Management German Cancio – WP 4 developments - n° 14

Software Distribution Architecture u A packaging tool takes care of installing/deinstalling/upgrading packages on a computer node and keeps an inventory of currently installed packages. u The packages themselves are stored on a managed Software Repository accessible via multiple protocols (eg. HTTP, FTP, shared file system, . . ) u Information on which packages are to be deployed on which nodes, and which packages are available on which repositories, is kept in the fabric-wide Configuration Database. u A 'glue' application (running on the target nodes) computes the necessary package upgrade operations and invokes the packaging tool. u (The SW generation and packaging process is outside the scope. ) German Cancio – WP 4 developments - n° 15

Software Package Manager (SPM) u The SPM is the glue application. Functionality: 1. Compares the packages currently installed on the local node with the packages listed in the configuration 2. Computes the necessary install/deinstall/upgrade operations 3. Invokes the packager with the right operation transaction set u The n n SPM is driven via a local configuration file For batch/servers: A component generates/maintains this cf file out of CDB information For desktops: Possible to write a GUI for locally editing the cf file u The SPM core is independent of which packaging format is used. German Cancio – WP 4 developments - n° 16

Software Package Manager (II) u Packager: the standard system packaging format is used (rpm for Linux, pkg for Solaris) n • rpmt (for rpm ‘transactional’) used for transactions handling Packager (rpmt) functionality: 1. Read operations (transaction) 2. downloads new packages from Repository 3. orders the transaction operations taking into account dependency information 4. Executes the operations (installs/removes/upgrades) German Cancio – WP 4 developments - n° 17

SPM Architecture Central Config DB Desktops Component GUI Local Config file “desired” configuration Packages (RPM, pkg) SPM Installed pkgs Transaction set Repository packages rpmt HTTP(S), NFS, FTP Package files external SPM RPM db filesystem German Cancio – WP 4 developments - n° 18

Software Package Management u Package n config flags: Ignore locally installed packages (useful for desktops) s Local exception list, or backup of previous configuration n Reboot when package is upgraded n Do not upgrade package when node is in production (eg. runlevel X) n Standard RPM flags (no dependencies, force, no pre/post installs, etc. . ) s Issue: per-package flags not supported by standard RPM libraries! German Cancio – WP 4 developments - n° 19

SPM and the CDB u SW n n packages are modelised in the Global Schema as follows: Repository configuration: List of available repositories, repository directories, and packages Node configuration: list of used repositories, and list of selected packages. u (see next slides) u The CDB template inclusion mechanism allows to define multiple default profiles which can then be refined n Production-packages-rh 72. tpl -> lxplus 7. tpl -> lxplus 043 n More flexibility than current ASIS profiles (Certified, In. Production) German Cancio – WP 4 developments - n° 20

SPM and Global Schema (I) SW repository structure (maintained by repository managers): /sw/known_repositories/Arep/url = (host, protocol, prefix dir) /owner = /extras = /directories/dir_name_X/path = (asis) /platform = (i 386_rh 61) /packages/pck_a/name = (kernel) /version = (2. 4. 9) /release = 31. 1. cern /architecture = (i 686) /dir_name_Y /path = (sun_system) /platform = (sun 4_58) /packages/pck_b/name = (SUNWcsd) /version = 11. 7. 0 /release = 1998. 09. 01. 04. 16 /architecture = (? ) German Cancio – WP 4 developments - n° 21

SPM and Global Schema (II) Node information (maintained by node administrator) /sw/used_repositories/0/rep_name_A = /1/rep_name_B = /sw/packages/package_name/version = /arch = /flags = German Cancio – WP 4 developments - n° 22

Issues u Multiplatform n support How to manage components for multiple platforms (now: RH 62, RH 72, in the future: RH X. Y, Solaris? ) n Is the global schema adequate for Solaris? n Is the SPM concept adequate for Solaris? u Backwards n compatibility: LCFG, SUE Probably not high priority. u Reusing existing software n SUE libraries and features n LCFG component libraries and components n ASIS libs u Dependencies between software package management and node configuration management German Cancio – WP 4 developments - n° 23