Testbed configuration and management CERN openlab Summer Student

Testbed configuration and management CERN openlab Summer Student Programme 05 August 2011 Georgi Zlatkov Openlab Summer Student georgi. zlatkov at cern. ch

Overview Why Puppet? Introduction to Puppet in a nutshell Puppet's modules Puppet and CERN Additional tools References CERN openlab Presentation 2011 2

Why Puppet? The Challenge: Manage a large number of systems At least four “coffee” breaks per day Watch You. Tube Go home early CERN openlab Presentation 2011 3

Why Puppet? Scenario: Massive hardware failure of systems. New servers are installed but need to be configured. CERN openlab Presentation 2011 4

Why Puppet? The Manual Way: Do everything by hand 5 servers OK 100 servers NO CERN openlab Presentation 2011 5

Why Puppet? The Puppet Way: Puppet package is pre-installed on the site platform Change hostnames Puppet brings each system to the desired end state CERN openlab Presentation 2011 6

Introduction to Puppet What Is Puppet? In author's own words: “Puppet is a system for automating system administration tasks. ” An open-source project written in Ruby and released under Apache 2. 0 (after v 2. 7. 0) A declarative language for describing system configuration Cross-platform – Linux, Unix, Windows CERN openlab Presentation 2011 7

Introduction to Puppet How Puppet Works? A client/server based application The server is called “puppetmaster” The client is called “node” or “puppet” SSL identity verification Puppet enforce new configurations to the system Idempotency Detect current state of the system Change if only needed Run on regular basis CERN openlab Presentation 2011 8

Introduction to Puppet CERN openlab Presentation 2011 9

Introduction to Puppet Installation and configuration of Puppet If installed with default package manager you will automatically get prerequisites Puppet and Facter from source tar balls (about Facter later) During installation the “/etc/puppet” directory will be created. Puppet's configuration files on the server puppet. conf (used by client's puppet) auth. conf (used by puppetmaster) fileserver. conf (used by puppetmaster) CERN openlab Presentation 2011 by puppetmaster) autosign. conf (used 10

Introduction to Puppet SSL identity verification Certificate request from the client # puppet agent --test --server. example. com --waitforcert 10 See requests # puppetca --list Three possible ways to sign certificate Automatically sign all requests (autosign. conf) Sign only specific requests (autosign. conf) Manually sign requests (puppetca --sign) CERN openlab Presentation 2011 11

Puppet in a nutshell Puppet's language Resources (file, service, user) – describe configurations using attributes(ensure, mode, source) (Parameterized) Class – a named collection of resources Definition – abstract description of class with the idea to be used multiple times Recipe/Manifest – sample puppet code(*. pp) Variables, Arrays, Hashes, Selectors and If/else, case statements CERN openlab Presentation 2011 12

Puppet in a nutshell Example: package { "ntp": ensure => installed, } service { "ntp": ensure => running, require => Package[“ntp”], } $file = “/etc/sample” file { $file: ensure => present, source => “puppet: ///example/$hostname”, } CERN openlab Presentation 2011 13

Puppet in a nutshell Facter Holds a stash of predefined variables Custom facts could be added Prefixed with “$” or “$: : ” (explicit) architecture => amd 64 domain => example. com facterversion => 1. 5. 7 fqdn => client. example. com hardwareisa => unknown hardwaremodel => x 86_64 hostname => client CERN openlab Presentation 2011 14

Puppet in a nutshell Templates Flat files which contains Embedded Ruby(ERB) variables Version=<%= pkg_version %> And called in a class content => template(“version-temp. erb”) Variable declaration on in the same class or in node declaration $pkg_version = “ 1. 2. 3” CERN openlab Presentation 2011 15

Puppet's modules Modules Stored at “/etc/puppet/modules” by default Collection of classes, definitions and resources Puppet search automatically for modules Simplifies system administration CERN openlab Presentation 2011 16

Puppet's modules Module Structure MODULE_PATH/ └──downcased_module_name/ ├──files/ ├──manifests/ │ ├──init. pp ├──templates/ └──README CERN openlab Presentation 2011 17

Puppet's modules Modules Repositories Puppet Forge http: //forge. puppetlabs. com/ David Schmitt's repository http: //git. black. co. at/ Ricardo Brito Da Rocha's repository https: //github. com/rochaporto/repositories Eshao https: //github. com/eshao/puppet Example 42 http: //www. example 42. com/ CERN openlab Presentation 2011 18

Puppet and CERN Scalability Flexibility Reliability Cross-Platform Easy to use Rapid Growth CERN openlab Presentation 2011 19

Additional tools Puppet Dashboard Ralsh Geppeto Puppet Module Tool Puppi CERN openlab Presentation 2011 20

Puppet and CERN Future Plans Scale up manage the IT-GT testbeds Work on 'real scalability' of puppetmasterd moving from Webrick to Mongrel Work with the “Agile. Infrastructure” internal group to work on best practices fro provisioning CERN openlab Presentation 2011 21

References Puppet. Labs http: //www. puppetlabs. com/ Puppet Cook. Book http: //www. puppetcookbook. com #puppet on chat. freenode. org Puppet Google Group http: //groups. google. com/group/puppet-users CERN's Puppet Users – puppet-users@cern. ch CERN openlab Presentation 2011 22