Dev Ops Configuration Andres Villalobos IT Dev Ops

Dev. Ops Configuration Andres Villalobos IT Dev. Ops Engineer – LSST IT South NSF/DOE Joint Status Review August 13 th, 2018 #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 1

Dev. Ops Philosophy Continuous Development (CD) Automated Deployments/Testing Developers Dev. Ops IT Operations Continuous Integration (CI) #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 2

Dev. Ops Current Situation - Configuration Control Management. • Puppet - Monitoring System. • TICK stack (Telegraf, Influx. DB, Chronograf and Kapacitor) + Grafana - Naming convention Integration. • Using naming convention as tags on the monitoring systems and as classifier on Puppet. #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 3

Dev. Ops Implementation - Puppet as a Configuration Control Management. Vagrant as a Test Suite. Automatic deployment and testing of new software releases. Jenkins (TBD/TBC) as an automatic test suite. Telegraf as monitoring data collector Influx. DB as Monitoring data archive. Grafana for Monitoring Dashboards / Data consumption. Chronograf for IT Specialized Monitoring data management. Kapacitor as an Enhanced alerting system. #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 4

Dev. Ops Implementation - Puppet as a Configuration Control Management. • Documented on the following Confluence Page: https: //confluence. lsstcorp. org/display/SYSENG/Configuration+Co ntrol+Management • Currently being used to deploy VMs on controlled environments and some servers in pathfinder exercises. • Designed in conjunction with NCSA (Andy Loftus mainly). #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 5

Dev. Ops Implementation - Puppet #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 6

Dev. Ops Implementation - Vagrant as a Test Suite. • https: //github. com/avillalobos/lsst_devops • With that software, you can deploy your own Puppet Server and from there deploy VMs locally connected. • YAML VMs Definitions (credits to Andy Loftus. ) #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 7

Dev. Ops Implementation • YAML VMs Definitions (credits to Andy Loftus. ) https: //github. com/avillalobos/lsst_devops/blob/master/nodes/Vagrantconf. ya ml #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 8

Dev. Ops Implementation - Automatic deployment and testing of new software releases. • While using Vagrant, we can automatically deploy new machines using latest software of any kind, to check compatibility. #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 9

Dev. Ops Implementation Deploy % of Tests succeed Notify Build CI Server Fetch Test Software Control Version Repository Measure Check in changes Software Manager #lsst 2018 Software Developer #3 Software Developer #2 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 Software Developer #1 10