Dev Ops Configuration Andres Villalobos IT Dev Ops
Dev. Ops Configuration Andres Villalobos IT Dev. Ops Engineer – LSST IT South NSF/DOE Joint Status Review August 13 th, 2018 #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 1
Dev. Ops Philosophy Continuous Development (CD) Automated Deployments/Testing Developers Dev. Ops IT Operations Continuous Integration (CI) #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 2
Dev. Ops Philosophy - IT Infrastructure Service Provider - Relationship between Service/Software providers and IT Infrastructure Service. - IT Hardware Procurement. - IT Hardware standard defined on LSE-309 must be followed. - IT will take care of deploy the service in the agreed method. - IT will not operate each software/system deploy, will just give operations/infrastructure support. #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 3
Dev. Ops Current Situation - Configuration Control Management. • Puppet - Monitoring System. • TICK stack (Telegraf, Influx. DB, Chronograf and Kapacitor) + Grafana - Naming convention Integration. • Using naming convention as tags on the monitoring systems and as classifier on Puppet. #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 4
Dev. Ops Implementation - Puppet as a Configuration Control Management. Vagrant as a Test Suite. Automatic deployment and testing of new software releases. Jenkins (TBD/TBC) as an automatic test suite. Telegraf as monitoring data collector Influx. DB as Monitoring data archive. Grafana for Monitoring Dashboards / Data consumption. Chronograf for IT Specialized Monitoring data management. Kapacitor as an Enhanced alerting system. #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 5
Dev. Ops Implementation - Puppet as a Configuration Control Management. • Documented on the following Confluence Page: https: //confluence. lsstcorp. org/display/SYSENG/Configuration+Co ntrol+Management • Currently being used to deploy VMs on controlled environments and some servers in pathfinder exercises. • Designed in conjunction with NCSA (Andy Loftus mainly). #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 6
Dev. Ops Implementation - Vagrant as a Test Suite. • https: //github. com/avillalobos/lsst_devops • With that software, you can deploy your own Puppet Server and from there deploy VMs locally connected. • YAML VMs Definitions (credits to Andy Loftus. ) #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 7
Dev. Ops Implementation • YAML VMs Definitions (credits to Andy Loftus. ) https: //github. com/avillalobos/lsst_devops/blob/master/nodes/Vagrantconf. ya ml #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 8
Dev. Ops Implementation - Automatic deployment and testing of new software releases. • While using Vagrant, we can automatically deploy new machines using latest software of any kind, to check compatibility. #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 9
Dev. Ops Implementation Deploy % of Tests succeed Notify Build CI Server Fetch Test Software Control Version Repository Measure Check in changes Software Manager #lsst 2018 Software Developer #3 Software Developer #2 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 Software Developer #1 10
Dev. Ops Implementation - Jenkins (TBD/TBC) as an automatic test suite. • Jenkins is already being used by DM, we can leverage a similar infrastructure to deploy VMs using puppet. That way you can test new software releases with the latest production configuration to see if can be integrated. #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 11
Dev. Ops Implementation - Telegraf as monitoring data collector Influx. DB as Monitoring data archive. Grafana for Monitoring Dashboards / Data consumption. Chronograf for IT Specialized Monitoring data management. Kapacitor as an Enhanced alerting system. #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 12
Monitoring System / Alerting System #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 13
Grafana Dashboards #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 14
Grafana Dashboards #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 15
Grafana Dashboards #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 16
Chronograf data Explorer #lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 17
#lsst 2018 LSST Project and Community Workshop 2018 • Tucson • August 13 - 17 18
- Slides: 18