UNIFYING DEVOPS MODEL Who generates the Generators stephanblackbuild

UNIFYING DEVOPS MODEL Who generates the Generators? stephan@blackbuild. com

ABOUTME. GROOVY About. Me. create { name 'Stephan Pauxberger' twitter '@black_build' email 'stephan@blackbuild' work { title 'Buildmanager' mode. Of. Operation FREELANCER clients ( 'Big companies', 'specific projects', 'usually green or brown field assignments' ) } } stephan@blackbuild. com

WALKING THE LINE stephan@blackbuild. com

THE PROBLEM It‘s groundhog day stephan@blackbuild. com

YOU ARE DOING GREAT! • You set up your environments via Puppet • You embrace CI/CD • Your Jenkins jobs are automatically generated • You have useful and up to date documentation in Confluence • Your environment is fully monitored via Icinga stephan@blackbuild. com

BUT WAIT! stephan@blackbuild. com

LET‘S ADD A NEW ENVIRONMENT • Create Puppet manifest(s) • Generate Jenkins Deploy Jobs / Pipelines • Request and setup a database • Request firewall changes via support ticket • Update system documentation • Setup Icinga • … • Rinse and repeat stephan@blackbuild. com

WE SHOULD HAVE KNOWN BETTER… Don‘t Repeat Yourself stephan@blackbuild. com Single Source of Truth

THE APPROACH Find yourself a model that can do both stephan@blackbuild. com

WHEN IN DEVELOPMENT, DO AS DEVELOPERS DO Consume Commit Build/ Test Store Consume stephan@blackbuild. com

WHAT WE NEED • A Schema • A Static Model • A Build Process • Tests • Consumers stephan@blackbuild. com

SCHEMA AND MODEL • XML • JSON • YAML • Ruby • Groovy DSL stephan@blackbuild. com

WHY GROOVY? • Modular • Type Safe (when activated) • IDE support (code completion, tests) • Loops and Conditions • Methods • Great Markup Support • Lots of Libraries • Great DSL, less Boilerplate stephan@blackbuild. com

LET‘S GET SOME HELP Supermodels don‘t just grow, they are made stephan@blackbuild. com

LET‘S BUILD A SCHEMA @DSL class Application { @Key String name @Default(field = "name") String title String description @Validate(message = "Need at least one environment") Map<String, Environment> environments } @DSL class Environment { @Key String name Stage stage Map<String, Server. Group> server. Groups } stephan@blackbuild. com

… AND A MODEL Application. create("My. App") { Wildfly. with. Template(version: '8. 0', configuration: MINIMAL) { environments { environment("Dev") { stage DEVELOPMENT server. Groups { iis("frontend") { hosts 'a. b. c. d', 'd. e. f. g' cluster. Ip '1. 2. 3. 4' deploy. User 'demo' } wildfly("application") { host "abc" } } } environment("Test") { … } } stephan@blackbuild. com

… PUT SUGAR ON TOP environments { [1. . 5]. each { count -> environment("DEV$count") { stage DEVELOPMENT server. Groups { iis("frontend") { hosts "a. b. c. $count" deploy. User 'demo' } wildfly("application") { host "abc$count" } } } stephan@blackbuild. com def env. Folder = new File("environments") environments(env. Folder. list. Files(). collect { Environment. create. From it })

A UNIT TEST def 'Higher wildfly version must enter lower stages first'() { given: Application my. App = Application. create. From(Configuration) when: List<Version> wildfly. Versions. Sorted. By. Stage = my. App. environments. sort { it. value. stage }. collect { it. value. wildfly. Version } then: 'versions already sorted' assert. Already. Sorted(wildfly. Versions. Sorted. By. Stage) } stephan@blackbuild. com

AND USE IT An on to the catwalk, we go stephan@blackbuild. com

A CONSUMER Docalot. run { gather { Application model = Config. create. From. Script(Configuration) main(Pojo. Collector) { source. Object model splitter { model. environments } } domain(Properties. Collector, 'status') { source. Folder new File("domains/data") } domain(Flag. Collector, 'disabled') { source. Folder new File("domains/data") } } convert { base. Dir new File("docs") directory(Single) { source "index" target ". " } directory(Splitted) { source "projects" } } publish { output(Confluence) { url "https: //confluence. comp. com" credentials. Id "abc-xyz" } } } stephan@blackbuild. com

JENKINS PIPELINES Config config = load. Model("Provider", Configuration) String env. Name = current. Build. parameters. target. Environment env = config. applications. my. App. environments[env. Name] stage 'Download' { with. Credentials(. . . ) { sh "curl $nexus. Url -u $credentials -o deploy. zip" } } stage "Unzip" { unzip(file: 'deploy. zip') } stage "deploy" { with. Class. Path(". ") { load. Script("Deploy. groovy"). run(env) } } stage "post deploy" {. . . } stephan@blackbuild. com

WHEN IN BUILDMANAGEMENT, DO AS DEVELOPERS (SHOULD) DO Create Jobs Commit Model Build/ Test Store Configure Pipeline Configure Deployment stephan@blackbuild. com

ALL THE GOOD THINGS • Unit-Tests • Mocking • Repeatability • Tagging / Branching • Continuous Integration / Inspection / Delivery stephan@blackbuild. com

CONSUMERS • Direct (native Java / Groovy) • Jenkins Job. DSL plugin (directly) • Jenkins Pipelines (via plugin) • Post Deploy Integration Tests (directly) • With Templates / Converters (via Docalot or Markup Templates) • Puppet Manifests • Support Tickets (for firewall rules, via Rest API of ticket system) • Documentation (via Docalot / Templates, pushed via CMS API) stephan@blackbuild. com

FURTHER DOWN THE RABBIT HOLE Let‘s put some clothes on stephan@blackbuild. com

ADVANCED TECHNIQUES: COMPOSITE MODELS stephan@blackbuild. com

COMPOSITE MODELS IN USE config. applications. each { app. Name, app -> div(class: 'app') { div(class: 'app. Header', app. Name) { app. environments. each { env. Name, env -> div(class: 'env') { div(class: 'title', env. title) div(class: 'description', env. description) div(class: 'version', env. version. installed. Version) if (env. status) div(class: env. status. Color, env. status. message) } } } stephan@blackbuild. com

ADVANCED TECHNIQUES: DECORATORS stephan@blackbuild. com

ADVANCED TECHNIQUES: CRYPTO PROVIDERS stephan@blackbuild. com

LET‘S WRAP IT UP And create our own collection stephan@blackbuild. com

MY FUTURE… RIGHT NOW FUTURE • Configure multiple Jenkins instances • Support tickets • Create necessary Jenkins Jobs (projects and deploy jobs) -> 2500 jobs • More documentation • (Maven, Gradle, C#, C++, Java. Script with Grunt / Webkit) • Configure Jenkins Pipelines -> All Pipelines have access to their model • Documentation (System Overviews) • Deployment stephan@blackbuild. com • Icinga • Puppet

…AND YOUR FUTURE • Take a look at the examples • Go out and do some modelling • Contribute your own examples and techniques • Live the life and tell the tale • Enjoy your new freedom as a buildmanagement rockstar stephan@blackbuild. com

USED TECHNOLOGIES • Jenkins (Pipeline, Job. DSL) • Klum Suite • Gradle, to build the model • Klum. AST • Nexus, to store models • Klum. Wrap (to be released as OSS) • Groovy • Klum. Lock (to be released as OSS) • Docalot • Custom Jenkins Plugins stephan@blackbuild. com

BUT, WHAT IS A ‚KLUM‘? • (K)onvenient Library for Unifying Models • Suite of libraries for Groovy • Type safe and IDE friendly models • Open Source • A way to create Buildmanagement‘s Next Top Model stephan@blackbuild. com

Q&A Stephan Pauxberger @black_build stephan@blackbuild. com Github: pauxus / blackbuild / klum-dsl

THANK YOU! stephan@blackbuild. com