Overview on CI Use JJB Jenkins Job Builder

Overview on CI • Use JJB (Jenkins Job Builder) to manage Jenkins jobs. • Supports x 86_64, s 390 x build platforms. • Packer image to cache the build artifacts and load them in x 86_64 build nodes. • https: //github. com/hyperledger/ci-management • ~140 jobs spread across 16 repositories. • Verify • Merge • Daily • Weekly • Release jobs • Nightly jobs • Types of Tests CI executes • Unit-test, Functional, Smoke, Integration, Regression, Performance, Vulnerable tests, Code analysis tests • Documentation • RTD • SDK API docs • Publish images and binaries • Docker images • Fabric and fabric-ca binaries • SDK artifacts • npm modules • mvn jar files

Current Vs Future Current State v Jobs are created with freestyle project type v CI scripts in a separate repo is lacking developers attention or contribution v Dependency on CI team to update changes required in CI v Optimize build scripts v Documentation on best practices v Reduce build time v Improve build notifications Future State v Convert fabric, fabric-ca, fabric-sdk-node, fabric-samples, fabric-sdk-java projects to pipeline v Write re-use functions to optimize build scripts v Improve CI documentation v Cache third party software versions in packer image which dynamically loads on x 86_64 build node v Build notifications to Slack, Rocket. Chat, Email v Generate release notes from nightly release jobs and create a seamless release process

Freestyle vs Pipeline v Freestyle v Dependency on CI team to update the Job configuration v CI scripts in a separate repo is lacking developers attention or contribution v Complex to manage the code v CI change has to test separately before push v Pipeline v Keep CI scripts in code repository v Easy to manage the code v Write Job configuration in Jenkinsfile (Groovy script) v No separate testing required. v More build information on pipeline stages, easy and flexible view

Deep Dive on JJB Configuration

JJB –Freestyle Job Configuration template § Parametrize the variables Jobs macros

JJB – Pipeline Job Configuration - JJB with Pipeline project template Jenkinsfile

Daily/Nightly Job Flow

Nightly/Daily Job Release Jobs Nightly Build Job & Release Kickoff Jobs 1. 2. 1 -stable – release-1. 2 1. 3. 0 -stable – master branch Release Jobs fabric-release-job fabric-binaries-x 86_64 fabric-ca-release-job fabric-ca-binaries-x 86_64

Q 3 CI Roadmap Ø Refactor CI Jobs. (Epic#) Ø Convert freestyle to pipeline project types. – FAB-10067 Ø Keep CI scripts in code repo to avoid dependency from CI Ø Integrate build notifications (email, slack/Rocket. Chat) in CI jobs. FAB-2180 Ø Update existing CI & Release process documentation. FAB-8461 Ø https: //ci-docs. readthedocs. io/en/latest/ Ø Work on CI Ad-hoc requests. Ø Continue doing build monitoring and analyze the build failures and create bugs in JIRA Ø Explore more on Jenkins-plugins to provide more features on community builds. Ø Key FAB’s Ø FAB-11398 -- Docker Hub reference documentation on fabric, fabric-ca images Ø FABB-19 -- fabric-baseimage release strategy Ø FAB-11018 -- Update baseimage with s 390 x/debian: stretch in release-1. 0, release-1. 1 branches Ø FAB-2180 -- Build notifications to Rocket. Chat/Slack/Email Ø FAB-8461 – Improve CI documentation Ø Pipeline FAB’s (FAB-10067) Ø FAB-11389 Ø FAB-11392 Ø FAB-11261

CI tasks & Bugs created by CI team Dashboard: https: //jira. hyperledger. org/secure/Dashboard. jspa? select. Page. Id=10103

We are active on v Rocket. Chat Channels (#ci-pipeline, #fabric-ci, #infra-support) v Create JIRA under FABCI Project, choose right EPIC for the task or create one v Push changes to Gerrit ci-management master branch and `cr-ci` in reviewers

Fabric Build Flow

Fabric Verify Job Flow Developers fabric-verify-unittests-x 86_64 Submit fabric-verify-buildchecks-x 86_64 Patch set Verified +1/-1 Slave – x 86_64 Code Review +2 fabric-docs-buildx 86_64 ca ifi tio ns JJB (Freestyle) ot N ild Bu fabric-verifyintegration-testsx 86_64 fabric-smoke-testsx 86_64

Fabric Verify Build Process Verify Build Checks • Build Images & Binaries • Make basic-checks • Publish Images and binaries on commit tag • If succeed, post “Run Docs. Build” or “Run Smoke. Test” comments Integration. Test • Pull Images and Binaries from Nexus based on the commit# • make integration-test Docs. Build • Run tox. ini • Publish RTD output to nexus log server Unit. Test • Pull Images and Binaries from Nexus based on the commit# • make unit-tests Smoke. Test • Pull Images and Binaries from nexus based on the patch set commit number • Run Byfn Eyfn Tests • If succeed, post “Run Unit. Test” and “Run Integration. Test” comments

Fabric Merge Job Flow Maintainers fabric-merge-x 86_64 Merge fabric-merge-end-2 end-x 86_64 Patch set • Build Fabric • Run “make unittests” Slave – x 86_64 Bu ild JJB (Freestyle) N ot ifi ca tio ns RTD https: //readthedocs. org/projects/hyperledger-fabric/ • Build Images and Binaries • Run e 2 e_cli • Run sdk-node e 2 e tests • Run sdk-java e 2 e tests • Run byfn eyfn tsts

SDK Build Flow

Developers SDK (Chaincode-node, sdk-node) Verify Job Build Images, Binaries gulp test/gulp headless/gulp test-e 2 e Verified +1/-1 Sla 86 – x _64 ve Merge Job Build Images, Binaries gulp test/gulp headless/gulp test-e 2 e x 90 Publish API docs Release Job Build Images, Binaries s ion icat otif ld N Maintainers JJB (Freestyle) Bui Code Review +2 s 3 – e av Sl Publish unstable npm modules gulp test/gulp headless/gulp test-e 2 e After release tag is created Publish stable npm modules * - Publishing npm modules happens only from x 86_64