Department of the Air Force Integrity Service Excellence

Department of the Air Force Integrity - Service - Excellence Do. D Enterprise Dev. Sec. Ops Initiative & Platform One Mr. Nicolas Chaillan Chief Software Officer, U. S. Air Force Co-Lead, Do. D Enterprise Dev. Sec. Ops Initiative V 5. 5 – UNCLASSIFIED 1

CSO Website – Continuously Updated! n Want to find information about the Dev. Sec. Ops initiative and the CSO? n Our latest documents/videos: https: //software. af. mil/dsop/documents/ Our latest training videos/content at: https: //software. af. mil/training/ n Platform One Services: https: //software. af. mil/dsop/services/ n More information about : n Platform One On Boarding: https: //software. af. mil/team/platformone/ n Cloud One: https: //software. af. mil/team/cloud-one/ n Repo One: https: //repo 1. dsop. io n Iron Bank: https: //ironbank. dsop. io n Registry One: https: //registry 1. dsop. io n Dev. Star: https: //software. af. mil/dsop-devstar/ n Our Events/News: https: //software. af. mil/events/ n. mil only IL 4 Chat (Matter. Most): https: //chat. collab. cdl. af. mil/ n Integrity - Service - Excellence 2

What is the Do. D Enterprise Dev. Sec. Ops Initiative? n Joint Program with OUSD(A&S), Do. D CIO, U. S. Air Force, DISA and the Military Services. n Bringing Enterprise IT Capabilities with Cloud One and Platform One – Cloud and Dev. Sec. Ops as Managed Services capabilities, on-boarding and support! Brings timeliness, modularity and enables reuse. n Technology: n Avoid vendor lock-in at the Infrastructure and Platform Layer by leveraging FOSS with Kubernetes and OCI containers (reusable lego blocks), n Created Iron Bank, the Do. D Centralized Artifacts Repository (DCAR) of hardened and centrally accredited containers: selecting, certifying, and securing best of breed development tools and software capabilities (over 170+ containers) https: //repo 1. dsop. io/dsop/ and https: //ironbank. dsop. io n Baked-in Zero Trust Security with our Sidecar Container Security Stack (SCSS) leveraging behavior detection, zero trust down to the container/function level. n Leveraging a Scalable Microservices Architecture with Service Mesh (Istio), baked-in security n Leveraging KNative to avoid lock-in to Cloud provider Serverless stacks and Kubeflow for AI/ML/Deep Learning n Standardizing metrics and define acceptable thresholds for Do. D-wide continuous Authority to Operate n Massive Scale Training with Self Learning Capabilities (train over 100 K people within a year) and bring state of the art Dev. Sec. Ops curriculum n Creating new Agile contracting language to enable and incentivize the use of Dev. Sec. Ops: https: //www. dau. edu/cop/it/DAU%20 Sponsored%20 Documents/Contracting%20 Considerations%20 for%20 Agile%20 Solutions%20 v 1. 0. pdf Integrity - Service - Excellence 3

Continuous Monitoring Application Layer Brings baked-in security and Microservices architecture enablement Fully containerized, leverages Do. D approved containers from DCAR Development Team selects tools from 172 approved containers or custom containers YOU Development Teams can build software/microservices leveraging hardened containers Service Mesh Layer Continuous Integration / Continuous Delivery (CI/CD) Layer Environment Agnostic Cloud One Preferred for unclassified (IL 2, IL 4, IL 5) Or SC 2 S/FENCES Or on-premise/embedded systems/classified environments Integrity - Service - Excellence Platform Layer Infrastructure Layer Cloud One CNCF compliant Kubernetes (K 8 S) Includes Site Reliability Engineers (SREs) etc. Development Team selects between approved K 8 S stacks Platform One Leverages the Sidecar Container Security Stack Understanding the Dev. Sec. Ops Layers

Software Ecosystem Multiple Innovation Hubs – One Platform Software Factories 43 PMOs & PEOs Across Services Ventures & Non. Traditional Startups Science & Technology Defense Industrial Base AF Ventures Do. D-wide Enterprise Services Integrity - Service - Excellence Other Agencies

Software Ecosystem Multiple Innovation Hubs, One Platform One N 2 X Pathfinder Colorado Springs, CO l l l JAIC Army Cyber AEGIS F-35 ABMS Thunder CAMP Space CAMP Colorado Springs, CO Oklahoma City, OK Colorado Springs, CO l NORAD l Rogue Blue Space Force l 76 th SWEG Omaha, NE l Kessel Run STRATCOM Boston, MA l l l Soni. Kube Hill AFB, UT l Ski CAMP Hill AFB, UT F-16 l Conjure Scott AFB, IL GBSD l 375 th Kobayashi Maru Blue Sky Los Angeles, CA l AOC F-35 ABMS Warner Robins, GA SMC l Corsair Ranch BESPIN Level. UP Tuscon, AZ Oahu, HI l l Montgomery, AL San Antonio, TX TRON Unified Platform 402 nd SWEG l PEO BES PACOM Integrity - Service - Excellence 6

Platform One Services n Full details at: https: //software. af. mil/dsop/services/ n Repo One – Do. D Centralized Container Source Code Repository (DCCSCR) Container source code, Infrastructure as Code, K 8 S distributions, etc. n Repo One is the central repository for the source code to create hardened and evaluated containers for the Department of Defense. It also includes various source code opensource products and infrastructure as code used to harden Kubernetes distributions. n Repo One is currently operated at https: //repo 1. dsop. io/dsop/. n n Iron Bank – Do. D Centralized Artifacts Repository (DCAR) 200+ containers available, 250 containers by end of 2020. n Iron Bank is the Do. D repository of digitally signed, binary container images that have been hardened according to the Container Hardening Guide coming from Iron Bank. Containers accredited in Iron Bank have Do. D-wide reciprocity across classifications. n Iron Bank is currently operated at https: //ironbank. dsop. io/. n Integrity - Service - Excellence 7

Platform One Services (continued) n Dev. Sec. Ops Platform (DSOP) The DSOP is a collection of approved, hardened Cloud Native Computer Foundation (CNCF)-compliant Kubernetes distributions, infrastructure as code playbooks, and hardened containers that implement a Dev. Sec. Ops platform compliant with the Do. D Enterprise Dev. Sec. Ops Reference Design, and its source code is hosted on Repo One. n Infrastructure as Code (Ia. C) Repositories: Platform One Ia. C: https: //repo 1. dsop. io/platform-one n D 2 IQ: https: //repo 1. dsop. io/platform-one/distros/d 2 iq n Rancher Federal: https: //repo 1. dsop. io/platform-one/distros/rancher-federal n Open. Shift 4. x: https: //repo 1. dsop. io/platform-one/distros/red-hat n n Kubernetes CNCF-compliant currently supported are: Open. Shift 4. x, Kubernetes upstream, D 2 IQ Konvoy and Rancher Federal RKE. Kubernetes CNCF-compliant to be supported soon: VMWare Tanzu and Oracle Kubernetes. n Platform One will be supporting the following environments: Amazon Web Services (AWS) IL-2, IL-5, S, S-SAP (when available), TS/SCI, and TS-SAP (FENCES), AWS Outpost n Azure IL-2, IL-5, S (when available), S-SAP (when available), Azure Stack n On-premise / Edge - VMWare v. Sphere n n The DSOP includes the various mandated containers of the Reference Design including Elasticsearch, Fluentd, and Kibana (EFK), Sidecar Container Security Stack (SCSS), etc. Integrity - Service - Excellence 8

Platform One Services (continued) n Party Bus – ABMS All Domain Common Environment: Platform One Shared Enterprise Environments (Multi-Tenant) (for Development, Test and Production) n These are environments that benefit from the Platform One Continuous ATO, hosted on Cloud One, SC 2 S and C 2 S managed by the Platform One team as multi-tenant environments. Perfect for smaller/medium sized teams. They provide Continuous Integration/Continuous Delivery (CI/CD) and various development tools/capabilities. n Impact Level (IL)-2, IL-5, Secret, and TS/SCI environments exist or are in development (pay per user model ($2, 000/user/month)) n Big Bang: Platform One Dedicated Dev. Sec. Ops Environments n Build, deliver and operate custom Infrastructure as Code and Configuration as Code with the deployment of dedicated environments at various classification levels with CI/CD pipelines and c-ATO. Perfect for large teams/programs that need a dedicated enclave (cost per Dev. Sec. Ops environment). n Build and deliver new hardened containers as needed for program specific software (pay per use/container). Integrity - Service - Excellence 9

Platform One Services (continued) n Cloud Native Access Point (CNAP) n The Cloud Native Access Point is available on Cloud One to provide access to Development, Testing, Staging and Production enclaves at IL-2, IL-4 and IL-5 that using Platform One Dev. Sec. Ops environments by using an internet-facing Cloud-native Zero trust environment. n The CNAP enables access to VDI options and allows thick endpoints (incl. mobile), including BYOD, government owned and contractor owned devices to connect at various impact level while enforcing device state/security. n Brings Single Sign On with various Do. D PKI options and IL 2 MFA options. n CNAP diagram: https: //software. af. mil/wp-content/uploads/2020/04/CNAP-Data-Flow. Diagram-v 5. 3 -NC. pptx Integrity - Service - Excellence 10

Platform One Services (continued) n Platform One Training/On-Boarding Options n Check out the CSO Dev. Sec. Ops / DAU training at https: //software. af. mil/training/ n n n Virtual Platform One Learning Hub that provides self service on-boarding [June 2020 Launch] 1 -day training Session: Introduction to Dev. Sec. Ops. Overview and understanding of the vision and activities. [June 2020 Virtual Launch] A 3 day Platform One Platform Workshop. Hands on code and User-Centered Design (UCD) to create your first Platform One Dev. Sec. Ops pipelines and deploy a “push button” Do. D Dev. Sec. Ops software factory. [Currently Available] A 6 -week full on-boarding, that concludes with own CI/CD pipeline and Minimum Viable Product (MVP) ready for production [Currently Available] A 2 -month full on-boarding, that concludes with your platform team being able to support your own Dev. Sec. Ops applications for development and production [July 2020 Virtual Launch] Customized training options (both at our locations or on your premises) (pay per use). Integrity - Service - Excellence 11

Platform One Services (continued) n Platform One Dev. Sec. Ops Managed Tools n Platform One Enterprise Chat: provides a collaboration solution suitable for connecting developer teams (pay per use): IL 4 (. mil email only) https: //chat. collab. cdl. af. mil/ n Platform One Party Bus (pay per use) n Platform One Multi-Level Security Data Transfer (CDS/Diode) (pay per use) n Unclassified to Secret (or S-SAR) n Unclassified to TS (or TS-SAR) n Platform One Stack Exchange: knowledge sharing service for software developers and engineers. (pay per use) n Platform One Cybersecurity/Pen-testing Services n Ability to pen-test a Dev. Sec. Ops environment at various classifications level (pay per use) Integrity - Service - Excellence 12

Dev. Sec. Ops Basic Ordering Agreements (BOAs) – Contract Vehicles n BOA 1: Cloud Services n Services to develop and deploy accredited, integrated and tested code at multiple classification levels and hybrid cloud architectures n Awarded 1 Nov 2019, 27 companies on-boarded n BOA 2: Dev. Sec. Ops Pipeline and Platform Integration and Licensing Services n Dev. Sec. Ops pipeline and platform integration and licensing service to support a wide collection of software and programming tools supporting the CI/CD of software products n Awarded 1 Nov 2019, 9 companies on-boarded n BOA 3: Software Dev. Sec. Ops Services n Technical services of full-stack Dev. Sec. Ops engineers, infrastructure engineers, and other key personnel n Awarded 15 Jan 2020, 19 companies on-boarded Integrity - Service - Excellence 13

Do. D Enterprise Dev. Sec. Ops Architecture Integrity - Service - Excellence

PLAN & DEVELOP Do. D Enterprise Dev. Sec. Ops Technology Stack (Exemplar) DEPLOY & OPERATE BUILD TEST “Continuous Integration & Continuous Delivery” Orchestration MONITOR SECURE Container and Container Management STORE ARTIFACTS SCALE

Why Kubernetes / Containers? n One of the most critical aspect of the Dev. Sec. Ops initiative is to ensure we avoid any vendor lock-in so the Do. D mandated: n Open Container Initiative (OCI) containers (no lock-in to containers/container runtimes/builders) n n Cloud Native Computing Foundation (CNCF) Kubernetes compliant cluster for container orchestration, no lock-in to orchestration options/networking/storage APIs. Saa. S vs COTS/FOSS containers: n Saa. S requires Fed. RAMP certification and will limit you to unclassified environments (IL 2 for Fed. RAMP moderate) which doesn’t satisfy most needs for Do. D programs. Often takes up to 1 year. n COTS/FOSS as containers: can be sold as a managed service deployed in Do. D cloud environments (including classified clouds) on Kubernetes and can be accredited at multiple classification levels, within weeks, by following the container hardening guide and vendor on-boarding process! n Get your container(s) onboarded on Iron Bank for Do. D wide reciprocity? https: //repo 1. dsop. io/dsop/dccscr/tree/master/contributor-onboarding n Containers are immutable and will allow the Do. D to centrally accredit and harden containers (FOSS, COTS, GOTS) (think of a true gold disk concept but that actually scale and works). n Continuous Monitoring is a critical piece of our Continuous ATO model and the Sidecar Container Security Stack (SCSS) brings those capabilities with Behavior, Zero Trust and CVE scanning. n Kubernetes will provide: n Resiliency: Self-healing so containers that crash can automatically be restarted, n Baked-in security: thanks to automatic injection of our Sidecar Container Security Stack (SCSS) to any K 8 S cluster with Zero Trust, n Adaptability: containers are “Lego” blocks and can be swapped with no downtime thanks to load balancing and modern routing (A/B testing, canary release etc. ), n Automation: thanks to our Infrastructure as Code (Ia. C) and Git. Ops model, n Auto-scaling: if load requires more of the same container, K 8 S will automatically scale based on compute/memory needs, n Abstraction layer: ensure we don’t get locked-in to Cloud APIs or to a specific platform as K 8 S is managed by CNCF and dozens of products are compliant with its requirements. Integrity - Service - Excellence 16

Key “Dev. Sec. Ops” Ingredients n Abstracted: to avoid drifts, be agnostic to environment (Cloud/on-premise/classified/disconnected…) and prevent lock-ins with Cloud or Platform layers, we leverage CNCF compliant Kubernetes and OCI compliant containers - open source stacks with U. S eyes on code and continuous scanning, n Git. Ops / Infrastructure as Code (Ia. C): no drift, everything is code (including configuration, networking etc. ) Instantiate entire stack automatically, n Continuous Integration/Continuous Delivery pipeline (CI/CD): fully containerized and using Infrastructure as Code (Ia. C), n Hardened Containers: hardened “Lego blocks” to bring options to development teams (one size fits all lead to shadow IT) n Software Testing: mandated high test coverage, n Baked-in Security: mandated static/dynamic code analysis, container security, bill of material (supply chain risk) etc. n Continuous Monitoring: n Centralized logging and telemetry, n Automated alerting, n Zero trust, leveraging Service Mesh as Sidecar (part of SCSS), down to the container level, n Behavior detection (automated prevention), n CVE scanning, n Chaos engineering: Dynamically kills/restarts container with moving target defense. Integrity - Service - Excellence 17

“Infrastructure as Code” Benefits The “Infrastructure as Code” concept is a critical Dev. Sec. Ops ingredient to ensure that production environments do not drift from development/testing environments. No human should make changes in production environments. Changes should only be made in source code and redeployed by the CI/CD pipeline. n No drift between environments, whether classified/disconnected/Cloud/on-premise, n Immutable, n Replicable, n Automated, n No human in production environments: reduces attack surface (disable SSH etc. ), insider threat and configuration drifts, n Everything is code: including playbooks, networking, tests, configuration etc. Integrity - Service - Excellence 18

What is Git. Ops? n Based on Infrastructure as Code concepts, makes Git the single source of truth of the desired state of your Infrastructure, Platform and Applications. n Benefits: n Everything is code: infrastructure, networking, configuration, sealed secrets etc. n Auditability & Compliance n Consistent deployments and rollback (no drifts between environment) n Configuration Management enforcement n Disaster Recovery Baked-in security: Kubernetes clusters pulls from Git. CI/CD won’t have access to production clusters. Removing human from production environments n Declarative manifests and playbooks n n Options: n Argo CD, Flux as FOSS. Projects are merging into a single FOSS and be part of CNCF. Integrity - Service - Excellence 19

Microservices Architecture (ISTIO) n Turnkey Service Mesh (ISTIO) architecture n ISTIO side car proxy, baked-in security, with visibility across containers, by default, without any developer interaction or code change n Benefits: n API Management, service discovery, authentication… n Dynamic request routing for A/B testing, gradual rollouts, canary releases, resilience, observability, retries, circuit breakers and fault injection n Layer 7 Load balancing n Zero Trust model: East/West Traffic Whitelisting, ACL, RBAC… n TLS encryption by default, Key management, signing… Integrity - Service - Excellence 20

DSAWG Dev. Sec. Ops Subgroup n Voting members: n n SAF/CSO (chair), Do. D CIO/DISA and A&S Sub groups: n n n Team 1: Do. D Enterprise Dev. Sec. Ops Ref Design (and following updates) Team 2: Kubernetes SRG Team 3: Containers SRG Team 4: Cloud Native Access Point Team 5: Work with NIST (Ron Ross) on Dev. Sec. Ops new publication based on Ref Design. Team 6: Continuous ATO Guidance, defining the: n Accreditation requirements to accredit Dev. Sec. Ops pipeline process and the various layers n Accreditation requirements to accredit teams to use the accredited pipelines n The expected deliverables / artifacts of pipelines/platforms + automation e. Mass etc. Team 7: Write the required training for SCAs and ISSMs and AOs to understand how to adopt to new c. ATO guidance Team 8: Dev. Sec. Ops Real-Time/Embedded systems Team 9: Dev. Sec. Ops Playbook / Best Practices Team 10: High Performance Computing (HPC) Team 11: Digital Engineering as a Service Integrity - Service - Excellence 21

Dev. Sec. Ops Platform Stack (continuously evolving) Integrity - Service - Excellence

Dev. Sec. Ops Product Stack (1) Source Repository Git. Hub Government Git. Lab Container Management technologies: Kubernetes Openshift VMWare Tanzu OKD Rancher (K 8 S only) D 2 IQ Konvoy Docker EE (K 8 S only) Container Packagers: Helm Kubernetes Operators API Gateways Kong Azure API AWS API Axway 3 Scale Apigee ISTIO (service mesh) Artifacts Artifactory Nexus Maven Archiva S 3 bucket Programming Languages C/C++ C#/. NET Core Java PHP Python Groovy Ruby R Rust Scala Perl Go Node. JS Swift Integrity - Service - Excellence Databases SQL Server My. SQL Postgre. SQL Mongo. DB SQLite Redis Elasticsearch Oracle etcd Hadoop/HDInsight Cloudera Oracle Big Data Solr Neo 4 J Memcached Cassandra Maria. DB Couch. DB Influx. DB (time) 23

Dev. Sec. Ops Product Stack (2) Message bus/Streams Kafka Flink Nats Rabbit. MQ Active. MQ Proxy Oauth 2 proxy nginx ldap auth proxy openldap HA Proxy Envoy Visualization Tableau Kibana Logstash Splunk Forwarder Fluentd Syslogd Filebeat rsyslog Webservers Apache 2 Nginx IIS Lighttpd Tomcat Docker base images OS: Alpine Busybox Ubuntu Centos Debian Fedora Universal Base Image Serverless (Faa. S) Knative AI/ML/Deep Learning Kubeflow Integrity - Service - Excellence 24

Dev. Sec. Ops Product Stack (3) Build MSBuild CMake Maven Gradle Apache Ant Tests suite Cucumber J-Unit Selenium Testing. Whiz Watir Sahi Zephyr Vagrant App. Verify nosetests Soap. UI Lean. FT Test coverage Ja. Co Emma Cobertura codecov CI/CD Orchestration Jenkins (open source) Cloud. Bees Jenkins Git. Lab Jenkins plugins Dozens (Need to verify security). Configuration Management / Delivery Ansible Terraform Kubernetes Operators Helm Security Tenable / Nessus Agents Fortify Twistlock Aqua Sonar. QBE Qualys Stack. Rox Aporeto Snort OWASP ZAP Contrast Security Open. VAS Metasploit Thread. Fix pylint JFrog Xray Open. SCAP (can check against DISA STIG) Open. Control for compliance documentation Integrity - Service - Excellence Security (2) Snyk Code Climate AJAX Spider Tanaguru (508 compliance) In. Spec OWASP Dependency-Check Burp HBSS Anchore Checkmarx SD Elements Clair Docker Bench Security Notary Sysdig Layered Insight Black. Duck Nexus IQ/Lifecycle/Firewall Run. Safe 25

Dev. Sec. Ops Product Stack (4) Monitoring Sensu EFK (Elasticsearch, Fluentd, Kibana) Splunk Nagios New Relic Sentry Promotheus Grafana Kiali Collaboration Rocket. Chat Matter. Most Pager. Duty Plan Jira Confluence Rally Redmine Pivotal Tracker Secrets Kubernetes Secrets Vault Credentials (Jenkins) Crypto. Move Documentation Javadoc RDoc Sphinx Doxygen Cucumber php. Documentator Pydoc Performance Apache AB Jmeter Load. Runner SSO Keycloak Integrity - Service - Excellence 26

Legacy to Dev. Sec. Ops => Strangler Pattern n Martin Fowler describes the Strangler Application: n One of the natural wonders of this area are the huge strangler vines. They seed in the upper branches of a fig tree and gradually work their way down the tree until they root in the soil. Over many years they grow into fantastic and beautiful shapes, meanwhile strangling and killing the tree that was their host. n To get there, the following steps were followed: First, add a proxy, which sits between the legacy application and the user. Initially, this proxy doesn’t do anything but pass all traffic, unmodified, to the application. n Then, add new service (with its own database(s) and other supporting infrastructure) and link it to the proxy. Implement the first new page in this service. Then allow the proxy to serve traffic to that page (see below). n Add more pages, more functionality and potentially more services. Open up the proxy to the new pages and services. Repeat until all required functionality is handled by the new stack. n The monolith no longer serves traffic and can be switched off. n n Learn more: https: //www. ibm. com/developerworks/cloud/library/cl-strangler-application-patternmicroservices-apps-trs/index. html and https: //www. michielrook. nl/2016/11/strangler-pattern-practice/ Integrity - Service - Excellence 27

Thank You! Nicolas Chaillan Chief Software Officer, U. S. Air Force af. cso@us. af. mil Integrity - Service - Excellence

Backup Slides Integrity - Service - Excellence

Nicolas M. Chaillan Chief Software Officer n Nicolas M. Chaillan is the Chief Software Officer at the U. S. Air Force and the Co-Lead for the Do. D Enterprise Dev. Sec. Ops Initiative. n He is the former Special Advisor for Cloud Security and Dev. Sec. Ops at OSD, A&S. n He was the Special Advisor for Cybersecurity at the Department of Homeland Security and the Chief Architect for Cyber. gov, the new robust, innovative and holistic. Gov cyber security architecture for all. gov agencies. n Chaillan is a technology entrepreneur, software developer, cyber expert and inventor. He is recognized as one of France’s youngest entrepreneurs after founding his first company at 15 years of age. n With 19 years of international tech, entrepreneurial and management experience, Chaillan is the founder of more than 12 companies, including AFTER-MOUSE. COM, Prevent-Breach, any. Guest. com, and more. n Over the last eight years alone, he has created and sold over 180 innovative software products to 40 Fortune 500 companies. n Chaillan is recognized as a pioneer of the computer language PHP. Integrity - Service - Excellence 30

Key “Continuous Security” Ingredients n Kubernetes hardening. n Automated injection of Sidecar Container Security Stack (SCSS) into all containers/pods running without manual action. RBAC/SSO/SELinux enabled n Compliant with CIS Kubernetes Benchmark, mapped to NIST 800 -53 n Nodes, master, etcd are hardened. n Automated backups of cluster and persistent storage! n n Sidecar Container Security Stack (SCSS): n Automated centralized logging and telemetry with Elasticsearch, Fluentd, Kibana (EFK), n Service Mesh (Istio): n Baked-in zero trust model down to the container level! n Strong identities automatically generated using certificates. n m. TLS tunnel injected across all container communication n Whitelist enforcement, Layer 7 load balancer etc n Container security: Continuous Scanning, Alerting, CVE scanning, Behavior detection both in development and production (Build, Registry, Runtime) with Twistlock (looking into Stack. Rox and Sysdig) n Container security and insider threat (custom policies detecting unapproved changes to Dockerfiles) with Anchore n Automated STIG compliance with Open. SCAP. Integrity - Service - Excellence 31

Questions about Repo One / Iron Bank? n Containers accredited in the Repo One / Iron Bank repository have Do. D-wide reciprocity across classifications. n Source code repo: Repo One: https: //repo 1. dsop. io/dsop n Iron Bank (Container binaries): https: //ironbank. dsop. io n Programs can contribute containers that have enterprise benefits to Repo One / Iron Bank and our team will accredit them Do. D-wide and maintain them. n If you need to accredit/harden custom containers Platform One can do this as a “managed service”, Pay per use model. n We are building a container which automatically download container updates into your K 8 S cluster, checks signatures and pushes them to your local registry (agnostic to your artifact repo) n Questions? Email af. cso@us. af. mil Integrity - Service - Excellence 32

“Cloud One” vs “Platform One” n Cloud One: Centralized team to provide Cloud Infrastructure with baked-in security to Do. D programs. Think of it as the Infrastructure team with baked-in security, CSSP and Authority to Operate (ATO). n Only contact Cloud One if you only need Cloud compute/storage, if you need a Dev. Sec. Ops stack (on Cloud One or not), contact « Platform One by Level. UP » n n n Point of Contact: Norris, Paul D Maj USAF AFLCMC (USA) paul. norris. 2@us. af. mil Platform One: n Centralized team to provide Dev. Sec. Ops/Software Factory with baked-in security to Do. D Programs. Think of it as the Platform Team with the ability to deploy a Dev. Sec. Ops (Kubernetes compliant) Platform and CI/CD pipeline with a Continuous ATO (c-ATO). You select from accredited tools to accelerate your ability to focus on delivering mission capabilities. n Contact Platform One if you need both Cloud and Dev. Sec. Ops capabilities! n Point of Contact: Slaughter, Rob Maj USAF - rob. slaughter@afwerx. af. mil; Bryan, Austen R Capt USAF - austen. bryan. 1@us. af. mil; Integrity - Service - Excellence 33

Questions about the Dev. Sec. Ops Basic Ordering Agreements (BOAs)? n Covers Cloud Services, Talent and Licenses so Do. D programs can get everything they need for a Dev. Sec. Ops environment, completely turnkey. n Pay per use models. n Not selected yet? We will have quarterly on-boarding events for new vendors/award opportunities! n Aims to award each order within 15 days (!!!) n Available to the entire Department of Defense n Point of Contact: Paul, Christopher C 2 d LT USAF AFLCMC (USA) christopher. paul. 3@us. af. mil n Slaughter, Robert C Maj USAF (USA) robert. slaughter. 3@us. af. mil n Wyler, Victoria R Capt USAF SAF-AQ (USA) victoria. r. wyler. mil@mail. mil n Integrity - Service - Excellence 34

Contribute your containers or get your COTS/FOSS containers accredited! n Containers are the easiest way to get accredited Do. D-wide across multiple classifications today. n Containers accredited in the DCCSCR/DCAR repository have Do. D-wide reciprocity across classifications. n Check out the vendor on-boarding guide at: https: //repo 1. dsop. io/dsop/dccscr/tree/master/contributoronboarding n By being compliant with the Do. D Enterprise Dev. Sec. Ops Container Hardening guide (last version at https: //software. af. mil/dsop/documents/), you can have your containers (FOSS/COTS/GOTS) accredited for Do. D use. n Recommend using the hardened STIG UBI 8 images (Universal Base Image which is lightweight RHEL but doesn’t need a license) from the DCAR repo as your base image so you don’t have to STIG your container base OS: https: //repo 1. dsop. io. Use the binary signed version on DCAR, do not rebuild it. n Key aspects: n Your container must be able to build offline. If you have dependencies, provide them with an automated script that will download new updates of those dependencies. ALL DEPENDENCIES must be included n Container must be able to be built offline, no downloads in Dockerfile! n Dockerfiles must be provided and be able to be rebuilt. If you have a different base OS (not UBI), it must be STIGed. Integrity - Service - Excellence 35

“Platform One by Level. UP” The Air Force Software Factory Team n Merged top talent across U. S. Air Force from various Factories (Kessel Run, Space. CAMP and UP). n Helps instantiate Dev. Sec. Ops CI/CD pipelines / Software Factories within days at various classification levels. n Manages Software Factories for Development teams so they can focus on building mission applications. n Provides Basic Ordering Agreement (BOA) Do. D-wide Dev. Sec. Ops contracts for Cloud Service, Talent and Licenses. Enables awards every 15/30 days with bulk discounts. n Decouples Development Teams from Factory teams with Dev. Sec. Ops and Site Reliability Engineer (SRE) expertise. n Partners with Cloud One to provide IL 2, 4, 5 and 6 access but also uses C 2 S/SC 2 S and various on-premise environments! n Self-learning and training capabilities to enable teams move to Scrum/Kanban/e. Xtreme Programming (XP) Agile practices. n Leverages the Do. D hardened containers while avoiding one-size-fits-all architectures. n Fully compliant with the Do. D Enterprise Dev. Sec. Ops Initiative (DSOP) with Do. D-wide reciprocity and an ATO. Leverages Zero Trust model. n Hardens the 172 Do. D enterprise containers (databases, development tools, CI/CD tools, cybersecurity tools etc. ). n Provides Software Enterprise Services with Collaboration tools, Cybersecurity tools, Source code repositories, Artifact repositories, Development tools, Dev. Sec. Ops as a Service, Chats etc. These services will be MANAGED services on Cloud One. Integrity - Service - Excellence 36

“Platform One by Level. UP” Managed Services “A La Carte” n Hardened Containers Options n Delivery of hardened enterprise containers with accreditation reciprocity (existing containers only). n Delivery of custom hardened containers as needed. n Continuous Integration / Continuous Delivery (CI/CD) Options n Delivery of existing hardened Kubernetes/Open. Shift/PKS playbooks (full Infrastructure as Code). n Delivery of a turnkey CI/CD pipeline (Software Factory) with complete « Infrastructure as Code » to instantiate on any environment (development teams picks the tools from the approved hardened containers) on various classified/unclassified environment. n Training/On-Boarding Options n 1 -day training Session: introduction to Dev. Sec. Ops. Overview and understanding of the vision and activities. n A 3 day introduction to the Platform One Dev. Sec. Ops tech stack. Hands on code and User-Centered Design (UCD) to deploy your first demo app to production. n A several week full on-boarding, that concludes with an MVP ready for production. n A several month full on-boarding, that concludes with your platform team being able to support your own Dev. Sec. Ops applications for development and production. n Customized training options (both at our locations or on your premises). n Contracting Support Options n Ability to leverage the Dev. Sec. Ops BOAs (Cloud Services, Talent and Licenses). n Enable access to Dev. Sec. Ops engineers/SREs Full-Time-Equivalent (FTEs) (Medics/Counselors) to assist Programs. Integrity - Service - Excellence 37

Cloud One n Air Force Cloud Office with turnkey access to AWS Gov. Cloud and Azure Government at IL 2, 4 and 5. IL 6 available by December 2019. n Simple “Pay per use” model with ability to instantiate your own Development and Production VPCs at various Impact Levels within days with full compliance/security and a baked-in ATO. n Enterprise Solution: we provide the guardrails to the cloud in a standard manner so you can focus on your mission n Fully Automated: All environmental stand-up is managed by Infrastructure as Code, drastically speeding up deployment, reducing manual work, and human error n Centralized Identities and Single-Sign-On (SSO): one login across the Cloud stack n Internet facing Cloud based VPN to connect to IL 5 enclaves with a Cloud Native Access Point (not using IAP/CAP). n Dev. Sec. Ops Focused: secure, mission driven deployments are built into the framework to ensure self-service and seamless deployments. Leverages Zero Trust model. n Proactive Scaling and System Monitoring: Mission Owners can see all operational metrics and provide rules and alerts to manage each mission their way n Accreditation Inheritance has been identified in the AF-Cloud One e. MASS accounts (AWS & Azure) to include inheritance from the CSP, USAF, Do. D and CSSP. All that’s left for the mission is the controls that are unique to them. Integrity - Service - Excellence 38

Problem Statement n What is Dev. Sec. Ops? n The software automated tools, services, and standards that enable programs to develop, secure, deploy, and operate applications in a secure, flexible and interoperable fashion. n Why should I care? Software and cybersecurity pervades all aspects of Do. D's mission (from business systems to weapons systems to Artificial Intelligence to cybersecurity to space) - establishing Dev. Sec. Ops capabilities will: n Deliver applications rapidly and in a secure manner, increasing the warfighters competitive advantage n Bake-in and enforce cybersecurity functions and policy from inception through operations n Enhance enterprise visibility of development activities and reduce accreditation timelines n Ensure seamless application portability across enterprise, Cloud and disconnected, intermittent and classified environments n Drive Do. D transformation to Agile and Lean Software Development and Delivery n Leveraging industry acquisition best practices combined with centralized contract vehicle for Dev. Sec. Ops tools and services will enable rapid prototyping, real-time deployments and scalability. n We cannot be left behind: China, Russia and North Korea are already massively implementing Dev. Ops. n Integrity - Service - Excellence

Value for Do. D Programs n Enables any Do. D Program across Do. D Services deploy a Do. D hardened Software Factory, on their existing or new environments (including classified, disconnected and Clouds), within days instead of a year. Tremendous cost and time savings. n Multiple Dev. Sec. Ops pipelines are available with various options (no one-size-fits-all) n Enables rapid prototyping (in days and not months or years) for any Business, C 4 ISR and Weapons system. Deployment in PRODUCTION! n Enables learning and continuous feedback from actual end-users (warfighters). n Enables bug and security fixes in minutes instead of weeks/months. n Enables automated testing and security. n Enables continuous Authorization to Operate (c-ATO) process. Authorize ONCE, use MANY times! n Brings a holistic and baked-in cybersecurity stack, gaining complete visibility of all assets, software security state and infrastructure as code. Integrity - Service - Excellence 40

From Waterfall to Dev. Sec. Ops Integrity - Service - Excellence 41

Training Options n Our latest training videos from DAU available at: https: //software. af. mil/training/ n Check out our curated You. Tube videos on Kafka, Kubernetes, Service Mesh, Microservices, Cloud etc. at https: //software. af. mil/training/ NEW: Federal employees/Military personnel (limited number of seats, free of charge): reach out to us at af. cso@us. af. mil if you want to pilot the access to the O’Reilly Online Learning Platform (all O’Reilly content + virtualized K 8 S env)! n Platform One Training/On-Boarding Options: n 1 -day training Session: introduction to Dev. Sec. Ops. Overview and understanding of the vision and activities n A 3 -day introduction to Level. UP Dev. Sec. Ops tech stack. Hands on code and User-Centered Design (UCD) to deploy your first demo app to production n A several week full on-boarding, that concludes with an MVP ready for production n A several month full on-boarding, that concludes with your platform team being able to support your own Dev. Sec. Ops applications for development and production n Customized training options (both at our locations or on your premises) n Follow the CNCF channel: https: //www. youtube. com/channel/UCvqb. FHw. N-nwal. WPj. PUKpv. TA n Integrity - Service - Excellence 42

Do. D Enterprise Dev. Sec. Ops Architecture* Program Source code repository pulls Centralized Do. D Enterprise Dev. Sec. Ops Artifacts Repository Application / Microservices built by Do. D Programs. Do. D Enterprise Dev. Sec. Ops Platform** pulls Microservices Architecture (ISTIO) Dev. Sec. Ops CI/CD pipeline** Sidecar Container Security Stack Artifacts Repository** pulls Fluentd Realtime pushes Elasticsearch Kubernetes *each Do. D Program can have its own instantiation of the Do. D Enterprise Dev. Sec. Ops Platform on any Cloud. ** can be installed with single command deployed on any Cloud. *** could be deployed inside an enclave or onpremises **** gives complete visibilities of assets, security/vulnerability state etc. can be integrated to existing cybersecurity shared services. Optional Abstraction Layer with Red Hat Open. Shift or PKS or CNCF compliant Kubernetes Product pulls Continuously Hardens Docker Public Images and Assesses Open Source Libraries Do. D OCIO/DISA Centralized Logs/Telemetry**** Per Do. D Service for Service-wide Visibility Logs/Telemetry**** Bare-metal, Gov. Cloud, AWS Secret, Azure Secret, mil Cloud, C 2 S, Jedi…*** 43

Self-Learning (1) n Recommended Videos (Part 1) Watch our playlists, available at different expertise levels and continuously augmented! n Kafka / KSQL (message bus, pub/sub, event driven): n n Beginners: https: //www. youtube. com/playlist? list=PLSIv_F 9 Tt. Llzz 0 zt 03 Ludtid 7 icr. XBesg Intermediate: https: //www. youtube. com/playlist? list=PLSIv_F 9 Tt. Llxx. XX 0 o. Czt 7 la. O 6 m. D 61 UIQw n Advanced: N/A n Kubernetes n Beginners: https: //www. youtube. com/playlist? list=PLSIv_F 9 Tt. Llyd. Fz. Qzk. YYDd. QK 7 k 5 c. EKub. Q n Intermediate: https: //www. youtube. com/playlist? list=PLSIv_F 9 Tt. Llx 8 d. SFH_j. FLK 40 Tt 7 KUXTN_ n Advanced: https: //www. youtube. com/playlist? list=PLSIv_F 9 Tt. Llytd. AJi. Vqb. Huc. WOvn 5 Lr. TNW n Integrity - Service - Excellence 44

Self-Learning (2) n Recommended Videos (Part 2) n Watch our playlists, available at different expertise levels and continuously augmented! n Service Mesh n Beginners: https: //www. youtube. com/playlist? list=PLSIv_F 9 Tt. Llxt. C 4 r. DIMQ 8 Qi. G 5 UBCjz 7 VH n Intermediate: https: //www. youtube. com/playlist? list=PLSIv_F 9 Tt. Llw. WK_Y_Cas 8 Nyw-Dsdb. H 6 vl n Advanced: https: //www. youtube. com/playlist? list=PLSIv_F 9 Tt. Llx 8 VW 2 MFONMRw. S_-2 r. SJwdn n Microservices n Beginners: https: //www. youtube. com/playlist? list=PLSIv_F 9 Tt. Llz_U 2_Ra. ONTGYLkz 0 lh-A_L n Intermediate: https: //www. youtube. com/playlist? list=PLSIv_F 9 Tt. Llxqju. AXxo. RMjvspa. EE 8 L 2 c. B n Advanced: https: //www. youtube. com/playlist? list=PLSIv_F 9 Tt. Llw 4 CF 4 F 4 t 3 g. VV 3 j 0512 CMsu Integrity - Service - Excellence 45

Self-Learning (3) n Recommended Books n A Seat at the Table – by Mark Schwartz (former CIO of USCIS, leader in Agile) This book is highly recommended for ALL leadership as it is not technical but focused on the challenges around business, procurement and how leadership can enable Dev. Ops across the organization and remove impediments. n The Phoenix Project – by the founders of Dev. Ops n The Dev. Ops Handbook – by Gene Kim, Patrick Debois. For those who drive to work like me (for hours), please note that these books are available as Audiobooks. Integrity - Service - Excellence 46

Current Challenges Integrity - Service - Excellence

Key Procurement Recommendations for Successful SW Acquisition n Mandate the use of Agile methodologies including for the creation of RFPs by using user stories instead of predefined unverified technology claims or expectation. Clearly established success metrics should be defined based on the targeted end-goals. It should allow for rapid change of scope based on learnings as long as it meets those end goals. n Incentivize the use of Microservices/Smaller code base instead of building giant monolith that cannot be supported or understood. The use of the 2 pizza rule by Amazon is a great example to size teams. https: //www. theguardian. com/technology/2018/apr/24/the-two-pizza-rule-and-the-secret-of-amazons-success n Incentivize the use of Dev. Sec. Ops with pre-defined metrics for: Test coverage % n Security scans (fuzzing, pen-test, static/dynamic code analysis) n Documentation n n Incentivize the use of Containers whenever possible to allow for scale and rapid deployment to any Cloud or on premise servers. n Incentivize the use of Container Management solution (Kubernetes) whenever possible to allow for rapid deployment, side-car container security and rolling updates. Integrity - Service - Excellence 48

Dev. Ops Challenges for Leadership n Leadership can enable the adoption of Dev. Sec. Ops by bringing its ENTIRE stack as a platform and by leveraging Dev. Sec. Ops solutions. n It is imperative not to “select” a limited option of tools. The key of microservices and containers is to be able to use the best solution to achieve the outcome desired. We should not limit options to the extend possible. n Need to establish baseline requirements / thresholds for cybersecurity, test coverage, test results, documentation. This shouldn’t be reinvented per office but global to Do. D as a standard practice to facilitate adoption. n Once those baseline requirements are set, OCIO can provide CI/CD Platform’ stack with embedded security as a sidecar container and provide pre-ATOs for systems using the Platform stack and automatically integrating the OCIO security baseline requirements. n Bringing the entire stack as a Platform as a Service is key to avoid that each office reinvents the wheel and builds their own baseline requirements. n Understand that Failing is a GOOD thing! It is part of learning. It allows us to understand what works and what doesn’t AS LONG AS we leverage rapid prototyping, which allows for QUICK failure and mostly painless. It also allows us to reprioritize rapidly and leverage learnings. Integrity - Service - Excellence 49

Dev. Ops Challenges for Acquisition Office n Dev. Ops is a complete disruption of the traditional Acquisition model. We need to leverage Sec 873/874 of the 2018 NDAA (check out https: //cdnapisec. kaltura. com/index. php/extwidget/preview/partner_id/2203981/uiconf_id/38241181/entry_id/1_gib 6 brbc/embed/dynamic ). n No more complex RFPs with long planning phases with deliverables and milestones and fixed budgets. Keep in mind that when we write RFPs, we assume we know what we need and know exactly what the solution is. This doesn’t allow for us to learn and adopt new ideas along the way. This is the main cause of current failures. The world/technology evolves, what we know evolves… We must be able to adapt continuously to guarantee success. n This is NOT scope creep but proper agile scope management. n RFPs should NOT define precise requirements with pre-defined technologies but focus on establishing mission outcomes and precise metrics to prove success of those end-goals. n There is no “beginning” or “end” of a project, the project will continuously evolve based on mission needs. Yes, a project might be terminated but the idea is continuous evolution and development. n The traditional accounting methods - where you have the R&D, development and deployment in production followed by maintenance/support phases - don’t apply anymore. We CONTINUOUSLY develop, deploy and learn. There are multiple releases per day. n New procurement tools must enable continuous development and incentivize the use of containers, microservices and Agile methods. n Most of the Dev. Ops tools are opensource and the only costs are Cloud hosting/computing/storage. Some members are worried about costs of licenses and think about consolidation for cost saving - this is a non-issue. We pay for what we use as a Iaa. S/Paa. S/Saa. S model. Integrity - Service - Excellence 50

But…How Do We Trust/Monitor? n Most people treat IT as a contractor/vendor relationship that provides IT services and most don’t trust that IT developers can/want to deliver on-time and efficiently. As long as we treat IT as a vendor, this relationship is bound to fail. n Thanks to Dev. Ops and Agile, we can continuously monitor everything. There is no need for phases and milestones, we can continuously check the status of the Product Backlog and the continuously delivered product in testing/staging environment or even in production. n We can immediately verify that our assumptions are correct and learn faster. n This provides BETTER and more TANGIBLE and qualified monitoring as we can leverage metrics and real-life feedback. n Thanks to Agile and Dev. Sec. Ops principles, we collect real-time telemetry to gain continuous visibility on our deployments. Integrity - Service - Excellence 51

But…How to Secure at Scale? n Mandating the use of containers and Kubernetes whenever possible will enable us to gain complete visibility of the container’s stack, thanks to the use of side-car containers. n A side-car container runs along side of all container pods, ensuring that no matter the size of the clusters, a security container will be present to get visibility, enforce security best practices and compliance requirements. n By leveraging Agile methodologies, integrating new Dev. Sec. Ops pipeline solutions and defining a security baseline per product to ensure proper compliance and security. n Each new product in the stack will be containerized in a Hardened Docker base image to enable any development team to reuse it without having to reinventing the wheel (similar to the gold images concept except this is using modern container technology). Integrity - Service - Excellence 52

Waterfall n The waterfall model is a relatively linear sequential design approach for certain areas of engineering design. In software development, it tends to be among the less iterative and flexible approaches, as progress flows in largely one direction ("downwards" like a waterfall) through the phases of conception, initiation, analysis, design, construction, testing, deployment and maintenance. n The waterfall development model originated in the manufacturing and construction industries; where the highly structured physical environments meant that design changes became prohibitively expensive much sooner in the development process. When first adopted for software development, there were no recognized alternatives for knowledge-based creative work. n Waterfall basically is a sequential model where software development is segregated into a sequence of pre -defined phases – including feasibility, planning, design, build, test, production, and support. On the other hand, Agile development methodology follows a linear sequential approach while providing flexibility for changing project requirements, as they occur. Integrity - Service - Excellence 53

Agile Terminologies Integrity - Service - Excellence

Agile n Agile software development describes an approach to software development under which requirements and solutions evolve through the collaborative effort of self-organizing and cross-functional teams and their customer(s)/end user(s). It advocates adaptive planning, evolutionary development, early delivery, and continual improvement, and it encourages rapid and flexible response to change. n The term agile (sometimes written Agile) was popularized, in this context, by the Manifesto for Agile Software Development. The values and principles espoused in this manifesto were derived from and underpin a broad range of software development frameworks, including Scrum and Kanban. n There is significant evidence that adopting agile practices and values improves the agility of software professionals, teams and organizations; n Agile enables you to learn directly for end users and implement changes while doing so. n Extreme programming (XP) is a software development methodology which improves software quality and responsiveness to changing customer requirements. It advocates frequent "releases" in short development cycles, which is intended to improve productivity and introduce checkpoints at which new customer requirements can be adopted. It includes: programming in pairs or doing extensive code review, unit testing of all code, avoiding programming of features until they are needed, flat management structure, code simplicity and clarity, expecting changes in the requirements as time passes and the problem is better understood, and frequent communication with the customer and among programmers. Integrity - Service - Excellence 55

Agile Scrum Sprints n Scrum Sprint is a repeatable fixed time-box during which a "Done" product of the highest possible value is created. It also has a maximum duration. Usually, a Sprint lasts for one month or less. n Usually, daily meetings are held to discuss the progress of the project undertaken and any difficulty faced by any team member of the team while implementing the project. The outcome of the sprint is a deliverable, albeit with some increments. The scrum is used for projects like Web Technology or development of a product for the new market, i. e. the product with lots of requirement or fast-changing requirement. Integrity - Service - Excellence 56

Agile User Story User stories are one of the primary development artifacts for Scrum and Extreme Programming (XP) project teams. A user story is a very high-level definition of a requirement, containing just enough information so that the developers can produce a reasonable estimate of the effort to implement it. As a [ type of user ], I want [ some goal, function ] so that [ some reason ]. As a HR Manager, I need to view a candidate’s status so that I can manage their application process throughout the recruiting phases. Learn more: http: //www. agilemodeling. com/artifacts/user. Story. htm Integrity - Service - Excellence 57

Agile Product Owner is responsible for product's current state of development and for maximizing the product's value. Product Owner can be one person, even if he represents a committee. His jobs includes: n Maintaining items in Product Backlog. n Assigning order to items in Backlog. n Ensuring that items in Product Backlog are clear to the Development Team. It is important to ensure that the product owners include end users that understand the end goals for the product. Integrity - Service - Excellence 58

Agile Product Backlog The product backlog comprises an ordered list of product requirements that a scrum team maintains for a product. The format of product backlog items varies, common formats include user stories, use cases, or any other requirements format the team finds useful. These will define features, bug fixes, non-functional requirements, etc. —whatever must be done to successfully deliver a viable product. The product owner prioritizes product backlog items (PBIs) based on considerations such as risk, business value, dependencies, size, and date needed. https: //en. wikipedia. org/wiki/Scrum_(software_development)#Product_backlog Integrity - Service - Excellence 59

More Agile Terminologies (1) n Development Team The development team is responsible for the implementation of the articles in Sprint Backlog. Although several members of the development team may specialize in different areas, the development team as a whole is responsible for the development of functionality. n Sprint Backlog refers to a subset of Product Backlog that is selected for a Sprint along with its delivery plan. Based on the items in the Sprint Backlog, Development Team decides how they will create a "Done" product. Integrity - Service - Excellence 60

More Agile Terminologies (2) n Daily Scrum is a fixed time, fixed place event that allows Development Team to synchronize and plan work for the next 24 hours based on the amount of work done since the last Daily Scrum. During Daily Scrum, Development Team members explain: What did I do yesterday that helped towards Sprint Goal? n What am I going to do today towards my Sprint Goal? n What Impediments I see towards accomplishing my Sprint Goal? n Daily Scrum usually lasts for 15 minutes, but can be followed by other meetings for detailed discussions. n n Sprint Review & Retrospective n Sprint Review is scheduled after the sprint ends to inspect the amount of work done and adapt the Product Backlog if necessary. Similarly, Retrospective is used to analyze what went right in the Sprint and what could be improved upon. This Retrospective feedback helps improve the process in Sprints to follow. Integrity - Service - Excellence 61

Agile vs Waterfall (1) Top 10 differences between Agile and Waterfall Methodology: 1. The software development process is divided into different phases in the Waterfall model while Agile methodology segregates the project development lifecycle into sprints 2. Waterfall is a structured software development methodology, and often times can be quite rigid, whereas the Agile methodology is known for its flexibility 3. According to the Waterfall model, software development is to be completed as one single project, which is then divided into different phases, with each phase appearing only once during the SDLC. However, the Agile methodology can be considered as a collection of many different projects, which are nothing but the iterations of the different phases focusing on improving the overall software quality with feedbacks from users or the QA team 4. If you want to use the Waterfall model for software development, then you have to be clear with all the development requirements beforehand as there is no scope of changing the requirements once the project development starts. The Agile methodology, on the other hand, is quite flexible, and allows for changes to be made in the project development requirements even after the initial planning has been completed Integrity - Service - Excellence 62

Agile vs Waterfall (2) 5. All the project development phases such as designing, development, testing, etc. are completed once in the Waterfall model while as part of the Agile methodology, they follow an iterative development approach. As a result, planning, development, prototyping and other software development phases can appear more than once during the entire SDLC 6. One of the major differences between Agile and Waterfall development methodology is their individual approach towards quality and testing. In the Waterfall model, the “Testing” phase comes after the “Build” phase, but, in the Agile methodology, testing is typically performed concurrently with programming or at least in the same iteration as programming 7. While Waterfall methodology is an internal process and does not require the participation of customers, the Agile software development approach focuses on customer satisfaction and thus, involves the participation of customers throughout the development phase. Integrity - Service - Excellence 63

Agile vs Waterfall (3) 8. The Waterfall model can be regarded as a stringently sequential process, however, the Agile methodology is a highly collaborative software development process, thereby leading to better team input and faster problem solving 9. The Waterfall model is best suited for projects which have clearly defined requirements and in which change is not expected at all, while Agile development supports a process in which the requirements are expected to change and evolve. Thus, if you are planning to develop a software that would require frequent overhauls and has to keep up with the technology landscape and customer requirements, Agile is the best approach to follow 10. The Waterfall model exhibits a project mindset and lays its focus strictly on the completion of project development, while Agile introduces a product mindset that focuses on ensuring that the developed product satisfies its end customers, and changes itself as the requisites of customers change Integrity - Service - Excellence 64

Dev. Ops Terminologies Integrity - Service - Excellence

Dev. Ops n Dev. Ops is a software engineering culture and practice that aims at unifying software development (Dev) and software operation (Ops). The main characteristic of the Dev. Ops movement is to strongly advocate automation and monitoring at all steps of software construction, from integration, testing, releasing to deployment and infrastructure management. Dev. Ops aims at shorter development cycles, increased deployment frequency, and more dependable releases, in close alignment with business objectives. n Dev. Ops is NOT ENOUGH! Dev. Sec. Ops is what must be implemented with the cybersecurity stack built-in into the Dev. Ops pipeline. Integrity - Service - Excellence 66

Microservices n A 'microservice' is a software development technique—a variant of the service-oriented architecture (SOA) architectural style that structures an application as a collection of loosely coupled services. n In a microservices architecture, services are fine-grained and the protocols are lightweight. The benefit of decomposing an application into different smaller services is that it improves modularity and makes the application easier to understand, develop, test, and more resilient to architecture erosion. It parallelizes development by enabling small autonomous teams to develop, deploy and scale their respective services independently. n It also allows the architecture of an individual service to emerge through continuous refactoring. Microservices-based architectures enable continuous delivery and deployment. Integrity - Service - Excellence 67

Example of Microservices Architecture Integrity - Service - Excellence 68

Containers (Docker…) No we are not talking about shipping containers! n Platform independence: Build it once, run it anywhere n Resource efficiency and density n Effective isolation and resource sharing n Speed: Start, create, replicate or destroy containers in seconds n Immense and smooth scaling n Operational simplicity n Improved developer productivity and development pipeline (thanks to Dev. Ops) Learn more about containers: https: //medium. freecodecamp. org/a-beginner-friendly-introduction-tocontainers-vms-and-docker-79 a 9 e 3 e 119 b Integrity - Service - Excellence 69

CI/CD Continuous Integration and Continuous Delivery are key parts of the Dev. Ops pipeline. n Continuous integration (CI) is the practice of consolidating all new source code into a shared version control server such as Git. Hub, several times a day. n Continuous delivery (CD) is used to deliver (release) software in short cycles, ensuring that the software can be reliably released at any time. It aims at building, testing, and releasing software with complete automation. The approach helps reduce the cost, time, and risk of delivering changes by allowing for more incremental updates to applications in production. A straightforward and repeatable deployment process is important for continuous delivery. n Extreme programming (XP) adopted the concept of CI and did advocate integrating more than once per day – perhaps as many as tens of times per day. Integrity - Service - Excellence 70

Agile vs Dev. Sec. Ops is the next evolution of agile and builds on the agile principles by adding the following: n Leverages Containers and Microservices concepts. n Leverages Cloud deployment for scalability and prototyping. n Continuous Integration/Continuous Delivery to rapidly prototype, test and deploy. n Leverage A/B testing and canary deployment for rapid feedback loops. n Embed security in the pipeline instead of an after-thought. Integrity - Service - Excellence 71

Automated Testing Automated testing is a key part of Dev. Sec. Ops. It is enabled by multiple tools that measure both test code coverage and test results. They are fully automated and do not require human action. It also enables new concepts like pair programming and peer code review. Agile brings several new models for creating the right tests: Test-driven development (TDD) is a software development process that relies on very short development cycles: requirements are turned into very specific test cases first, then the software is built to pass the tests. Acceptance test–driven development (ATDD) is a development methodology based on communication between the business customers, the developers, and the testers. ATDD encompasses many of the same practices as specification by example, behavior-driven development (BDD), example-driven development (EDD), and support-driven development also called story test–driven development (SDD). All these processes aid developers and testers in understanding the customer's needs prior to implementation and allow customers to be able to converse in their own domain language. Integrity - Service - Excellence 72

Extreme Dev. Ops Risk Management Example – Chaos Monkey (Netflix) Chaos Monkey is a tool invented in 2011 by Netflix to test the resilience of its IT infrastructure. It works by intentionally disabling computers in Netflix's production network to test how remaining systems respond to the outage. Chaos Monkey is now part of a larger suite of tools called the Simian Army designed to simulate and test responses to various system failures and edge cases. As part of the Dev. Ops movement, special attention is paid to the safe operation of computer systems, thus providing a sufficient level of confidence despite frequent releases. By contributing to the Dev. Ops Tool Chain, Chaos Monkey meets the need for continuous testing. They are part of the pattern "Design for failure”, "designed to support failure": a computer application must be able to support the failure of any underlying software or hardware component. https: //en. wikipedia. org/wiki/Chaos_Monkey Integrity - Service - Excellence 73

Kubernetes (Container Orchestration) n Kubernetes is a portable, extensible open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available. n Google open-sourced the Kubernetes project in 2014. Kubernetes builds upon a decade and a half of experience that Google has with running production workloads at scale, combined with best-of-breed ideas and practices from the community. n Kubernetes provides a container-centric management environment. It orchestrates computing, networking, and storage infrastructure on behalf of user workloads. This provides much of the simplicity of Platform as a Service (Paa. S) with the flexibility of Infrastructure as a Service (Iaa. S), and enables portability across infrastructure providers. Integrity - Service - Excellence 74

Kubernetes Benefits n Allows for the deployment of complex applications (ie. Multiple microservices) with a single command. n Allows for rolling updates and rollbacks with no downtime. n Deploys security stack as a side-car container to gain complete visibility, including security posture, MFA enforcement, identity management, logging etc. n Auto-scaling – allows to provision resources dynamically based on use and maximums, saving significant Cloud computing costs n Restarts containers seamlessly and automatically in case of crash or other issue. Integrity - Service - Excellence 75

Helm helps you manage Kubernetes applications — Helm Charts helps you define, install, and upgrade even the most complex Kubernetes application. Charts are easy to create, version, share, and publish — so start using Helm and stop the copyand-paste madness. The latest version of Helm is maintained by the CNCF - in collaboration with Microsoft, Google, Bitnami and the Helm contributor community. n Enable the creation of packages for easy deployment of complex container/Kubernetes based solutions. n Allows for versioning, rolling-update and rollbacks. Integrity - Service - Excellence 76

Dev. Ops Benefits (1) n Implementing Dev. Ops allows organizations to get more done. Dev. Ops promotes teamwork by eliminating silos and encouraging collaboration. Teams that adopt the Dev. Ops model are able to increase lead time, create new features at a faster pace, all while driving innovation and increasing employee engagement and communication. In turn, they are making applications more secure and stable. n When leveraging Dev. Ops and implementing continuous integration and continuous delivery (CI/CD), organizations can see a tremendous improvement in deployment frequency, lead time, detection of cybersecurity vulnerabilities and flaws, mean time to repair and mean time to recovery. Learn more : https: //www. forbes. com/sites/forbestechcouncil/2018/07/19/the-two-biggestdisruptions-to-cybersecurity-since-the-invention-of-the-firewall/ Integrity - Service - Excellence 77

Dev. Ops Benefits (2) n Allows for rapid experimentation and uncertainty while focusing on mission end goals. n Enable rapid prototyping and A/B testing or canary releases. n New services and innovations are made available n Frequency of deployments is increased n Collaboration between various department is increased n Time spent maintaining applications and fixing is greatly reduced n Number of end users who are using organizations’ services is increased n Performance and quality of applications is improved n Time-to-market is reduced n Time needed for testing, development and operations is reduced. Integrity - Service - Excellence 78

Dev. Ops Benefits (3) n Avoids shadow IT by enabling all directorates/divisions/services to reuse the Dev. Sec. Ops pipeline instead of reinvent the wheel and building their own pipeline without supervision n Avoids technical debt by continuously fixing bugs and security issues thanks to automated tests and real-time scans. Integrity - Service - Excellence 79

Dev. Ops Metrics n Mean-time to recovery: shows how long it would take applications in the production stage to recover from failure n Mean-time to production: show long it takes when new code is committed into code repository n Average lead-time: shows how long it would take for a new requirement to be delivered and deployed n Deployment speed: shows how fast you can deploy a new version of the application between staging, test and production n Deployment frequency: shows how often you can deploy a new release into production environment and testing / QA. n Production failure rate: shows how often software fails during production Integrity - Service - Excellence 80

Dev. Sec. Ops Metrics n Ability to detect and prevent security flaws and injections. n Ability to perform fuzzing and static/dynamic source code analysis. n Ability to monitor container security including container base images and libraries. Integrity - Service - Excellence 81

Dev. Ops Challenges (1) 1. Culture Solution: Organizations should focus on building a collaborative culture with shared goals. This also includes finding employees who are Dev. Ops champions in the organization. 2. Test automation Solution: Many organizations neglect test automation while focusing on CI/CD deployments. Continuous testing is key for Dev. Ops success, and security must be considered from the outset. 3. Legacy systems Solution: Include modeling for legacy infrastructure and applications in your Dev. Ops plans. Installing new hardware or software to coexist with older systems is always difficult. 4. App complexity Solution: Consider application architecture changes based on on-premises, cloud, and containers early on in the process. 5. No Dev. Ops plan Solution: Create a clear plan that includes milestones, project owners, and well-defined deliverables. Integrity - Service - Excellence 82

Dev. Ops Challenges (2) 6. Managing environments Solution: Your organization can standardize and automate complex Dev. Ops environments with cloud sandboxes and other tools. 7. Skillset Solution: Teams need training on Dev. Ops. Organizations should standardized processes and establish common operational procedures. 8. Budget Solution: Remember that opensource does not mean free, and factor in integration and operational complexity in your costs. 9. Tools Solution: Avoid fragmented toolset adoption, which can add to your costs. 10. Leadership support Solution: Educate leadership about the benefits of Dev. Ops, in order to gain resource and budget support. Integrity - Service - Excellence 83

Recommended Books n A Seat at the Table – by Mark Schwartz (former CIO of USCIS, leader in Agile) This book is highly recommended for ALL leadership as it is not technical but focused on the challenges around business, procurement and how leadership can enable Dev. Ops across the organization and remove impediments. n The Phoenix Project – by the founders of Dev. Ops n The Dev. Ops Handbook – by Gene Kim, Patrick Debois. For those who drive to work like me (for hours), please note that these books are available as Audiobooks. Integrity - Service - Excellence 84

Thank You! Nicolas Chaillan Chief Software Officer, U. S. Air Force af. cso@us. af. mil Integrity - Service - Excellence