ISTIO ENVOY Security For Security Subcommittee Micro Service
ISTIO & ENVOY – Security For Security Subcommittee
Micro Service Architecture - Background Internet SLB/ADC/API-Gateway ES EM EM IM 1 IM 2 IM 4 Image IM 3 IM 5 IM 1 IM 2 IM 4 Image Monolithic Application IS 1 IM 3 IM 5 ES IS 1 IS 2 IS 4 IS 5 IS 3 Micro Service based In monolithic application, entire application is scaled-out, waste of memory, no rolling upgrades, massive integration time, big maintenance. Micro Service architecture: • Each individual module is a micro service • Each module can scale-out independently • Typically each micro service is implemented as container 2
Micro Service Architecture - Challenges • Need for load balancing among internal hierarchy Internet • • • Special load balancers – Expensive Each consumer balancing its connections across producers. Independent LB decision. May load one instance of producers. Special logic at each service. • Need for discovery of service instances SLB/ADC/API-Gateway • Special logic at the each service. • Need for health checks ES IS 1 • Need for tracing/visibility IS 2 IS 3 • IS 4 IS 5 Special logic at each service • Need for network firewall capability • IS 4 Special logic at each service • Authenticate and Authorize consumers • Special logic at each service level • Need for Mutual TLS among producers and consumers. • Special logic to get certificate enrolled. • Special logic to initiate/accept TLS connections 3
Micro Service Architecture – Development Challenges • Polyglot • Any special logic at each service level need to support multiple language bindings. • Coordination among all development teams • Third party micro services (binary) – Can’t put our special logic there. • In some instances, it was found that 50% of the logic is related to non-application specific. Internet SLB/ADC/API-Gateway ES IS 1 IS 2 IS 4 IS 5 IS 3 Let application service developers not worry about micro-service tax. 4
Micro Service Architecture – Service Mesh technology Internet Istio-ingress envoy IS 1 ES ES ES SC SC ISTIO Control Plane Pilot/Mixer/ CA SC SC SC IS 4 IS 5 SC IS 2 SC IS 3 SC (Side Car) – One for each Service instance (A container in each POD) - Envoy proxy - Service discovery, Load balancing, Failure handling, Circuit breaking (Limits), Fault injection (for troubleshooting), Health checks - Mutual TLS ISTIO Control plane: - Pilot: - Service discovery glue between Envoy and K 8 S - Traffic rule configuration - Security - Role based Access control (Pluggable RBAC engine, support for local RBAC, K 8 S RBAC adapters, but facility to add new adapters – Example AAF RBAC adapter) - Certificate Authority 5
ONAP - Non-ISTIO to ISTIO ONAP SDC SO OOF SDNC APPC MUSIC AAF • • Certificate Enrolment client Authentication & Authorization Service Discovery/registration TLS, Visibility etc… Non ISTIO DCAE MC VFC POLICY DB ONAP SDC SO OOF SDNC APPC ISTIO with Envoy (CNCF) DCAE MC VFC POLICY DB 6
Service to Envoy proxy – Security Concerns & Mitigation K 8 S POD 1 K 8 S POD 2 Name space 2 SVC 1 Instance 1 Envoy Name space 1 IP Tables Tproxy TLS SVC 2 Instance 1 Name space 1 IP tables tproxy K 8 S POD 2 Name space 2 Linux Kernel ISTIO Pilot configures IP Tables to redirect the traffic coming from services to Envoy and Envoy to services. No configuration required at the service level to use Envoy as proxy Envoy acts as transparent proxy. Ensure that Source IP of the connection is maintained on both sides. Even though traffic is clear between instance and side car, it is not considered a security issue as both side car and service belong to same POD, hence same network name space. Clear traffic is never seen on the wire.
ISTIO CA - Features • Can generate self signed CA root certificate • Can take externally generated CA certificate and private key • Support for certificate chain (CA certificate signed by Intermediate CA) • CA redundancy • Two types of user certificate issue methods • Native K 8 S based for K 8 S orchestrated services • Non-K 8 S orchestrated services (VMs/Baremetal) • Certificate renewal • SPIFFE naming convention 1. Create Service account and Deploy service K 8 S Master 2. POD deploy ment event ISTIO CA 4. Send key/cert using secret mount SVC ISTIO CA 3. Generate Cert, Sign using CA key Envoy: 2. Send CSR - Auto reload of renewed certificates/keys 4. Send Cert - Integration with PKCS 11 based security Node agent - Authentication – Mutual TLS 1. Key pair generation and VM/Baremetal - Authorization - ISTIO RBAC or adapters to CSR remote RBAC SC 3. Create user key/certificate (signed by CA cert)
Casablanca – ISTIO work • Rancher to instantiate ISTIO software components • Status : Currently manual installation of ISTIO components is done. • Migration path from NON-ISTIO to ISTIO • ISTIO Side car injection related changes • Securing the private keys of ISTIO CA and Envoy private keys. • Provide option to use either ISTIO RBAC and AAF CA RBAC – A way to choose the option (AAF CA RBAC is Stretch goal) • Any glue/tools to simplify the ISTIO deployment Any glue logic and scripts – Put it as part of MSB project. Goal – Entire ONAP using ISTIO all the time (No option to choose). Fix any gaps to make that happen.
HW Security - Casablanca work ISTIO CA Go Crypto 11 (PKCS 11 Interface) SSHSM TPM /SGX plugin Hardware • Secure CA private key • Distribute CA private key securely to the Hardware (TPM/SGX) – Started conversations with ISITO security group. Extend the work done on AAF CA to ISTIO CA – Note : Current AAF CA is only testing purpose only. Envoy proxy Open. SSL Lib. P 11 • Secure user private key • Envoy takes care of RBAC. • ISTIO RBAC is quite powerful, but if lacks features, can add new adapter to talk to ONAP RBAC engine (AAF) SSHSM TPM /SGX plugin Hardware Since Mutual TLS/RBAC is implemented in one way (C++ in case of envoy), provides opportunity to do good job of HW security & Accelerating TLS, thereby universal security and improving performance
Summary • Service mesh technologies take away the tax of Micro-Service architecture out of services and consolidate in side cars. - Less error prone Faster development Easy maintenance Polyglot independent. • K 8 S native support • Security is one big piece of Service mesh - Enable Mutual TLS - Enable RBAC - Enables security without each developer putting effort independently in their projects - Provides opportunities to provide HW security and acceleration with one implementation. 11
Inter working with AAF
Overlap items • CA - AAF has CA and is being used by some services in Beijing release. - ISTIO has its own CA. • RBAC - AAF has RBAC and some services in Beijing release is already using AAF RBAC. - ISTIO has RBAC, which is simpler (Need to see if is good enough to satisfy ONAP requirements) • Three choices (Assuming that ISTIO Service mesh is chosen) • Choice 1: Use ISTIO CA and ISTIO RBAC across all ONAP services (Presented before) • Choice 2: Use ISTIO CA and use AAF RBAC • Choice 3: Use ISTIO CA and AAF CA together and use AAF RBAC (Being shown in next slide as it is superset) 13
Choice 3 (Keeping services that use AAF CA ) 1 CA Key Generation Station (Secure system) 2 2 ISTIO CA AAF CA ISTIO Mixer 5 S 1 AAF RBAC 3 3 Side car AAF RBAC Plugin 6 5 CADI 4 S 2 7 No implication (based on current understanding) on existing services that are using CADI for authentication & authorization Additional RBAC plugin development at the ISTIO is required 1. ONAP Administrator creates CA key pair and selfsigned CA certificate (or it could be CA key pair and signed by root/intermediate CA) 2. ONAP Administrator using CA CLI/GUI, upload the CA private key/certificate and any chain in both ISTIO CA instances and AAF CA instances. 3. When S 1 or S 2 is brought up by OOM, corresponding CA will issue service certificates. Services get its own key/certificate & CA chain. 4. TLS establishment : Based on peer ID, service (or its side car) authenticates using CA certificate it has. 5. Authorization: When there is HTTP operation, its method, URL and other operation specific information, requester ID is sent to the RBAC engine to figure out the roles and permissions. 6. RBAC plugin talks to AAF RBAC to get the permissions (in case of ISTIO) 7. If permissions are allowed for that operation, operation proceeds. Otherwise, operation is denied. 14
Recommendation • Assumptions: • AAF RBAC is MUST • Recommendation for default setup • ISTIO CA • ISTIO Mixer with AAF RBAC plugin • Options: • If some deployment likes to use “ISTIO RBAC”, it should be possible. • If some deployment likes to use their own RBAC, ISTIO provides that option. 15
s
- Slides: 16