Approach Use reuse exiting TOSCA normative node capability
Approach: • Use reuse exiting TOSCA normative node, capability and relationship types where possible • Model Kubernetes types (for now), then model similar container managers like Swarm, etc. and look for common base types, properties that can be abstracted. Kubernetes Analysis: 2 types of containers “Dumb” (no HA, no Autoscale) = Pod Template “Smart” (HA, Scaling) = Replication. Controller Template kind: “Pod” (i. e. Type) kind: “Replication. Controller” (i. e. Type) id: redis-master labels: kind: Pod name: redis api. Version: v 1 beta 1 role: master desired. State: redis-sentinel: "true" manifest: version: v 1 beta 1 id: redis-master containers: - name: master image: kubernetes/redis: v 1 cpu: 1000 ports: - container. Port: 6379 volume. Mounts: - name: data mount. Path: /redis-master-data env: - key: MASTER value: "true" - name: sentinel image: kubernetes/redis: v 1 ports: - container. Port: 26379 env: - key: SENTINEL value: "true" volumes: - name: data source: empty. Dir: {} id: redis kind: Replication. Controller api. Version: v 1 beta 1 desired. State: replicas: 1 replica. Selector: name: redis pod. Template: desired. State: manifest: version: v 1 beta 1 id: redis containers: - name: redis image: kubernetes/redis: v 1 cpu: 1000 ports: - container. Port: 6379 volume. Mounts: - name: data mount. Path: /redis-master-data volumes: - name: data source: empty. Dir: {} labels: name: redis
• Kubernetes Analysis: Pod Modeling: TOSCA Type mapping A Pod is an aggregate of Docker Container Requirements of 1. . N homogenous Container (topologies) “Redis-master” Template of Kubernetes “Pod” Type: kind: “Pod” (a Template of type “Pod”) id: redis-master kind: Pod labels: api. Version: v 1 beta 1 name: redis desired. State: role: master manifest: redis-sentinel: "true" version: v 1 beta 1 (non-numeric) id: redis-master containers: ------------------------------------------------- name: master (TOSCA template name) image: kubernetes/redis: v 1 (TOSCA Container. App; create artifact of type image. Docker) cpu: 1000 (TOSCA Container capability; num_cpus, cpu_frequency) ports: (TOSCA End. Point capability) - container. Port: 6379 (TOSCA Endpoint; port, ports) volume. Mounts: (TOSCA Attachment capability) - name: data mount. Path: /redis-master-data (TOSCA Attaches. To Rel. ; location) env: - key: MASTER value: "true” # passed as Envirronment vars to instance ------------------------------------------------ name: sentinel image: kubernetes/redis: v 1 ports: - container. Port: 26379 env: - key: SENTINEL value: "true” # passed as Env. var. ------------------------------------------------volumes: - name: data source: empty. Dir: {} TOSCA Types for Kubernetes: Kubernetes. Pod tosca. groups. Placement derived_from: tosca. groups. Placement version: <version_number> metadata: <tosca: map(string)> description: <description> properties: TBD attributes: TBD # Allow get_property() against targets: [ tosca. nodes. Container. App. Kubernetes ] Kubernetes. Container tosca. nodes. Container. App derived_from: tosca. nodes. Container. App metadata: <tosca: map(string)> version: <version_number> description: <description> properties: environment: <tosca: map of string > requirements: - host: # hosted on kubelets type: Container. Runtime. Kubernetes - ports: capability: End. Point properties: ports, etc. - volumes: capability: Attachment relationship: Attaches. To properties: location, device occurrences: [0, UNBOUNDED]
Kubernetes Analysis: Pod Modeling: TOSCA Template Mapping: Simple “Group Approach”: • Using the Types defined on the previous slide the TOSCA Topology Template looks like this for “redis-master” TOSCA Topology for Kubernetes “: “Redis-master” Template of Kubernetes “Pod” Type: redis-master-pod Kubernetes. Pod kind: “Pod” (a Template of type “Pod”) id: redis-master kind: Pod labels: api. Version: v 1 beta 1 name: redis desired. State: role: master manifest: redis-sentinel: "true" version: v 1 beta 1 (non-numeric) id: redis-master containers: ------------------------------------------------- name: master image: kubernetes/redis: v 1 cpu: 1000 ports: - container. Port: 6379 volume. Mounts: - name: data mount. Path: /redis-master-data env: - key: MASTER value: "true” # passed as Envirronment vars to instance ------------------------------------------------ name: sentinel image: kubernetes/redis: v 1 ports: - container. Port: 26379 env: - key: SENTINEL value: "true” # passed as Env. var. ------------------------------------------------volumes: - name: data source: empty. Dir: {} type: tosca. groups. Placement implied “Invites. To” Relationship version: 1. 0 metadata : implied “Invites. To” Relationship name: redis role: master redis-sentinel: true targets: [ master-container , sentinel-container ] master-container Kubernetes. Container derived_from: Kubernetes. Container metadata: <tosca: map(string)> version: <version_number> description: <description> artifacts: kubernetes/redis: v 1 properties: requirements: - host: properties: num_cpus: 1000 ? - port: capability : End. Point properties: port: 6379 - volume: capability: Attachment relationship: Attaches. To properties: location, device occurrences: [0, UNBOUNDED] interfaces: inputs: MASTER: true sentinel-container Kubernetes. Container derived_from: Kubernetes. Container. . Choice: or use Docker. Runtime type to allow use of template on Swarm, etc. ? Issue: location property lost as there is no “Attaches. To” relationship in the topology. Create new Capability Type? Issue: Are there more than 1 volumes / mount points allowed?
NEW Direction Proposal: Membership (Member. Of) direction is wrong for management (group): Comment: Direction of relationship implies orchestration order TBD: Ratios may be needed: 2 “masters” to 3 “sentinels” master-VM sentinel-VM VM VM derived_from: Kubernetes. Container. . TOSCA Groups (logical) my_compute_node Compute Occurrences: 0, N Occurrences: 1, 1 implied “Member. Of” Relationship (a virtual host container which can be viewed as a virtual “Hosted. On”) type: tosca. groups. Placement # With ratios simple # notation example sources: { [ master-container, 2 ], [ sentinel-container, 3] } implied “Member. Of” Relationship (a virtual host container which can be viewed as a virtual “Hosted. On”) TBD additional requirements on the “host” environment.
However: We do not want to “buy into” Docker file as a Capability Type: Old Style: Docker capability type that mirrors a Dockerfile: tosca. capabilities. Container. Docker: derived_from: tosca. capabilities. Container properties: version: type: list required: false entry_schema: version publish_all: type: boolean default: false required: false publish_ports: type: list entry_schema: Port. Spec required: false expose_ports: type: list entry_schema: Port. Spec required: false volumes: type: list entry_schema: string required: false Instead we want to use Endpoints (for ports) and Attachments (for volumes) This allows Docker, Rocket and containers to be modeled with other TOSCA nodes (i. e. , via Connects. To) and leverage underlying Compute attached Block. Storage TBD: Need to show this
GENERIC IS BETTER: We do not need to define Kubernetes specific Types (use generic Group and Container. App types) : Kubernetes Pod reuses “Docker” Container. App type which can now reference other Container. App types like Rocket (Rkt) tosca. groups. Placement tosca. groups. Root derived_from: tosca. groups. Placement version: <version_number> metadata: <tosca: map(string)> description: <description> properties: TBD attributes: TBD # Allow get_property() against targets: [ Container. App. Docker, Container. App. Rocket, . . . ] Policies: Security, Scaling, Update, etc. • • “Applies. To” group (members) • i. e. , targets • Not using “Binds. To” as that implies it is coupled to an implementation Container. App. Docker tosca. nodes. Container. App derived_from: tosca. nodes. Container. App metadata: <tosca: map(string)> version: <version_number> description: <description> properties: environment: <tosca: map of string > requirements: - compute: # a. k. a. host capability: Container. Docker type: Container. Runtime. Kubernetes - network: # a. k. port(s) capability: End. Point properties: ports, etc. - storage: # a. k. a. volue/attachment capability: Attachment relationship: Attaches. To properties: location, device occurrences: [0, UNBOUNDED] • There is no need for a “Kubernetes” Runtime type, just use the real Container’s built-in runtime requirement • (don’t care to model or reference Kubelets) • Homogenous Pods/Containers for Kubernetes is still an issue, but • this is a current Kubernetes limitation • (heterogonous is possible in future) Container. App. Rocket Container. App derived_from: Kubernetes. Container. . # Potential additional attributes on # Container capability capabilities: Container. App: attribute: response_time:
TOSCA Policy – Entities that compose Policy (Event, Condition, Action) model Policy Definition Event Type (new): <event_type_name>: derived_from: <parent_event_type> version: <version_number> description: <policy_description> Event name of a normative TOSCA Event Type Condition described as a constraint of an attribute of the node (or capability) identified) by the filter. Action Describes either: a)a well-known strategy b)an implementation artifact (e. g. , scripts, service) to invoke with optional property definitions as inputs (to either choice) <policy_name>: type: <policy_type_name> description: <policy_description> properties: <property_definitions> # allowed targets for policy association targets: [ <list_of_valid_target_templates> ] * triggers: <trigger_symbolic_name_1>: event: <event_type_name> # TODO: Allow a TOSCA node filter here # required node (resource) to monitor target_filter: node: <node_template_name> <node_type> # Used to reference another node related to # the node above via a relationship requirement: <requirement_name> # optional capability within node to monitor capability: <capability_name> # required clause that compares an attribute # with the identified node or capability # for some condition: <constraint_clause> action: # a) Define new TOSCA normative strategies # per-policy type and use here OR # b) allow domain-specific names <operation_name>: # (no lifecycle) # TBD: Do we care about validation of types? # If so, we should use a TOSCA Lifecycle type description: <optional description> inputs: <list of property assignments > implementation: <script> | <service_name> <trigger_symbolic_name_2>: . . . <trigger_symbolic_name_n>:
Possible TOSCA Metamodel and Normative Type additions Node. Type, Rel. Types <node_type_name>: metadata: description: > allow tags / labels for search of instance model type: map of string derived_from: <parent_node_type_name> version: <version_number> description: <node_type_description> properties: <property_definitions> attributes: <attribute_definitions> requirements: - <requirement_definitions> capabilities: <capability_definitions> interfaces: <interface_definitions> artifacts: <artifact_definitions> tosca. capabilities. Container: derived_from: tosca. capabilities. Root properties: num_cpus: type: integer required: false constraints: - greater_or_equal: 1 cpu_frequency: type: scalar-unit. frequency required: false disk_size: type: scalar-unit. size required: false mem_size: type: scalar-unit. size required: false attributes: utilization: description: referenced by scaling policies type: # float (percent) | integer (percent) | # scalar-percent ? required: no ? constraints: - in_range: [ 0, 100 ]
TOSCA Policy Mapping – Example Senlin “scaling_out_policy_ceilometer. yaml” using the Kubernetes “redis” example from earlier slides (and its pod, container): TOSCA Policy Definition Symbolic name for the trigger (could be used to reference an externalized version; however, this would violate a Policy’s integrity as a “Security document” Target is a Kubernetes Pod of the tosca. groups. placement type my_scaling_policy: type: tosca. policies. scaling properties: # normative TOSCA properties for scaling min_instances: 1 max_instances: 10 default_instances: 3 increment: 1 # target the policy at the “Pod” targets: [redis-master-pod ] triggers: resize_compute: # symbolic name event: tosca. events. resource. utilization target_filter: node: master-container requirement: host Describe NODE to attach an capability: Container alarm | alert | event to condition: utilization greater_than 80% action: # map to SENLIN: : ACTION: : RESIZE i. e. , Using the “node”, “req”, scaleup: # logical operation name “cap” and “condition” keys inputs: # optional inputs parameters would expressed as a number: 1 descriptive “filter” strategy: BEST_EFFORT Implementation: <script> | <service_name>. . . List optional input parms. here TOSCA normative event type (name) that would map to domain-specific names (e. g. , Open. Stack Ceilometer) Find the attribute via the topology: a) Navigate to node (directly or via the requirement name) and optionally the Capability name b) The condition to map & register with the target monitoring service (e. g. , Ceilometer) TODO: Need a % data type for TOSCA Note: we combined the Senlin “Action” of SENLIN: ACTION: RESIZE with the strategy: BEST_EFFORT to have one name
Senlin to TOSCA Policy Mapping – scaling_out_policy_ceilometer. yaml Senlin WIP Definition type: Scaling. Policy version: 1. 0 alarm: type: SENLIN: : ALARM: : CEILOMETER properties: meter: cpu_util op: gt threshold: 50 period: 60 evaluations: 1 repeat: True schedule: start_time: "2015 -05 -07 T 07: 00 Z" end_time: "2015 -06 -07 T 07: 00 Z" handlers: - type: webhook action: SENLIN: : ACTION: : RESIZE params: type: CHANGE_IN_CAPACITY number: 1 strategy: BEST_EFFORT credentials: user: john password: secrete - type: email addresses: - joe@cloud. com Type-Version Use TOSCA types and versions Alarm Is implementation specific and should not be in the policy itself. It should use intelligent defaults for different Impls. or ones set by a user configuration file Schedule TOSCA could add these fields as optional “properties” on the base tosca. policies. Root type handler Describe these in the “action: ” section of the TOSCA policy Credentials should never be in a policy definition TOSCA Normative properties (for any scaling policy) TOSCA Definition my_scaling_policy: type: tosca. policies. scaling properties: # normative TOSCA properties for scaling min_instances: 1 TOSCA Normative max_instances: 10 a) Find out what node default_instances: 3 (resource) to place “alar increment: 1 onto # target the policy at the “Pod” targets: [redis-master-pod ] b) Find out what condition triggers: set into alarm on that compute_trigger: # symbolic name event: tosca. events. resource. utilization tim target_filter: -n node: master-container s requirement: host d capability: Container d condition: utilization greater_than 80% t action: # map to SENLIN: : ACTION: : RESIZE scaleup: # logical operation name rul inputs: # optional input parameters m number: 1 # Should use “increment” property co strategy: BEST_EFFORT | LEAST_USED Implementation: Senlin. resize <webhook name> TOSCA Action a) Maps to a “well-known” (perhaps normative) strategy name. Could use Senlin names. • (logical operation == strategy name) b) Maps to an implementation; could be a Senlin web hook c) Options for how “strategy” to be passed into the “resize” function/operation. Another is to put the strategy as the operation name not as input parameter. th pe ev st qu -
Senlin to TOSCA Policy Mapping – placement_rr. yaml Senlin WIP Definition # Sample placement policy doing round-robin type: senlin. policy. placement version: 1. 0 description: A policy for node placement scheduling. properties: availability_zones: # Valid values include: # ROUND_ROBIN, WEIGHTED, SOURCE strategy: ROUND_ROBIN candidates: - AZ 1 - AZ 2 regions: # Valid values include: # ROUND_ROBIN, WEIGHTED, SOURCE strategy: ROUND_ROBIN candidates: - Region. One - Region. Two Region and AZ really describe 2 Policies • They are related. • But, customers want to describe these separately (2 different types of logical containers) as one is based upon security, the other based upon logical location. • Each can be combined by policy service (as they have the same “target(s)” TOSCA Definitions (Treat each placement policy as independent) my_zone_placement_policy: type: tosca. policies. placement properties: container type: zone container_number: N/A # unused in this use case containers: [ AZ 1, AZ 2] targets: [<node/group to be placed>] triggers: placement_trigger_1: # symbolic name event: tosca. events. resource. schedule target_filter: N/A # use declared “targets” condition: N/A # trigger based upon event only action: # map to SENLIN service / strategy schedule: # logical operation name strategy: ROUND_ROBIN weighting_map: { [ AZ 1: 2 ], [ AZ 2: 1 ] } implementation: Senlin. service. placement + my_region_placement_policy: type: tosca. policies. placement properties: container type: region containers: [ region. One, region. Two ] targets: [<node/group to be placed>] triggers: placement_trigger_2: event: tosca. events. resource. schedule action: # map to SENLIN service / strategy schedule: # logical operation name strategy: ROUND_ROBIN implementation: Senlin. service. placement = container(s) to place on
Senlin to TOSCA Policy Mapping – Load Balancing is a TOSCA app. requirement, fulfilled by Load. Balancer node when orchestrated. # Load-balancing policy spec using Neutron LBaa. S service type: senlin. policy. loadbalance version: 1. 0 description: A policy for load-balancing the nodes in a cluster. properties: pool: # Protocol used for load balancing protocol: HTTP # Port on which servers are running on the members protocol_port: 80 # Name or ID of subnet for the port on which members can be # connected. subnet: private-subnet # Valid values include: ROUND_ROBIN, LEAST_CONNECTIONS, SOURCE_IP # This can be the “algorithm” property in LB capability lb_method: ROUND_ROBIN session_persistence: # type of session persistence, valid values include: # SOURCE_IP, HTTP_COOKIE, APP_COOKIE # ? ? What Open. Stack resource uses this, neutron? ? type: SOURCE_IP # Name of cookie if type set to APP_COOKIE cookie_name: whatever vip: # Name or ID of Subnet on which VIP address will be allocated subnet: public-subnet # IP adddress of the VIP # address: <ADDRESS> load # Max #connections per second allowed for this VIP connection_limit: 500 # Need to add this to TOSCA as property # Protocol used for VIP protocol: HTTP # TCP port to listen on protocol_port: 80 Cluster – • a specialized TOSCA Group • with Load. Balancing Requirement • Fulfilled by a Load. Balancer node when realized) • Specification section 5. 8. 12 tosca. nodes. Load. Balancer some_cluster tosca. groups. Placement type: tosca. groups. Placement sources: [ member ] requirements: load_balancer: type: tosca. capabilities. Load. Balancer occurrences: [0, UNBOUNDED] client_endpoint: type: tosca. capabilities. Endpoint tosca. capabilities. Load. Balancer : derived_from: tosca. capabilities. Root properties: algorithm : <string> session_type: # Session/cookie stuff could go here if needed. connection_limit: <integeer> tosca. capabilities. Endpoint: # see sectioon 5. 4. 4. 1 derived_from: tosca. capabilities. Root properties: protool : <string> port: Port. Def secure: boolean url_path: string port_name: string network_name: PUBLIC | PRIVATE | <name> initiator: string ports: map of Port. Def http: //git. openstack. org/cgit/stackforge/senlin/tree/examples/policies/lb_policy. yaml
Backup
Software Component 1: SW Component with VM image deployment artifact master Container. App. Kubernetes redis_master Properties • TBD Attributes • TBD Kubernetes. Pod Properties • TBD Attributes • TBD Artifacts: • kubernetes/redis: v 1 • type: image. Docker Requirements Container. Runtime. Docker Properties • TBD Lifecycle. Standard Hosted. On create … Capabilities Container. Runtime. Docker sentinel Hosted. On Container. App. Kubernetes Properties • TBD Attributes • TBD Requirements Artifacts: • kubernetes/redis: v 1 • type: image. Docker
‘wordpress’ Docker Image Docker Hub (Repository) wordpress_container ‘mysql’ Docker Image mysql_container Container. Application. Docker artifacts: - my_image: type: Image. Docker URI: wordpress repository: docker_hub - my_image: type: Image. Docker URI: mysql repository: docker_hub Requirements Container Runtime. Docker. Links. To Capabilities Docker. Link
- Slides: 15