Richard Seroter Director of Product Management Google Cloud
- Slides: 32
Richard Seroter Director of Product Management, Google Cloud When to Use What Integration Offering: 2020 Edition
Integrators face too many choices Choice is good! Tyranny of small decisions can overwhelm us “The choice of when to be a chooser may be the most important choice we have to make” Accept voluntary constraints, use rules and standards to limit decisions we face @rseroter
We want to be confident that we used the right integration technology to solve a given problem. That’s not easy. @rseroter
Review technology trends that matter to integrators Access decision criteria for choosing an integration technology Look at today’s options for integration @rseroter
Tech Trends That Matter @rseroter
Tech Trends That Matter MULTI-CLOUD 80% Using multiple providers today Source: A Guidance Framework for Managing Vendor Lock-In Risks in Cloud Iaa. S (Gartner) @rseroter
Tech Trends That Matter KUBERNETES 60% Enterprises using Kubernetes Source: The State of Kubernetes 2020 (VMware) @rseroter
Tech Trends That Matter MICROSERVICES 52% Companies using microservice concepts, tools, and methods Source: Cloud Adoption in 2020 (O’Reilly) @rseroter
Tech Trends That Matter MANAGED SERVICES >80% By 2024, enterprises shift from Iaa. S migration to managed platforms Source: Evolve Your Infrastructure and Operations Organization to Remain Relevant in the Cloud Era (Gartner) @rseroter
Tech Trends That Matter LOW CODE 65% Of app development activity in 2024 will be low-code Source: Predicts 2020: The Future of Application Architecture, Development, Integration and Platforms @rseroter
Tech Trends That Matter EDGE COMPUTING >50% By end of 2021, large enterprises will deploy one edge use case Source: Exploring the Edge: 12 Frontiers of Edge Computing (Gartner) @rseroter
Tech Trends That Matter OPEN SOURCE 55 32 Percentage decrease from 2019 to 2022 of proprietary software used in the enterprise Source: The State of Enterprise Open Source (Red Hat) @rseroter
Tech Trends That Matter MODERNIZATION 59% Moving forward with app modernization efforts Source: The State of IT Modernization 2020 (IDC) @rseroter
Decision Criteria @rseroter
Decision Criteria Is this an event stream or series of messages? Batch or real-time needs? How latency-sensitive is the data? Are there Qo. S requirements like guaranteed delivery? 8 PAYLOAD CHARACTERISTICS @rseroter
Decision Criteria Are we talking to on-premises systems? Integrating with public cloud? 7 ENDPOINT TYPE Is the endpoint using a fixed API? Are we creating the integrated endpoint? @rseroter
Decision Criteria Does the integration require a series of processing steps? Is control flow logic needed? 6 WORKFLOW NEEDS Is the logic distinct from the transport channels? Is there a high volume of concurrent workflows? @rseroter
Decision Criteria How often do message definitions change? Do endpoints change often? 5 RATE OF CHANGE What’s the frequency of adding new input sources? Do components change together, or separately? @rseroter
Decision Criteria Does the solution run in containers? Will the system never move? 4 PORTABILITY Is the software based on open source? How do devs get independent access? @rseroter
Decision Criteria Do app teams or ops teams own the integration? How complicated are deployments? 3 OPS BURDEN How automation-friendly is the integration? How much work to upgrade and patch? @rseroter
Decision Criteria Are there many developers who know the technology? What’s the learning curve? 2 AVAILABLE SKILLS Is the ecosystem of tools and components mature? What is the support system for users? @rseroter
Decision Criteria Does this improve speed-tomarket and lower costs? Is vendor support available? 1 STRATEGIC VALUE Does this build on existing investments? Does this align with company goals? @rseroter
Integration Technology Options @rseroter
Integration Technology Options Supports batch, bulk processing Technology Supports Real-time, low-latency processing Queuing Engine Offers delivery guarantees, Qo. S Description Connectivity to cloud-hosted endpoints Centralized broker that decouples senders and receivers of messages for publish/subscribe scenarios with transient data. Offers durability, message filtering, delivery policies, and more. Connectivity to on-premises endpoints Enables short and long-running workflows Supports CI/CD of components, configs Container-friendly Example Products Offers automated operations Rabbit. MQ, Google Cloud Pub/Sub, Azure Service Bus, Amazon SQS, Apache Active. MQ Matches a widely available skillset @rseroter
Integration Technology Options Supports batch, bulk processing Technology Supports Real-time, low-latency processing Event Stream Processor Offers delivery guarantees, Qo. S Description Connectivity to cloud-hosted endpoints Based on a durable, persistent log. There’s no routing of data, as consumers do nondestructive reads from the shared log. This solution scales well and handles highthroughput. Example Products Apache Kafka, AWS Kinesis, Azure Event Hubs Connectivity to on-premises endpoints Enables short and long-running workflows Supports CI/CD of components, configs Container-friendly Offers automated operations Matches a widely available skillset @rseroter
Integration Technology Options Supports batch, bulk processing Technology Supports Real-time, low-latency processing Workflow, Data Pipeline Engine Offers delivery guarantees, Qo. S Description Connectivity to cloud-hosted endpoints Execute a series of steps to automate a business process or manipulate a stream of data. Often offers a connector library. May provide control flow and statefulness. Connectivity to on-premises endpoints Enables short and long-running workflows Supports CI/CD of components, configs Container-friendly Example Products Offers automated operations Google Cloud Composer, Azure Logic Apps, Spring Cloud Data Flow Matches a widely available skillset @rseroter
Integration Technology Options Supports batch, bulk processing Technology Supports Real-time, low-latency processing Integration Broker Offers delivery guarantees, Qo. S Description Connectivity to cloud-hosted endpoints Full-featured engine that offers message routing, connectors, orchestration, and B 2 B tooling. Designed for steady streams of data versus bulk traffic. Connectivity to on-premises endpoints Enables short and long-running workflows Supports CI/CD of components, configs Container-friendly Example Products Microsoft Biz. Talk Server, Oracle ESB, TIBCO Offers automated operations Matches a widely available skillset @rseroter
Integration Technology Options Supports batch, bulk processing Technology Supports Real-time, low-latency processing ETL/ELT Tools Offers delivery guarantees, Qo. S Description Connectivity to cloud-hosted endpoints Move large amounts of data through customized pipelines. Data is transformed so that it integrates with target system. May offer data sorting, joining, aggregation, validation, and more. Connectivity to on-premises endpoints Enables short and long-running workflows Supports CI/CD of components, configs Container-friendly Example Products Offers automated operations Google Cloud Data Fusion, Informatica, Talend, Azure Data Factory Matches a widely available skillset @rseroter
Integration Technology Options Supports batch, bulk processing Technology Supports Real-time, low-latency processing Function-as-a-Service Platforms Offers delivery guarantees, Qo. S Description Connectivity to cloud-hosted endpoints Serverless compute that responds to events and may scale to zero instances when there’s no load. Instances are usually short-lived and stateless. Connectivity to on-premises endpoints Enables short and long-running workflows Supports CI/CD of components, configs Container-friendly Example Products Offers automated operations Google Cloud Run, Azure Functions, AWS Lambda, Knative, Open. Faa. S Matches a widely available skillset @rseroter
Integration Technology Options Supports batch, bulk processing Technology Supports Real-time, low-latency processing API Gateway Offers delivery guarantees, Qo. S Description Connectivity to cloud-hosted endpoints API mediator that handles routing, protocol and security translation, service aggregation, rate limiting, caching, monetization, and more. Connectivity to on-premises endpoints Enables short and long-running workflows Supports CI/CD of components, configs Container-friendly Example Products Offers automated operations Azure API Management, Google Cloud Endpoints, Mulesoft, Amazon API Gateway Matches a widely available skillset @rseroter
Integration Technology Options Supports batch, bulk processing Technology Supports Real-time, low-latency processing Communication Frameworks/Protocols Offers delivery guarantees, Qo. S Description Connectivity to cloud-hosted endpoints Advanced RPC communication between components that may offer load balancing, backpressure mechanisms, and bi-directional streaming. Connectivity to on-premises endpoints Enables short and long-running workflows Supports CI/CD of components, configs Container-friendly Example Products RSocket, g. RPC Offers automated operations Matches a widely available skillset @rseroter
You will use multiple integration tools. Choose when (and how) to choose. Thank you! @rseroter
