An Overview of Cloud Computing Yahoo Raghu Ramakrishnan

An Overview of Cloud Computing @ Yahoo! Raghu Ramakrishnan Chief Scientist, Audience and Cloud Computing Research Fellow, Yahoo! Research Reflects many discussions with: Eric Baldeschwieler, Jay Kistler, Chuck Neerdaels, Shelton Shugar, and Raymie Stata and joint work with the Sherpa team, in particular: Brian Cooper, Utkarsh Srivastava, Adam Silberstein, Rodrigo Fonseca and Nick Puz in Y! Research Chuck Neerdaels, P. P. Suryanarayanan and many others in CCDI 1

Questions • What is cloud computing? – Horizontal and functional services • What’s it going to change? – Software business models, science, life • How many clouds will there be? – 1, 2, 3, infinity • What’s new in cloud computing? – HPC grids, ASPs, hosted services, Multics (!) – Emerging “cloud stack” to support a broad class of programs, including data intensive applications 2

Pie-in-the-sky SCENARIOS 3

Living in the Clouds • We want to start a new website, Freds. List. com • Our site will provide listings of items for sale, jobs, etc. • As time goes on, we’ll add more features – And illustrate how more cloud capabilities (and corresponding infrastructure components) are used as needed • List of capabilities/components is illustrative, not exhaustive • Our cloud provides a “dataset” abstraction – Freds. List doesn’t worry about the underlying components 4

Step 1: Listings Scenario Freds. List wants to store listings as (key, category, description) Freds. List. com application 1234323, transportation, For sale: one bicycle, barely used 5523442, childcare, Nanny available in San Jose 215534, wanted, Looking for issue 1 of Superman comic book DECLARE DATASET Listings AS ( ID String PRIMARY KEY, Category String, Description Text ) Simple Web Service API’s PNUTS Database 5

Step 2: System Evolution Fred belatedly realizes prices are useful information! Freds. List. com application 1234323, transportation, For sale: one bicycle, barely used 5523442, childcare, Nanny available in San Jose ALTER DATASET Listings ADD (Price Float) 215534, wanted, Looking for issue 1 of Superman comic book 32138, camera, Nikon D 40, USD 300 Simple Web Service API’s vs. PNUTS Database Not every record in a dataset has values defined for all fields declared for the dataset Schemas are flexible, and evolve 6

Step 3: Search Federation of systems offering different capabilities Freds. List’s customers quickly ask for keyword search Freds. List. com application “dvd’s” “bicycle” “nanny” ALTER Listings SET Description SEARCHABLE Simple Web Service API’s PNUTS Vespa Database Search Tribble Messaging 7

Step 4: Photos Freds. List decides to add photos/videos to listings Federation of systems offering different performance points Freds. List. com application ALTER Listings ADD Photo BLOB Simple Web Service API’s PNUTS Foreign key MOb. Stor photo → listing Storage Database Vespa Search Tribble Messaging 8

Step 5: Data Analysis Freds. List wants to analyze its listings to get statistics about category, do geocoding, etc. Freds. List. com application Pig query to analyze categories Hadoop program to generate fancy pages for listings Hadoop program to geocode data ALTER Listings MAKE ANALYZABLE Simple Web Service API’s Grid PNUTS Foreign key MOb. Stor Vespa photo → listing Compute Database Batch export Storage Search Tribble Messaging 9

Step 6: Performance Freds. List wants to reduce its data access latency And by now, Fred is global, and wants georeplication! Freds. List. com application ALTER Listings MAKE CACHEABLE Simple Web Service API’s Grid PNUTS Foreign key MOb. Stor Vespa memcached photo → listing Compute Database Batch export Storage Search Caching Tribble Messaging 10

Data Serving vs. Analysis • Very different workloads, requirements • Data from serving system is one of many kinds of data (click streams are another common kind, as are syndicated feeds) to be analyzed and integrated • The result of analysis often goes right back into serving system 11

Motherhood-and-Apple-Pie EYES TO THE SKIES 12

Why Clouds? • On-demand infrastructure to create a fundamental shift in the OE curve: – Do things we can’t do – Build more robustly, more efficiently, more globally, more completely, more quickly, for a given budget • Cloud services should do heavy lifting of heavy-lifting of scaling & high-availability – Today, this is done at the applevel, which is not productive 13

Requirements for Cloud Services • Multitenant. A cloud service must support multiple, organizationally distant customers. • Elasticity. Tenants should be able to negotiate and receive resources/Qo. S on-demand. • Resource Sharing. Ideally, spare cloud resources should be transparently applied when a tenant’s negotiated Qo. S is insufficient, e. g. , due to spikes. • Horizontal scaling. It should be possible to add cloud capacity in small increments; this should be transparent to the tenants of the service. • Metering. A cloud service must support accounting that reasonably ascribes operational and capital expenditures to each of the tenants of the service. • Security. A cloud service should be secure in that tenants are not made vulnerable because of loopholes in the cloud. • Availability. A cloud service should be highly available. • Operability. A cloud service should be easy to operate, with few operators. Operating costs should scale linearly or better with the capacity of the service. 14

Types of Cloud Services • Two kinds of cloud services: – Horizontal (“Platform”) Cloud Services • Functionality enabling tenants to build applications or new services on top of the cloud – Functional Cloud Services • Functionality that is useful in and of itself to tenants. E. g. , various Saa. S instances, such as Saleforce. com; Google Analytics and Yahoo!’s Index. Tools; Yahoo! properties aimed at end-users and small businesses, e. g. , flickr, Groups, Mail, News, Shopping • Could be built on top of horizontal cloud services or from scratch • Yahoo! has been offering these for a long while (e. g. , Mail for SMB, Groups, Flickr, BOSS, Ad exchanges) 15

Opening Up Yahoo! Search Phase 1 Giving site owners and developers control over the appearance of Yahoo! Search results. Phase 2 BOSS takes Yahoo!’s open strategy to the next level by providing Yahoo! Search infrastructure and technology to developers and companies to help them build their own search experiences. 16

BOSS Offerings BOSS offers two options for companies and developers and has partnered with top technology universities to drive search experimentation, innovation and research into next generation search. API A self-service, web services model for developers and start-ups to quickly build and deploy new search experiences. CUSTOM Working with 3 rd parties to build a more relevant, brand/site specific web search experience. This option is jointly built by Yahoo! and select partners. ACADEMIC Working with the following universities to allow for wide-scale research in the search field: • University of Illinois Urbana Champaign • Carnegie Mellon University • Stanford University • Purdue University • MIT • Indian Institute of Technology Bombay • University of Massachusetts (Slide courtesy Prabhakar Raghavan) 18

Partner Examples 19

Horizontal Cloud Services • Horizontal cloud services are foundations on which tenants build applications or new services. They should be: – Semantics-free. Must be "generic infrastructure, ” and not tied to specific app-logic. • May provide the ability to inject application logic through well-defined APIs – Broadly applicable. Must be broadly applicable (i. e. , it can't be intended for just one or two properties). – Fault-tolerant over commodity hardware. Must be built using inexpensive commodity hardware, and should mask component failures. • While each cloud service provides value, the power of the cloud paradigm will depend on a collection of well-chosen, loosely coupled services that collectively make it easy to quickly develop and operate innovative web applications. 20

Yahoo! Cloud Stack EDGE Brooklyn Horizontal Cloud Services YCPI … WEB VM/OS Horizontal Cloud Services. PHP y. Apache APP VM/OS Horizontal Cloud Serving Grid Services … STORAGE Sherpa Horizontal Cloud Services… MOBStor App Engine Data Highway Monitoring/Metering/Security Provisioning (Self-serve) YCS BATCH Hadoop Horizontal…Cloud Services 22

Yahoo! CCDI Thrust Areas • Fast Provisioning and Machine Virtualization: On demand, deliver a set of hosts imaged with desired software and configured against standard services – Multiple hosts may be multiplexed onto the same physical machine. • Batch Storage and Processing: Scalable data storage optimized for batch processing, together with computational capabilities • Operational Storage: Persistent storage that supports low-latency updates and flexible retrieval • Edge Content Services: Support for dealing with network topology, communication protocols, caching, and BCP Rest of today’s talk 23

Web Data Management • CRUD • Point lookups • Scan oriented workloads • Focus on sequential disk I/O • $ per cpu cycle Large data analysis (Hadoop) Structured record storage (PNUTS/Sherpa) and short scans • Index organized table and random I/Os • $ per latency • Object retrieval Blob storage (SAN/NAS) and streaming • Scalable file storage • $ per GB 24

Hadoop: Batch Storage/Analysis Why is batch processing important? [Workflow] High-level query layer (Pig) Map-Reduce HDFS • Whether it’s – – response-prediction for advertising machine-learned relevance for Search, or content optimization for audience, data-intensive computing is increasingly central to everything Yahoo! does – Hadoop is central to addressing this need • Hadoop is a case-study in our cloud vision – Processes enormous amounts of data – Provides horizontal scaling and faulttolerance for our users – Allows those users to focus on their app logic 25

The World Has Changed • Web serving applications need: – Scalability! • Preferably elastic – – Flexible schemas Geographic distribution High availability Reliable storage • Web serving applications can do without: – Complicated queries – Strong transactions 26

MOb. Stor • Yahoo!’s next-generation globally replicated, virtualized media object storage service • Better provisioning, easy migration, replication, better BCP, and performance • New features (Evergreen URLs, CDN integration, REST API, …) • The object metadata problem addressed using Sherpa, though MOb. Stor is focused on blob storage. 2727

Storage & Delivery Stack 28

PNUTS / SHERPA To Help You Scale Your Mountains of Data 29

CCDI—Research Collaboration Yahoo! Research CCDI • • • Raghu Ramakrishnan Brian Cooper Utkarsh Srivastava Adam Silberstein Rodrigo Fonseca Chuck Neerdaels P. P. S. Narayan Kevin Athey Toby Negrin Plus Dev/QA teams 30

Yahoo! Serving Storage Problem – Small records – 100 KB or less – Structured records – lots of fields, evolving – Extreme data scale - Tens of TB – Extreme request scale - Tens of thousands of requests/sec – Low latency globally - 20+ datacenters worldwide – High Availability - outages cost $millions – Variable usage patterns - as applications and users change 31 31

What is PNUTS/Sherpa? A B 42342 42521 E W C 66354 W D E 12352 75656 E C F 15677 E Parallel database CREATE TABLE Parts ( ID VARCHAR, Stock. Number INT, Status VARCHAR … ) Structured, flexible schema A B C D E F 42342 42521 66354 12352 75656 15677 E W W E C E Geographic replication Hosted, managed infrastructure 33 33

What Will It Become? A B C D E F 42342 42521 66354 12352 75656 15677 E W W E C E Indexes and views A B C D E F 42342 42521 66354 12352 75656 15677 E W W E C E 35

Design Goals Scalability Consistency • • • Thousands of machines Easy to add capacity Restrict query language to avoid costly queries Per-record guarantees Timeline model Option to relax if needed Geographic replication Multiple access paths • • Asynchronous replication around the globe Low-latency local access Hash table, ordered table Primary, secondary access High availability and fault tolerance Hosted service • • Automatically recover from failures Serve reads and writes despite failures Applications plug and play Share operational cost 36 36

Technology Elements Applications Tabular API PNUTS API YCA: Authorization PNUTS • Query planning and execution • Index maintenance Distributed infrastructure for tabular data • Data partitioning • Update consistency • Replication YDOT FS • Ordered tables YDHT FS • Hash tables Tribble • Pub/sub messaging Zookeeper • Consistency service 37 37

Data Manipulation • Per-record operations – Get – Set – Delete • Multi-record operations – Multiget – Scan – Getrange • Web service (RESTful) API 38 38

Tablets—Hash Table 0 x 0000 Name Description Grapes are good to eat $12 Limes are green $9 Apple is wisdom $1 Strawberry 0 x 2 AF 3 0 x 911 F 0 x. FFFF Strawberry shortcake Price $900 Orange Arrgh! Don’t get scurvy! $2 Avocado But at what price? $3 Lemon How much did you pay for this lemon? $1 Tomato Is this a vegetable? $14 Banana The perfect fruit $2 New Zealand $8 Kiwi 39 39

Tablets—Ordered Table A Name Description Price Apple is wisdom $1 Avocado But at what price? $3 Banana The perfect fruit $2 Grapes are good to eat $12 New Zealand $8 How much did you pay for this lemon? $1 Limes are green $9 H Kiwi Lemon Lime Q Orange Strawberry Tomato Arrgh! Don’t get scurvy! $2 Strawberry shortcake $900 Is this a vegetable? $14 Z 40 40

Flexible Schema Posted date Listing id Item Price 6/1/07 424252 Couch $570 6/1/07 763245 Bike $86 6/3/07 211242 Car $1123 6/5/07 421133 Lamp $15 Color Conditi on Good Red Fair 41

Detailed Architecture Remote regions Local region Clients REST API Routers Tribble Tablet Controller Storage units 42 42

Tablet Splitting and Balancing Each storage unit has many tablets (horizontal partitions of the table) Storage unit may become a hotspot Storage unit Tablet Overfull tablets split Tablets may grow over time Shed load by moving tablets to other servers 43 43

QUERY PROCESSING 44 44

Accessing Data 4 Record for key k 1 Get key k 3 Record for key k SU SU 2 Get key k SU 45 45

Bulk Read 1 {k 1, k 2, … kn} 2 Get k 1 Get k 2 SU SU Get k 3 Scatter/ gather server SU 46 46

Range Queries in YDOT • Clustered, ordered retrieval of records Apple Avocado Grapefruit…Pear? Banana Blueberry Canteloupe Grape Kiwi Lemon Grapefruit…Lime? Lime…Pear? Router Lime Mango Orange Strawberry Apple Tomato Avocado Watermelon Banana Blueberry Storage unit 1 Canteloupe Storage unit 3 Lime Storage unit 2 Strawberry Storage unit 1 Strawberry Tomato Watermelon Storage unit 1 Lime Mango Orange Canteloupe Grape Kiwi Lemon Storage unit 2 Storage unit 3 47

Updates 1 8 Write key k Sequence # for key k Routers Message brokers 3 Write key k 2 7 Sequence # for key k 4 Write key k 5 SU SU SU 6 SUCCESS Write key k 48 48

ASYNCHRONOUS REPLICATION AND CONSISTENCY 49 49

Asynchronous Replication 50 50

Consistency Model • Goal: Make it easier for applications to reason about updates and cope with asynchrony • What happens to a record with primary key “Alice”? Record inserted Update v. 1 Update v. 2 v. 3 v. 4 Update v. 5 v. 6 Generation 1 v. 7 Delete Update v. 8 Time As the record is updated, copies may get out of sync. 51 51

Example: Social Alice East West User Status Alice ___ User Status Alice Busy Alice Free User Status Alice ? ? ? Record Timeline ___ Busy Free 52

Consistency Model Read Stale version v. 1 v. 2 v. 3 v. 4 Stale version v. 5 v. 6 Generation 1 v. 7 Current version v. 8 Time In general, reads are served using a local copy 53 53

Consistency Model Read up-to-date Stale version v. 1 v. 2 v. 3 v. 4 Stale version v. 5 v. 6 Generation 1 v. 7 Current version v. 8 Time But application can request and get current version 54 54

Consistency Model Read ≥ v. 6 Stale version v. 1 v. 2 v. 3 v. 4 Stale version v. 5 v. 6 Generation 1 v. 7 Current version v. 8 Time Or variations such as “read forward”—while copies may lag the master record, every copy goes through the same sequence of changes 55 55

Consistency Model Write Stale version v. 1 v. 2 v. 3 v. 4 Stale version v. 5 v. 6 Generation 1 v. 7 Current version v. 8 Time Achieved via per-record primary copy protocol (To maximize availability, record masterships automaticlly transferred if site fails) Can be selectively weakened to eventual consistency (local writes that are reconciled using version vectors) 56 56

Consistency Model Write if = v. 7 ERROR Stale version v. 1 v. 2 v. 3 v. 4 Stale version v. 5 v. 6 Generation 1 v. 7 Current version v. 8 Time Test-and-set writes facilitate per-record transactions 57 57

Consistency Techniques • Per-record mastering – Each record is assigned a “master region” • May differ between records – Updates to the record forwarded to the master region – Ensures consistent ordering of updates • Tablet-level mastering – Each tablet is assigned a “master region” – Inserts and deletes of records forwarded to the master region – Master region decides tablet splits • These details are hidden from the application – Except for the latency impact! 58

Mastering A B C D E F 42342 42521 66354 12352 75656 15677 E W W E C E A B C D E F Tablet master A B C D E F 42342 42521 66354 12352 75656 15677 E W W E C E 59 59

Bulk Insert/Update/Replace Client Source Data Bulk manager 1. Client feeds records to bulk manager 2. Bulk loader transfers records to SU’s in batches • Bypass routers and message brokers • Efficient import into storage unit 60

Bulk Load in YDOT • YDOT bulk inserts can cause performance hotspots • Solution: preallocate tablets 61

Index Maintenance • How to have lots of interesting indexes and views, without killing performance? • Solution: Asynchrony! – Indexes/views updated asynchronously when base table updated 62

SHERPA IN CONTEXT 63 63

Types of Record Stores • Query expressiveness S 3 PNUTS Simple Object retrieval Retrieval from single table of objects/records Oracle Feature rich SQL 64

Types of Record Stores • Consistency model S 3 PNUTS Best effort Eventual consistency Timeline consistency Object-centric consistency Oracle ACID Strong guarantees Program centric consistency 65

Types of Record Stores • Data model PNUTS Couch. DB Flexibility, Schema evolution Object-centric consistency Oracle Optimized for Fixed schemas Consistency spans objects 66

Types of Record Stores • Elasticity (ability to add resources on demand) Oracle PNUTS S 3 Inelastic Elastic Limited (via data distribution) VLSD (Very Large Scale Distribution /Replication) 67

Data Stores Comparison Versus PNUTS • User-partitioned SQL stores – Microsoft Azure SDS – Amazon Simple. DB • Multi-tenant application databases – Salesforce. com – Oracle on Demand • • • More expressive queries Users must control partitioning Limited elasticity • Highly optimized for complex workloads Limited flexibility to evolving applications Inherit limitations of underlying data management system • • • Mutable object stores – Amazon S 3 • Object storage versus record management 68

Application Design Space Get a few things Sherpa My. SQL Oracle Big. Table Scan everything Everest Records MOb. Stor YMDB Filer Hadoop Files 69 69

SQL/ACID Consistency model Updates Structured access Global low latency Availability Operability Elastic Alternatives Matrix Sherpa Y! UDB My. SQL Oracle HDFS Big. Table Dynamo Cassandra 70 70

Further Reading Efficient Bulk Insertion into a Distributed Ordered Table (SIGMOD 2008) Adam Silberstein, Brian Cooper, Utkarsh Srivastava, Erik Vee, Ramana Yerneni, Raghu Ramakrishnan PNUTS: Yahoo!'s Hosted Data Serving Platform (VLDB 2008) Brian Cooper, Raghu Ramakrishnan, Utkarsh Srivastava, Adam Silberstein, Phil Bohannon, Hans-Arno Jacobsen, Nick Puz, Daniel Weaver, Ramana Yerneni Asynchronous View Maintenance for VLSD Databases, Parag Agrawal, Adam Silberstein, Brian F. Cooper, Utkarsh Srivastava and Raghu Ramakrishnan SIGMOD 2009 (to appear) Cloud Storage Design in a PNUTShell Brian F. Cooper, Raghu Ramakrishnan, and Utkarsh Srivastava Beautiful Data, O’Reilly Media, 2009 (to appear) 71

QUESTIONS? 72 72