CDMI An Introduction January 2013 Big Data Big

CDMI: An Introduction January 2013

Big Data “Big Data” refers to datasets whose size is beyond the ability of typical tools to capture, store, manage and analyze Complexity Big Data Speed Volume 2

Trends Shaping the Big Data Market ¡ Scale – 10’s PB of data, millions of users – “Life time retention” of data ¡ Object Access – Object = Data + Metadata – APIs/Protocols to provide object access ¡ Automatic Data Management – Object granular automated data mobility and management ¡ Distributed Access – Multi-location repositories, data mobility, local view/access 3

Cloud Storage ¡ Commonly used term to describe a data store deployment ¡ Apt to describe a Big Data store, increasingly characterized by: – Object based access and APIs – Multi-protocol support (NAS: NFS/CIFS/Object, SAN: FC/i. SCSI) – Automatic policy-driven management based on object metadata – Provides services like billing, audit, security – Globally federated clouds ¡ Formal term: Data Storage as a Service (Daa. S)

What is CDMI? ¡ Cloud Data Management Interface ¡ CDMI is a open standard for cloud storage – Defines client-to-cloud and cloud-to-cloud interactions – Defines object semantics for content stored in the cloud Net. App Confidential - Internal Use Only 5

What does CDMI let you do? ¡ CDMI allows clients to store and retrieve from the cloud – Named Objects (Accessed by name, similar to files) – Nameless Objects (Accessed by ID) – Queues (first-in-first-out structured storage) ¡ CDMI allows clients to organize stored content – Containers (Groups of named objects, similar to directories) ¡ CDMI allows clients to manage stored content – User Metadata (Structured metadata attached to objects) – Data System Metadata (Allows clients to request storage characteristics) – Versions of objects ¡ CDMI allows clients to discover content – Query and Notifications Net. App Confidential - Internal Use Only 6

Why a standard? ¡ End customers want it: – Customers want cross-cloud interoperability – Customers want to be able to avoid vendor lock-in, switch cloud vendors and second-source cloud solutions ¡ Application developers want it: – Developers don’t want to develop custom code to talk with each vendor’s cloud – Developers want to leverage common libraries, documentation, test tools, expertise, etc. ¡ Cloud storage vendors want it: – Tired of inventing yet another new API that needs to be documented, tested and maintained – Don’t want their API to be controlled and lead by vendors ¡ Cloud service providers don’t want it (yet): – They like API lock-in, and feel they can adopt CDMI later. Net. App Confidential - Internal Use Only 7

Who created CDMI? ¡ A Technical Working Group in SNIA comprising several major vendors in the Storage and Networking industry ¡ Each of the major contributors have provided at least 20 pages of spec text And many more… Net. App Confidential - Internal Use Only 8

What does CDMI look like? ¡ CDMI is a wire protocol, like CIFS and NFS, that provides access to objects ¡ CDMI is based on RESTful HTTP, and is designed to be a very simple protocol ¡ CDMI uses JSON, the Web 2. 0 data format, and can be directly accessed by any web browser ¡ Basic CDMI functionality includes Create, Read, Update and Delete of objects ¡ Advanced CDMI functionality includes notifications, query, snapshots, serialization, and compliance Net. App Confidential - Internal Use Only 9

What does CDMI look like? ¡ CDMI GET /mycontainer/photos/eqd. jpg HTTP/1. 1 Host: cdmi. example. com Date: Mon, 26 Mar 2007 19: 37: 58 +0000 Authorization: Basic cm 9 vd. Dpyb 291 <Image Data> Net. App Confidential - Internal Use Only 10

Opportunities for Partners! ¡ Develop CDMI clients to make legacy applications CDMI aware ¡ Develop CDMI protocol conformance and performance testing tools ¡ Develop tools to migrate data from non-CDMI compliant clouds to CDMI compliant clouds ¡ Develop utilities/services for cloud provides using CDMI data system metadata ¡ Design applications using object metadata Net. App Confidential - Internal Use Only 11

How to learn more? ¡ CDMI specification: – http: //snia. org/sites/default/files/CDMI%20 v 1. 0. 2. pdf ¡ Participate in CDMI Plug. Fests: – http: //www. snia. org/cloudplugfest ¡ Use the opportunities to develop Net. App Confidential - Internal Use Only 12