Proprietary Confidential Proprietary Confidential Proprietary Confidential Proprietary Confidential

秘密 Proprietary Confidential▲ 云存储技术和应用

存储发展历史-块访问 秘密 Proprietary Confidential▲

存储发展历史-块访问 秘密 Proprietary Confidential▲

存储发展历史-文件访问 秘密 Proprietary Confidential▲

存储发展历史-对象存储 秘密 Proprietary Confidential▲

存储发展历史-对象存储设备OSD 秘密 Proprietary Confidential▲ CPU Applications System Call Interface File System User Component File System Storage Component Object Interface Block Interface File System Storage Component Block I/O Manager Storage Device SCSI Block I/O Manager Storage Device

存储发展历史-对象存储设备OSD 秘密 Proprietary Confidential▲ Acce ss Re ques Client t DAT SECRET KEY Eth. SAN switch GE A N A SECRET KEY T N ME MA SECRET KEY Object-based Storage Devices Managers Validate Capability

存储发展历史-对象存储vs块存储 秘密 Proprietary Confidential▲

存储发展历史总结 秘密 Proprietary Confidential▲

云存储需求 秘密 Proprietary Confidential▲

秘密 Proprietary Confidential▲ 云存储有哪些？ Google Cloud Amazon Cloud Microsoft Azure IBM Blue Cloud Google File System S 3 Live Services Hadoop Big. Table Simple. DB SQL Services DB 2 Google App Engine EC 2 . NET Services Xen & Power. VM Map. Reduce Simple Queue Sharepoint Services Websphere EBS Dynamic CRM Services Tivoli Software

云存储的热点 秘密 Proprietary Confidential▲ 分布式文件系统（Google GFS、Hadoop HDFS。。） n No. SQL数据库（Google Big. Table、Facebook Cassandra。。） n 对象存储（Amazon S 3） n 分布式块存储（Amazon EBS） n

分布式文件系统应用场景 n Google： l n Baidu： l n 搜索、App. Engine、Gmail、Youtube、Map。。 Hadoop HDFS搜索、数据分析 Taobao： l l TFS存储图片网页 Hadoop HDFS数据分析 秘密 Proprietary Confidential▲

GFS-接口(API) l 秘密 Proprietary Confidential▲ 提供文件系统接口 Ø Ø Ø 非标准的POSIX API 仅仅提供几种最基本的操作–create, delete, open, close, read, write 利用写时拷贝（ copy-on-write）技术实现快照功能

GFS-体系结构 l 文件被切割成固定大小的块（Chunk） l 一个主Master，多个Shadow Master l 多个Chunkserver l 多client 秘密 Proprietary Confidential▲

Amazon S 3 概念 n 数据模型 n 接口 n 秘密 Proprietary Confidential▲

秘密 Proprietary Confidential▲ S 3存储的发展 S 3存��象数量（�） 1020 1250 1000 750 500 250 0 Q 1 2006 Q 1 2007 Q 1 2008 Q 1 2009 Q 1 2010 数据来源：http: //www. snia. org

S 3的价格 秘密 Proprietary Confidential▲

S 3接口：RESTful API n n 秘密 Proprietary Confidential▲ HTTP协议不仅仅可以用来进行网络传输，而且可以用以下几 种API来实现所有的Web操作 l PUT = "here's some new info" (Create) l GET = "give me some info" (Retrieve) l POST = "here's some update info" (Update) l DELETE = "delete some info" (Delete) HTTP API符合CRUD (Create, Retrieve, Update, and Delete) 原则

REST GET示例 秘密 Proprietary Confidential▲ GET /Nelson HTTP/1. 1 Host: quotes. s 3. amazonaws. com Request Date: Wed, 01 Jun 2010 12: 00 GMT Authorization: AWS 15 B 4 D 3461 F 177624206 A: x. QE 0 di. Mb. LRepdf 3 YB+FIEXAMPLE= HTTP/1. 1 200 OK x-amz-id-2: j 5 ULAWp. Fb. JQJpuk. Us. Z 4 tf. XVOj. VZEx. Lt. Ey. NTv. Y 5 fe+h. HIegs. N 5 p 578 JLTVpk. Frp. Lx-amzrequest-id: BE 39 A 20848 A 0 D 52 B Date: Wed, 01 Jun 2010 12: 00 GMT x-amz-meta-family: Muntz Last-Modified: Sun, 1 Jan 2010 12: 00 GMTE Tag: "828 ef 3 fdfa 96 f 00 ad 9 f 27 c 383 fc 9 ac 7 f“ Content-Type: text/plain Content-Length: 5 Connection: close Server: Amazon. S 3 ha-ha Response

NOSQL数据库 n n n 概念 Key-value Column Document Graphy 秘密 Proprietary Confidential▲

What is No. SQL? No. SQL=Non relational或者Not only SQL n No. SQL是相对于RDBMS的，具有以下特征 n l l l l l n 非关系 分布式 开源 水平扩展 schema-free easy replication support simple API 最终一致性 支持海量数据（Huge Data） 不支持ACID No. SQL于2009年发展迅速（http: //nosql-database. org） 秘密 Proprietary Confidential▲

秘密 Proprietary Confidential▲ No. SQL四大家族 Key-Value Column Document Graph Dynamo Big. Table Mongo. DB Neo 4 j Riak HBase Couch. DB Hyper. Graph. D B Memcached Cassandra …… …… Redis Hyper. Table Tokyo Cabinet/Ty rant …… Voldemort ……

Dynamo n购物车问题 n不开源，公布了paper n技术特征： l一致性哈希 l虚节点 l. Vector Clock l. Hinted Handoff l. Anti-entropy l. Merkle Tree l. Gossip 秘密 Proprietary Confidential▲

秘密 Proprietary Confidential▲

秘密 Proprietary Confidential▲

秘密 Proprietary Confidential▲

秘密 Proprietary Confidential▲

秘密 Proprietary Confidential▲

秘密 Proprietary Confidential▲

秘密 Proprietary Confidential▲

秘密 Proprietary Confidential▲

秘密 Proprietary Confidential▲

秘密 Proprietary Confidential▲