Meta Sync File Synchronization Across Multiple Untrusted Storage

Meta. Sync File Synchronization Across Multiple Untrusted Storage Services Seungyeop Han (syhan@cs. washington. edu) Haichen Shen, Taesoo Kim*, Arvind Krishnamurthy, Thomas Anderson, and David Wetherall University of Washington *Georgia Institute of Technology USENIX ATC '15 1

File sync services are popular 400 M of Dropbox users reached in June 2015 USENIX ATC '15 2

Many sync service providers Dropbox (2 GB) Google Drive (15 GB) MS One. Drive (15 GB) Box. net (10 GB) USENIX ATC '15 Baidu(2 TB) 3

Can we rely on any single service? USENIX ATC '15 4

Existing Approaches • Encrypt files to prevent modification – Boxcryptor • Rewrite file sync service to reduce trust – SUNDR (Li et al. , 04), DEPOT (Mahajan et al. , 10) USENIX ATC '15 5

Meta. Sync Higher availability, greater capacity, higher performance Stronger confidentiality & integrity Meta. Sync: Can we build a better file synchronization system across multiple existing services? USENIX ATC '15 6

Goals • Higher availability • Stronger confidentiality & integrity • Greater capacity and higher performance • No service-service, client-client communication • No additional server • Open source software USENIX ATC '15 7

Overview • • • Motivation & Goals Meta. Sync Design Implementation Evaluation Conclusion USENIX ATC '15 8

Key Challenges • Maintain a globally consistent view of the synchronized files across multiple clients • Using only the service providers’ unmodified APIs without any centralized server • Even in the presence of service failure USENIX ATC '15 9

Design Choices • How to manage files? – Content-based addressing & hash tree • How to update consistently with unmodified APIs? – Client-based Paxos (p. Paxos) • How to spread files? – Stable deterministic mapping • How to protect files? – Encryption from clients • How to make it extensible? – Common abstractions USENIX ATC '15 10

Design Choices • How to manage files? – Content-based addressing & hash tree • How to update consistently with unmodified APIs? – Client-based Paxos (p. Paxos) • How to spread files? – Stable deterministic mapping • How to protect files? – Encryption from clients • How to make it extensible? – Common abstractions USENIX ATC '15 11

Overview of the Design Meta. Sync 1. File Management Object Store Synchronization Replication Backend abstractions Dropbox Google Drive USENIX ATC '15 Local Storage One. Drive Remote Services 12

Object Store • Similar data structure with version control systems (e. g. , git) • Content-based addressing – File name = hash of the contents – De-duplication – Simple integrity checks • Directories form a hash tree – Independent & concurrent updates USENIX ATC '15 13

Object Store head = f 12… Dir 1 abc… Dir 2 Large. bin 4 c 0… 20 e… small 1 small 2 blob • Files are chunked or grouped into blobs • The root hash = f 12… uniquely identifies a snapshot USENIX ATC '15 14

Object Store old = f 12… Dir 1 abc… Dir 2 head = 07 c… Large. bin 4 c 0… 20 e… Large. bin 1 ae… small 1 small 2 blob • Files are chunked or grouped into blobs • The root hash = f 12… uniquely identifies a snapshot USENIX ATC '15 15

Overview of the Design Meta. Sync 2. Consistent update Object Store Synchronization Replication Backend abstractions Dropbox Google Drive USENIX ATC '15 Local Storage One. Drive Remote Services 16

Updating Global View Client 1 Global View Client 2 Head Current root hash Previously synchronized point v 0 ab 1… Prev Head master USENIX ATC '15 17

Updating Global View v 1 c 10… Client 1 Global View Client 2 Prev Head v 0 ab 1… Prev Head master USENIX ATC '15 18

Updating Global View Head Client 1 Global View Client 2 Prev v 0 ab 1… v 1 c 10… Prev Head master USENIX ATC '15 19

Updating Global View Head Client 1 Global View Client 2 Prev v 0 ab 1… v 1 c 10… Prev Head master USENIX ATC '15 20

Updating Global View v 2 f 13… Client 1 Global View Client 2 Prev v 0 ab 1… Head v 1 c 10… Prev Head v 2 7 b 3… master USENIX ATC '15 21

Updating Global View v 2 f 13… Client 1 Global View Prev v 0 ab 1… v 1 c 10… Client 2 Head v 2 7 b 3… Prev Head master USENIX ATC '15 22

Updating Global View v 3 a 31… Client 1 Global View Prev v 0 ab 1… v 1 c 10… Client 2 Head v 2 7 b 3… Prev Head master USENIX ATC '15 23

Consistent Update of Global View Dropbox Google Drive Meta. Sync One. Drive Meta. Sync root= b 05… root= f 12… • Need to handle concurrent updates, unavailable services based on existing APIs USENIX ATC '15 24

Paxos • Multi-round non-blocking consensus algorithm – Safe regardless of failures – Progress if majority is alive Proposer Acceptor USENIX ATC '15 25

Metasync: Simulate Paxos • Use an append-only list to log Paxos messages – Client sends normal Paxos messages – Upon arrival of message, service appends it into a list – Client can fetch a list of the ordered messages • Each service provider has APIs to build appendonly list – Google Drive, One. Drive, Box: Comments on a file – Dropbox: Revision list of a file – Baidu: Files in a directory USENIX ATC '15 26

Metasync: Passive Paxos (p. Paxos) • Backend services work as passive acceptor • Acceptor decisions are delegated to clients New root = 1 S 2 fetch(S 1) New root = 2 Accepted root = 1 S 3 fetch(S 2) Clients Passive Storage Services USENIX ATC '15 27

Paxos vs. Disk Paxos vs. p. Paxos • Disk Paxos: maintains a block per client Gafni & Lamport ’ 02 Acceptor … computation Propose require # msgs Accept disk blocks Propose Check append-only Propose Check Proposer Paxos Disk Paxos p. Paxos O(clients x acceptors) O(acceptors) Requires acceptor API O(acceptors) USENIX ATC '15 28

Overview of the Design Meta. Sync Object Store 3. Replicate objects Synchronization Replication Backend abstractions Dropbox Google Drive USENIX ATC '15 Local Storage One. Drive Remote Services 29

Stable Deterministic Mapping • Meta. Sync replicates objects R times across S storage providers (R<S) • Requirements – Share minimal information among services/clients – Support variation in storage size – Minimize realignment upon configuration changes • Deterministic mapping – E. g. , map(7 a 1…) = Dropbox, Google USENIX ATC '15 30

Implementation • Prototyped with Python – ~8 k lines of code • Currently supports 5 backend services – Dropbox, Google Drive, One. Drive, Box. net, Baidu • Two front-end clients – Command line client – Sync daemon USENIX ATC '15 31

Evaluation • How is the end-to-end performance? • What’s the performance characteristics of p. Paxos? • How quickly does Meta. Sync reconfigure mappings? USENIX ATC '15 32

Evaluation • How is the end-to-end performance? • What’s the performance characteristics of p. Paxos? • How quickly does Meta. Sync reconfigure mappings? USENIX ATC '15 33

End-to-End Performance Synchronize the target between two computers Dropbox Google Meta. Sync Linux Kernel 920 directories 15 k files, 166 MB 2 h 45 m > 3 hrs 12 m 18 s Pictures 50 files, 193 MB 415 s 143 s 112 s (S = 4, R = 2) Performance gains are from: • Parallel upload/download with multiple providers • Combined small files into a blob USENIX ATC '15 34

Latency of p. Paxos 35 30 Latency (s) 25 20 Google 15 One. Drive 10 Box Dropbox Baidu 5 0 1 2 3 # of Proposers 4 5 Latency is not degraded with increasing concurrent proposers or adding slow backend storage service USENIX ATC '15 35

Latency of p. Paxos 35 30 Latency (s) 25 Google 20 Dropbox 15 One. Drive 10 Baidu Box All 5 0 1 2 3 # of Proposers 4 5 Latency is not degraded with increasing concurrent proposers or adding slow backend storage service USENIX ATC '15 36

Conclusion • Meta. Sync provides a secure, reliable, and performant files sync service on top of popular cloud providers – To achieve a consistent update, we devise a new client-based Paxos – To minimize redistribution, we present a stable deterministic mapping • Source code is available: – http: //uwnetworkslab. github. io/metasync/ USENIX ATC '15 37