An EndtoEnd Approach to Globally Scalable Network Storage

An End-to-End Approach to Globally Scalable Network Storage Presented in cs 294 -4 P 2 P Systems by Sailesh Krishnamurthy 15 October 2003

Logistical Networking n n Models sync/async aspects of communication Single “fabric” that unifies: n n Data Storage Data Transportation n Internet scalability goals n Claim: end-to-end design principles vital

Background: SAN vs NAS n Current trends in storage networking n n SAN - Storage Area Network NAS - Network Attached Storage

More on NAS v. SAN n NAS n n n Wires: TCP/IP Protocol: NFS, CIFS SAN n n Wires: Fiber Channel Protocol: Encapsulated SCSI Where is the File System ?

Traditional networks n General goals n n Minimize delay Minimize probability of corruption Maximize probability of delivery Assumptions in traditional storage nets n When storage is closely coupled, delay and probability of corruption can be low while availability is high.

SANs cannot scale to SWANs n n In the SWAN, resources can be intermittently unavailable So we need e 2 e strategies n n n Simple retries Redundant data accesses spread across nw High-latency archival backups

Correctness in the wild n n n SANs are “controlled” environments and correctness is not an issue In the SWAN, data storage may not be reliable. Data accuracy must be checked by producers and consumers - at endpoints

SWAN Security n n SWANs are not physically localized SAN security assumptions don’t hold n n Again, e 2 e approaches are required Do. S is a gotcha for SWANs n n Can’t be prevented with e 2 e strategies Imitate techniques for handling Do. S in IP

Unbounded Size/Duration n n Since single store may not have all the resources all the time, the endpoint has to manage distribution of data Unbounded duration allocation hurts resource sharing. n Should this be managed at endpoints ?

Logistical Networking n Storage Networking n n IP networks: interconnection fabric of storage pool Logistical Networking n n Storage part of the networking infrastructure Shared resource fabric exposing storage resources n n Similar to how internet exposes bandwidth resources Storage Stack n n Bottom-up, layered e 2 e design approach Internet Backplane Protocol (IBP)

Storage Stack Applications ex. Node tools and services ex. Node A data structure for aggregating network storage IBP Allocation and management of storage on network storage depots Local Access Physical

IBP - Internet Backplane Protocol n n First layer of stack that’s globally accessible Abstracts access layer resources (file/block storage services) n Expose underlying storage resources to maximize freedom at higher levels n n Enable scalable internet style resource sharing n n Implement only indispensable & common functions Mask peculiarities of access layer resource Abstract service based on data blocks that are managed as “byte arrays”

IP vs IBP n IP vs link layer n n Agg. of link layer packets masks packet size limits Simple fault detection faulty datagrams dropped Global addressing masks diffs b/w LANs IP Property n n Any participant of a routed IP n/w can use any link layer connection IBP “byte array” indep. n n Agg. access layer blocks masks fixed block size Simple fault detection drop faulty byte arrays Global addressing (IP) maks diff b/w acc layer IBP Property n Any participant of an IBP n/w can use any access layer storage resource

Issues with IBP n Do. S vulnerability is much worse n In IP: n n n In IBP: n n n Do. S attacks require constant sending of data Does not profit the attacker in any way Once data block is allocated it remains used Using remote storage does benefit the attacker Strong semantics (reliability) of traditional storage/SAN are difficult to implement in the SWAN

IBP Solutions n Time-limited storage allocations n n When a lease expires, the storage can be reused for some other user Soft storage semantics in IBP n n IBP is a “best-effort” service Allocated storage can be revoked at any time Storage Management Data Transfer Depot Management IBP_Allocate, IBP_Manage IBP_Store, IBP_Load, IBP_Copy, IBP_mcopy IBP_Status

ex. Node - Flexible Aggregation of Network Storage n Implement abstractions w/ strong properties n n n Higher layer construct Aggregates primitive IBP byte-arrays Need to maintain state that represents the agg. n ex. Node aggregates IBP byte-arrays as the Unix inode aggs. disk blocks

e 2 e services for storage n ex. Node can hold additional metadata for services: n n n Redundancy Framing of data into segments w/ checksums ex. Node is analogous to the state of a TCP connection, data on disk analogue of a TCP stream

Relation to p 2 p systems n Paper compares with Napster/Gnutella n n n In file sharing allocations are at endpoints. . leads to large data transfers Appropriate comparison is Oceanstore ? My view n n n ex. Node infrastructure is a way to create storage services from smaller blocks Can be useful in an Oceanstore-like setting Can alleviate some SAN shortcomings ?