IBP and Condor Micah Beck Assoc Prof Director

IBP and Condor Micah Beck Assoc. Prof. & Director Condor Week, Madison May 6, 2003

Logistical Networking Research at UTK University of Tennessee • Micah Beck • James S. Plank • Jack Dongarra University of California, Santa Barbara • Rich Wolski Funding • Dept. of Energy Sci. DAC • National Science Foundation ANIR • UT Center for Info Technology Research

IBP: The Internet Backplane Protocol • A scalable mechanism for deploying shared storage resources throughout the network • A general store-and-forward overlay networking infrastructure • A state management infrastructure for distributed applications and active services

The Network Storage Stack • Our adaption of the network stack architecture for storage • Like the IP Stack • Each level encapsulates details from the lower levels, while still exposing details to higher levels Applications Logistical File System Logistical Tools L-Bone ex. Node IBP Local Access Physical

IBP: How it Works • Storage provisioned on community “depots” • Very primitive service (similar to block service, but more sharable) • Goal is to be a common platform (exposed) • Also part of end-to-end design • Best effort service – no heroic measures • Availability, reliability, security, performance • Allocations are time-limited! • Leases are respected, can be renewed • Permanent storage is to strong to share!

The Network Storage Stack Lo. RS: The Logistical Runtime System: Aggregation tools and methodologies The L-bone: Resource Discovery & Proximity queries The ex. Node: A data structure for aggregation IBP: Allocating and managing network storage (like a network malloc)

L-Bone: January 2003 Current Storage Capacity: 13 TB

Multithreaded Transfers

Caching/Staging

Point-to-Multipoint

Heterogeneous Multicast

Relationship to Work of Condor Group • Wide Area File Management/Access • Management of Computation State • As a Storage Allocation Layer for • Kangaroo, Ne. St • Disk. Router • If routers can have disks, how about disks with processors?

Routers, Depots and the Network Functional Unit in in router send out NFU disk/RAM in out depot load execute RAM in store in out depot in

Scalable Operations • IBP Depots with NFU define a State Transformation Substrate • Processes run at “endpoints” but can use NFUs to transform data in network • Is it processor-in-storage or active networking? • All state is exposed

Logistical Computing and Internetworking http: //loci. cs. utk. edu Micah Beck mbeck@cs. utk. edu