Parallel NFS p NFS Update Feb 5 th

Parallel NFS (p. NFS) Update Feb 5 th, 2007 Rahul Iyer (some slides stolen from Garth Goodson’s Connectathon ’ 06 talk)

Outline Introduction to p. NFS What’s changed since Cthon ’ 06 – Integration with sessions – Issues/Questions Status of the Linux client Server Prototypes Conclusion 2

Introduction Proposition: – Extend NFSv 4 to support data parallelism Goals: – Remove single server bottleneck – Standardize client interface for parallel data access – Meet needs of HPC, and Linux cluster communities Leverage NFSv 4 implementations – Add small set of protocol extension (p. NFS) – Separation of metadata and data Now part of NFSv 4. 1 draft 3

p. NFS Architecture p. NFS metadata protocol Client – standardized NFSv 4. 1 Storage-access protocol – files (v 4. 1), objs, blocks Control protocol – not standardized p. NFS protocol Storage-access protocol Metadata Server (MDS) Control protocol Data Servers (DSs) 4

Example of a p. NFS Write LAYOUTGET WRITE LAYOUTCOMMIT 5

What’s in V 4. 1 p. NFS specification? Description of Client-Metadata Server protocol – This is the core p. NFS protocol Includes: – – Semantics of layouts Crash recovery Security considerations Definition of core data structures and operations Description of the Client-Data Server file-layout – This is the NFSv 4 file-layout protocol 6

What’s not in the v 4. 1 p. NFS spec. Object and Block-layout protocol definitions These are addressed in separate documents – Currently as IETF WG drafts 7

Outline Introduction What’s changed since Cthon ’ 06 – Implications of sessions integration – Issues/Questions Status of the Linux client Server Prototypes Conclusion 8

What’s Changed since Cthon ’ 06 p. NFS now requires sessions – Session between client and MDS – Session between client and each DS – p. NFS callback races resolved Layout now part of “locking state” • Tied to the lease period Introduction of new device types – Simple vs Complex device types Formalized Crash Recovery Mechanisms – e. g. LAYOUTCOMMIT reclaim during Grace Period 9

Changes since Cthon '06 (Contd. ) New Flags and Attributes – Return on Close – MDS Threshold Clarified Open Issues Spec currently undergoing editing passes – To make it “flow” better 10

Implications of Sessions Setting up State on the MDS – Client does EXCHANGE_ID and CREATE_SESSION Setting up state on the DS – Client calls CREATE_SESSION on each DS – EXCHANGE_ID MUST NOT be called – Loss of MDS session may require DS session setup Issues for Discussion – CREATE_SESSION to the Data Servers • What Sequence IDs does the client use? • Differentiating DS vs MDS/regular NFSv 4. 1 server • Which exported filesystem is the session for? 11

Implications of Sessions (Contd. ) Layouts part of “locking infrastructure” – Layouts tied to clientid – Clients must not use layout beyond lease expiration • Callback path is tied to a session! • Layouts cannot be recalled without a session • Client may need to return layouts before destroying session • Server may need to recall layouts before dropping session state – Only operations to MDS renew leases • Cannot assume that I/O to DS renews leases 12

Outline Introduction What’s changed since Cthon ’ 06 – Implications of sessions integration – Issues/Questions Status of the Linux client Server Prototypes Conclusion 13

Linux Client Architecture Divided into two parts – Generic piece • For core p. NFS protocol • Tied into the NFSv 4 client • Called Generic client – Back end specific piece • Different for each back end (files/objects/blocks) • Called Layout driver 14

Status of the Linux Client Where we are – Basic p. NFS ops work – Basic sessions stuff works • CREATE_SESSION, DESTROY_SESSION, SEQUENCE • No slotid management – What’s left • Slotid management • Callbacks • Layout management • Crash Recovery 15

Outline Introduction What’s changed since Cthon ’ 06 – Implications of sessions integration – Issues/Questions Status of the Linux client Server Prototypes Conclusion 16

p. NFS Server Prototypes NFSv 4 Files – Netapp – IBM (GPFS) – Sun Objects – Panasas Blocks – EMC IBM, Panasas, EMC use p. NFS enabled Linux server – Linux server provides hooks to export various cluster Filesystems 17

Outline Introduction What’s changed since Cthon ’ 06 – Implications of sessions integration – Issues/Questions Status of the Linux client Server Prototypes Conclusion 18

Conclusions p. NFS is a first class citizen of NFSv 4. 1 Integration with sessions almost complete – Few issues remain Prototypes proceeding well Linux p. NFS Mailing List – pnfs AT linux-nfs DOT org The latest draft is at http: //nfsv 4 -editor. org/draft-08/draft-ietf-nfsv 4 -minorversion 1 -08. html My contact: iyer AT netapp DOT com 19

Notable NFSv 4. 1 changes Changes to Locking Infrastructure – Clientid in OPEN/CLOSE/LOCK etc. is zero • Clientid obtained from current session id – Sequence ids in OPEN/CLOSE/LOCK etc is 0 • Only once semantics provided by sessions – OPEN_CONFIRM is illegal • Sequence id confirmation no longer required – RENEW is illegal • SEQUENCE op now renews leases Client establishes the callback channel – Can be same as the fore-channel – Different connection can be bound to session too 20