Filesystems FUSE Michael Mitchell Adding features to filesystems

Filesystems & FUSE Michael Mitchell

Adding features to filesystems Could we. . . Automatically encrypt/decrypt? Automatically compress/decompress? Present tars and zips as directory trees? Show an SQL table as a directory?

Where do we put this code? Modify each application Modify existing libraries or new library New filesystem layer Existing filesystems

Issues with core filesystems C only ◦ Limited libraries ◦ Kernel mode, more complicated No access to network or other files Kernel: poor place for complex features

FUSE: Filesystems in USErspace Now a program (fuse application) is responsible for dirs and files When file is needed, kernel asks fuse application for it Fuse application can access anything to get raw data ◦ ◦ Existing local filesystems Remote trees and network connections /dev/random Etc.

How FUSE Works Application makes a file-related syscall Kernel figures out that the file is in a mounted FUSE filesystem The FUSE kernel module forwards the request to your userspace FUSE app Your app tells FUSE how to reply

Other Key Features Cross-platform: Linux/BSD/OS X Wide language support: natively in C, with bindings in C++, Java, C#, Haskell, TCL, Python, Perl, Shell Script, SWIG, OCaml, Pliant, Ruby, Lua, Erlang, PHP

FUSE: The Pros Makes it easy to write new filesystems ◦ Without knowing how the kernel works ◦ Without breaking unrelated things ◦ More quickly/easily than traditional file systems built as a kernel module Makes it safe for sysadmins to let users they don’t trust use custom file systems

FUSE: The Cons Performance ◦ Context switches ◦ Apps slower than kernels Swappable ◦ Fuse content not generally cacheable Permissions ◦ User and “anyone” permissions fine ◦ Group permissions tough

FUSE Architecture Implemented as a Linux Kernel Module. Re-routes calls to the VFS layer from user programs to a special file /dev/fuse A userland program can then use libfuse to read and write /dev/fuse Calls exposed by libfuse are meant to mimic (mostly) those available to userland programs in UNIX

FUSE Architecture

What do people do with FUSE? Hardware-based: ext 2, iso, ZFS… Network-based: NFS, smb, SSH… Nontradtional: Gmail, My. SQL… Loopback: compression, conversion, encryption, virus scanning, versioning… Synthetic: search results, application interaction, dynamic conf files…

Writing a FUSE Filesystem Write an ordinary application that defines certain functions/methods that FUSE will call to handle operations ◦ ~35 possible operations Many operations have useful defaults ◦ Useful filesystems can define only ~4 ◦ Full-featured ones will need to define most

My FUSE Image/Audio/Video files treated as both a file and a directory Can 'cd' into media file, and 'ls' the contents So far, just one dirent called 'info' for each Using 'cat' on the 'info' file will dump autogenerated info based on file type Images: ◦ imagemagick identify 'file‘ Audio/Video: ◦ /usr/bin/mplayer -frames 0 -identify 'file'

How its currently working if (is. Audio(full. Path) || is. Video(full. Path) || is. Image(full. Path)) { ret = RETURN_ERRNO(lstat(full. Path, statbuf)); // modify statbuf to report is directory statbuf->st_mode |= S_IFDIR; } statbuf->st_mode &= ~(S_IFREG);

Problems Files don't want to be both a directory and a file! It turns it into a socket! stat(); ◦ ◦ S_IFREG 0100000 regular file S_IFDIR 0040000 directory statbuf->st_mode |= S_IFDIR; statbuf->st_mode &= ~(S_IFREG); Problem: ◦ S_IFSOCK 0140000 socket

My FUSE Like a ubiquitous /proc Enables features like: Micro mount points for filesystems Hidden filesystems Covert channel communication & steganography Permission controls; certain apps can access certain private directories ◦ Simplify the 'plugin' interface ◦ ◦

Questions?