Lazy Asynchronous IO For EventDriven Servers Khaled Elmeleegy

Lazy Asynchronous I/O For Event-Driven Servers Khaled Elmeleegy, Anupam Chanda and Alan L. Cox Department of Computer Science Rice University, Houston, Texas. Willy Zwaenepoel School of Computer and Communication Sciences EPFL, Lausanne, Switzerland.

Event-Driven Architecture n Event-driven architecture is widely used for servers Performance n Scalability n 2

Problem n Developing event-driven servers is hard for many reasons n We focus on problems with non-blocking I/O: Incomplete coverage n Burden of state maintenance n 3

Lazy Asynchronous I/O (LAIO) n Addresses problems with non-blocking I/O n Universality n n Simplicity n n Covers all I/O operations Requires less code High performance for event-driven servers n Meets or exceeds alternatives 4

Outline n n Background Lazy Asynchronous I/O (LAIO) Evaluation Conclusions 5

Outline n Background n n Event-driven servers Lazy Asynchronous I/O (LAIO) Evaluation Conclusions 6

Event-Driven Servers n n n Event loop processes incoming events For each incoming event, it dispatches its handler Single thread of execution Handler #1 Event Loop Handler #2 Handler #k 7

Event Handler I/O operation (Network/Disk) To event loop Complete handling event 8

Event Handler n If the I/O operation blocks n The server stalls I/O operation (Network/Disk) Block To event loop Complete handling event 9

Outline n n Background Lazy Asynchronous I/O (LAIO) API n Example of usage n Implementation n Evaluation Conclusions 10

LAIO API Return Type Function Name Parameters int laio_syscall int number, … void* laio_gethandle void int laio_poll laio_completion[] completions, int ncompletions, timespec* ts 11

laio_syscall() n Lazily converts any system call into an asynchronous call n If the system call doesn’t block laio_syscall() returns immediately n With return value of system call n n Else if it blocks: laio_syscall() returns immediately n With return value -1 n errno set to EINPROGRESS n Background LAIO operation n 12

laio_gethandle() n Returns a handle representing the last issued LAIO operation n If operation didn’t block, NULL is returned 13

laio_poll() n n Returns a count of completed background LAIO operations Fills an array with completion entries n n One for each operation Each completion entry has Handle n Return value n Error value n 14

Event Handler With LAIO n If operation completes n Handler continues execution laio_syscall() No To event loop Operation completed Yes Complete handling event 15

Event Handler With LAIO n If operation doesn’t complete n n laio_syscall() returns immediately Handler records LAIO handle Returns to event loop Completion notification arrives later To event loop laio_syscall() No le () dle Han and h t e io_g Operation completed = la Yes Complete handling event 16

LAIO Implementation n n LAIO uses scheduler activations Scheduler activations n The kernel delivers an upcall when an operation Blocks - laio_syscall() n Unblocks - laio_poll() n 17

laio_syscall() – Non-blocking case Application laio_syscall() • Save context • Enable upcalls Issue operation Library System call blocks? No • Disable upcalls • Return retval 18

laio_syscall() – Blocking case Application laio_syscall() • Save context • Enable upcalls Library Issue operation System call blocks? Yes 19

laio_syscall() – Blocking case Application laio_syscall() • Save context • Enable upcalls Library Issue operation System call blocks? Yes Kernel Background laio_syscall Upcall on a new thread 20

laio_syscall() – Blocking case Application laio_syscall() • Save context • Enable upcalls Library Issue operation System call blocks? Yes Kernel Background laio operation upcall handler Library Steals old stack using stored context Upcall on a new thread 21

laio_syscall() – Blocking case Application laio_syscall() • Save context • Enable upcalls Library Issue operation System call blocks? Yes Kernel Background laio operation • Disable upcalls • errno = EINPROGRESS • Return -1 upcall handler Library Steals old stack using stored context Upcall on a new thread 22

Unblocking case upcall handler() Library • Construct completion structure: • laio operation handle. • System call return value. • Error code. • Add completion to list of completions. n List of completions is retrieved by the application using laio_poll() • Background laio operation completes, thread dies • Upcall on the current thread Kernel 23

Outline n n n Background Lazy Asynchronous I/O (LAIO) Evaluation n n Methodology Experiments n n LAIO vs. conventional non-blocking I/O LAIO vs. AMPED Programming complexity Conclusions 24

Methodology n n n Flash web server Intel Xeon 2. 4 GHz with 2 GB memory Gigabit Ethernet between machines Free. BSD 5. 2 -CURRENT Two web workloads Rice 1. 1 GB footprint n Berkeley 6. 4 GB footprint n 25

Experiments: LAIO vs. conventional non-blocking I/O Server-Network-Disk Threaded Blocking operations Flash-NB-B Single Disk I/O Flash-NB-AIO Single Flash-LAIO Single Disk I/O other than read and write None Compare performance 26

Performance: Large Workload 27

Performance: Small Workload 28

Why Be Lazy? n Most potentially blocking operations don’t actually block n n Experiments: 73% - 86% of such operations don’t block Lower overhead n Micro-benchmark: AIO is 3. 2 times slower than LAIO for non-blocking operations 29

Experiments: LAIO vs. AMPED Server-Network-Disk Threaded Blocking operations Flash-LAIO Single None Flash-NB-AMPED Process-based helpers None Compare performance 30

Asymmetric Multi-process Event. Driven (AMPED) Architecture n n n Event-driven core Potentially blocking I/O handed off to a helper Helper can block as it runs on a separate thread of execution Helper #1 Handler #1 Helper #2 Handler #2 Event Loop Helper #n Handler #k 31

Flash-NB-AMPED n n Stock version of flash Two relevant helpers n n File read: reads a file from disk Name conversion: checks for a file’s existence and permissions 32

Performance of LAIO vs. AMPED 33

Performance of LAIO vs. AMPED 34

Programming Complexity n Where performance is comparable: Flash. LAIO-LAIO vs. Flash-NB-AMPED n Flash-LAIO is simpler No helpers n No state maintenance n 35

State Maintenance With Nonblocking I/O n If operation does not complete n Handler returns to event loop handler I/O operation No To event loop Operation completed Yes Complete handling event 36

State Maintenance With Nonblocking I/O n If operation does not complete n n Handler returns to event loop Operation is continued later handler I/O operation No Operation completed To event loop Yes Complete handling event 37

State Maintenance With Nonblocking I/O n If operation does not complete: n n n handler Handler returns to event loop Operation is continued later This may require To storing state event loop I/O operation Store state No Operation completed Yes Complete handling event 38

Lines of Code Comparison Component Flash-NB-AMPED Flash-LAIO lines of code File read 550 15 Name Conversion 610 375 Partial-write state maintenance 70 0 Total code size 8860 8020 9. 5% reduction in lines of code 39

Conclusions n LAIO is universal n n Supports all system calls LAIO is simpler Used uniformly n No state maintenance n No helpers n Less lines of code n n LAIO meets or exceeds the performance of other alternatives 40

LAIO source can be found at: http: //www. cs. rice. edu/~kdiaa/laio 41