Proactive Serverless Function Resource Management Sixth International Workshop

Proactive Serverless Function Resource Management Sixth International Workshop on Serverless Computing (Wo. SC 6) 2020 Erika Hunhoff, Shazal Irshad, Vijay Thurimella*, Ali Tariq, Eric Rozner University of Colorado Boulder, *Thrive, Inc 1

Background Freshen Design Evaluation Outline Discussion Questions 2

Background Freshen Design Evaluation Outline Discussion Questions 3

Open. Whisk Container function 1 IDCat Serverless Application 4

User. ID: Erika Open. Whisk Function Trigger function 1 IDCat Serverless Application 5

User. ID: Erika Open. Whisk Function Trigger function 1 del!” o m L M e the “Give m is” “Here it Cat Model Database IDCat Serverless Application 6

User. ID: Erika Open. Whisk Function Trigger function 1 does work! del!” o m L M e the “Give m is” “Here it Cat Model Database IDCat Serverless Application 7

User. ID: Erika Open. Whisk Function Trigger function 1 del!” o m L M e the “Give m is” “Here it Store “Erik a’s Ca t, Bro wn T “Sav abby ed!" ” Cat Model Database Cat Database IDCat Serverless Application 8

User. ID: Erika Open. Whisk Function Trigger function 1 Result: “Success!” del!” o m L M e the “Give m is” “Here it Store “Erik a’s Ca t, Bro wn T “Sav abby ed!" ” Cat Model Database Cat Database IDCat Serverless Application 9

User. ID: Erika Open. Whisk Function Trigger function 1 Result: “Success!” del!” o m L M e the “Give m is” “Here it Store “Erik a’s Ca t, Bro wn T “Sav abby ed!" ” Cat Model Database Cat Database Improve Efficiency? 10

User. ID: Erika Open. Whisk Function Trigger function 1 Result: “Success!” del!” o m L M e the “Give m is” “Here it Store “Erik a’s Ca t, Bro wn T “Sav abby ed!" ” Cat Model Database Cat Database Improve Efficiency? 11

Long-lived connection or connection pool User. ID: Erika Microservice Request Handler Result: “Success!” … Request Handler del!” o m L M e the “Give m is” “Here it Store “Erik a’s Ca t, Bro wn T “Sav abby ed!" ” Cat Model Database Cat Database IDCat Microservice 12

Persistent connection or connection pool /init (code, extras) Open. Whisk Cat Model Database Cat Database Runtime Reuse in Serverless 13

Open. Whisk Function Trigger Cat Model Database function 1 Cat Database Runtime Reuse in Serverless 14

Open. Whisk Cat Model Database Function Trigger function 1 Cat Database Runtime Reuse in Serverless 15

Open. Whisk Function Trigger function 1 Cat Model Database Cat Database Runtime Reuse in Serverless 16

Open. Whisk Function Trigger function 1 Runtime Reuse in Serverless – Room for Improvement? Cat Model Database Cat Database 17

Background Freshen Design Evaluation Outline Discussion Questions 18

We propose a new serverless runtime primitive, freshen, as a mechanism to enable proactive serverless function resource management. Overview 19

Time=0 /init /run function 1 Freshen Design 20

Time=0 /init /freshen /run freshen function 1 Freshen Design 21

Time=0 /init /freshen /run freshen Prediction window function 1 Freshen Design 22

Time=0 /init /freshen /run freshen Prediction window Policy Options: • Prediction • Concurrency • Forced blocking function 1 Freshen Design 23

Time=0 /init Reuse Dynamic State Proactive Init Phase /run function 1 Function Code Runtime Reuse Freshen /run function 1 24

Time=0 /init Reuse Dynamic State Proactive Init Phase /run function 1 Function Code Runtime Reuse Freshen /run function 1 25

Time=0 /init Init Reuse Dynamic State Proactive Init Phase /run Function 1 Function Code Runtime Reuse Freshen /run Function 1 26

Time=0 /init Init Reuse Proactive Init Phase /freshen /run Dynamic State freshen Function 1 Function Code Runtime Reuse /freshen /run freshen Function 1 27

Serverless Function Prediction useful for many reasons (scheduling, resource utilization, coldstart avoidance, etc. ) Some cases are easier to predict, e. g. chained functions Many applications consist of multiple functions May be infrastructure overheads 28

Serverless Function Prediction Some cases are easier to predict, e. g. , chained functions Prediction useful for many reasons (scheduling, resource utilization, coldstart avoidance, etc. ) 29

Serverless Function Prediction Some cases are easier to predict, e. g. , chained functions Many applications consist of multiple functions Prediction useful for many reasons (scheduling, resource utilization, coldstart avoidance, etc. ) 30

Serverless Function Prediction useful for many reasons (scheduling, resource utilization, coldstart avoidance, etc. ) Some cases are easier to predict, e. g. , chained functions Many applications consist of multiple functions May be infrastructure overheads 31

What Can Freshen Do? 32

User. ID: Erika Open. Whisk Function Trigger Function 1 Result: “Success!” del!” o m L M e the “Give m is” “Here it Store “Erik a’s Ca t, Bro wn T “Sav abby ed!" ” Cat Model Database Cat Database What Can Freshen Do? 33

User. ID: Erika Open. Whisk Function Trigger Function 1 Result: “Success!” del!” o m L M PREFETCH e the “Give m is” “Here it Store “Erik a’s Ca t, Bro wn T “Sav abby ed!" ” Cat Model Database Cat Database What Can Freshen Do? 34

User. ID: Erika Open. Whisk Function Trigger Function 1 Result: “Success!” del!” o m L M e the “Give m is” “Here it Store “Erik a’s Ca t, Bro wn T “Sav abby ed!" ” WARM CONNECTION Cat Model Database Cat Database What Can Freshen Do? 35

User. ID: Erika Open. Whisk Function Trigger Function 1 Result: “Success!” del!” o m L M e the “Give m is” “Here it Store “Erik a’s Ca t, Bro wn T “Sav abby ed!" ” Cat Model Database Cat Database What Can Freshen Do? MORE? 36

Background Freshen Design Evaluation Outline Discussion Questions 37

Freshen Motivation PREFETCH WARM CONNECTION Reduces latency to access data 38

Freshen Motivation PREFETCH WARM CONNECTION TCP connections warmed send traffic more efficiently 39

Background Freshen Design Evaluation Outline Discussion Questions 40

Connection state manipulation • How to access? • Beyond TCP Function prediction • Scalability • Generality Who is responsible for freshen? • Developer • Libraries • Inference Discussion Other freshen actions • Memory allocation? • Caches? • Things we have not yet thought of? 41

Connection state manipulation • How to access? • Beyond TCP Function prediction • Scalability • Generality Who is responsible for freshen? • Developer • Libraries • Inference Discussion Other freshen actions • Memory allocation? • Caches? • Things we have not yet thought of? 42

Connection state manipulation • How to access? • Beyond TCP Function prediction • Scalability • Generality Who is responsible for freshen? • Developer • Libraries • Inference Discussion Other freshen actions • Memory allocation? • Caches? • Things we have not yet thought of? 43

Connection state manipulation • How to access? • Beyond TCP Function prediction • Scalability • Generality Who is responsible for freshen? • Developer • Libraries • Inference Discussion Other freshen actions • Memory allocation? • Caches? • Things we have not yet thought of? 44

Background Freshen Design Evaluation We propose a new serverless runtime primitive, freshen, as a mechanism to enable proactive serverless function resource management. Discussion Questions https: //www. serverlesscomputing. org/wosc 6/#p 11 Erika Hunhoff erika. hunhoff@colorado. edu 45