Ashish Gupta Bin Lin Peter Dinda Department of

Ashish Gupta Bin Lin Peter Dinda Department of Computer Science Northwestern University 1

Outline • Motivation behind Resource Borrowing • Measuring user comfort • System Design for Measuring User Comfort • Controlled Study • Its Results • Conclusions 3

New Models of Distributed Computing 9 am 11 am Working 1 pm Break Screen Saver time 3 pm 4 pm 5 pm 9 am 4

Observation and ideas Unused Resources Idea: Why not borrow the unused resources ? Problem: Performance Slowdown 5

Effect of performance slowdown 6

Effect of performance slowdown 7

Effect of performance slowdown 8

The question we want to answer What level of resource borrowing leads to user discomfort for significant fraction of users ? Depends on Other Factors as well… 9

CPU RAM Disk 10

Context Person Hardware Others ? 11

Power Users Typical Users Beginners 12

13

Outline • Motivation behind Resource Borrowing • Measuring user comfort • System Design for Measuring User Comfort • Controlled Study • Its Results • Conclusions 14

How can we find this out ? Actually test it with user feedback A system for emulating resource borrowing and measuring user discomfort using direct feedback http: //comfort. cs. northwestern. edu 15

How can we find this out ? Measure existing contention 1 16

How can we find this out ? Emulate Resource Borrowing 2 % CPU Usage vs. Time Final Contention Borrowed Contention Real Contention 17

How can we find this out ? Allow User Feedback 3 F 11 18

How can we find this out ? Find MAX Level of tolerable contention 4 % CPU Usage vs. Time Final Contention MAX Borrowed Contention 19

How is it useful ? Comfort = f (Contention) To find a quantitative relationship between resource borrowing and user comfort 20

How is it useful ? Enhances the degree of workstation sharing 21

Interactivity Aware Scheduling Interactive More intelligent scheduling of interactive tasks possible via user feedback and our results 22

Virtual Machine Scheduling • Virtuoso: Distributed Computing with Virtual Machines • User pays for the resources he uses • Need to know minimal resources needed to keep an interactive VM user happy 23

Outline • Motivation behind Resource Borrowing • Measuring user comfort • System Design for Measuring User Comfort • Controlled Study • Its Results • Conclusions 24

The client design 26

Testcases • Each testcase stores details of resource borrowing • Testcase consists of exercise functions for each resource • Exercise function is a vector of contention values • Example: CPU contention: [0, 0. 5, 1. 0, 1. 5, 2. 0] at 1 Hz 27

Flowchart of Testcase Execution User feedback No Testcase running Resource Borrowing Phase Begins (Testcase Runs) No user feedback User expressed discomfort Testcase Exhausts after some time Testcase Ends immediately 29

Resource Exercisers • CPU Exerciser – Contention describes the expected extra number of threads in ready queue – Fractional resource borrowing using stochastic scheduling methods – Validated to contention level of 10 • Disk Exerciser – Random seek and read/write in a large file (twice the memory) – Validated to contention level of 7 • Memory Exerciser – Borrows a fraction of physical memory: from 0 to 1 30

$Memory Exerciser • Contention indicates fraction of total memory allocated in addition to occupied$

Memory Exerciser • Contention indicates fraction of total memory allocated in addition to occupied memory • Pages are touched frequently to keep them in physical memory Occupied Mem. 0 MB 0. 5 Memory Contention 1024 MB Clipping Point 31

32

Outline • Motivation behind Resource Borrowing • Measuring user comfort • System Design for Measuring User Comfort • Controlled Study • Its Results • Conclusions 33

The control study description • 33 undergrad and grad students – Note: may not be rep. of PC users population • 1. 5 hrs each • Each user was assigned 4 popular tasks to do – MS Word – MS Powerpoint – MS Explorer searching and saving information – Quake III 34

Machine Configuration 35

Testcases for the Control Study Three types of testcases: step, ramp and blank • Ramp allows us to know the point of discomfort for each resource • Step and Ramps: comparing the manner of borrowing resources • Blank: To test the background level of discomfort 36

Testcases for the Control Study • Testcases different for each application – Some apps more intensive than others • Example of Word and Quake • Requires Calibration 38

Outline • Motivation behind Resource Borrowing • Measuring user comfort • System Design for Measuring User Comfort • Controlled Study • Its Results • Conclusions 39

Resource borrowing vs User Discomfort • CDFs for discomfort contention level – Not all contentions cause discomfort: exhausted region • CDFs allow us to read %age of people discomforted for a given contention – We want to borrow resources such that only a small fraction are discomforted – Metric c 0. 05 : At what contention do we discomfort only 5% of the people ? 40

Aggregated Results Exhausted Region Discomfort Region • C 0. 05 = 0. 35 ( aggregated over all applications) 41

Resource borrowing vs User Discomfort • CDFs for discomfort contention level – Not all contentions cause discomfort: exhausted region • CDFs allow us to read %age of people discomforted for a given contention – We want to borrow resources such that only a small fraction are discomforted – Metric c 0. 05 : At what contention do we discomfort only 5% of the people ? 42

Results overview Aggregated Results Dependence on Resource Dependence on Context Dependence on Users Dependence on Step/Ramp 43

Aggregated Results 44

Aggregated Results Exhausted Region Discomfort Region • C 0. 05 = 0. 35 ( aggregated over all applications) 45

Aggregated Results Exhausted Region Discomfort Region • Can borrow a lot, discomforting only a small fraction • C 0. 05=0. 33 46

Aggregated Results Exhausted Region Discomfort Region • Can borrow a lot, discomforting only a small fraction • C 0. 05 = 1. 11 47

Dependence on Resource 48

An overview of factors Dependence on Resource User sensitivity by task and resource (Low, Medium, High) Users more tolerant for disk and memory borrowing, than CPU Results Qualitative , but the testcase range is wide 49

Dependence on Resource CPU 50

Dependence on Resource Memory 51

Dependence on Resource Disk 52

Dependence on Context 53

Comparison of CPU discomfort Dependence on Context Quake Powerpoint IE Word 54

Dependence On Context - CPU Dependence on Context Word Powerpoint IE Quake 55

Dependence on Context - Memory Word Powerpoint IE Quake 56

Dependence on Context - Disk Dependence on Context Powerpoint Word IE Quake 57

Dependence on User 59

Dependence on User Beginner Typical Power 60

Internet Wide-Area Study • To gain more insight into our results • No controls here, hope to get more general and complete data • Users can download a client from http: //comfort. cs. northwestern. edu • Clean installation/un-installation and privacy options Join us ! 64

Outline • Motivation behind Resource Borrowing • Measuring user comfort • System Design for Measuring User Comfort • Controlled Study • Its Results • Conclusions 65

Conclusion • Disk and memory can be borrowed more aggressively, CPU less so – Currently Seti@home, P 2 P file sharing are very conservative in disk and memory usage • A throttle for resource borrowing very useful – Don’t run full blast. Have the ability to carefully control how much you borrow through stochastic scheduling • CDFs can be used to set the throttle • Knowing the user context helps • Using Direct user feedback can be useful – We are moving ahead in this direction, esp in scheduling of Virtual Machines for satisfactory interactive performance 66

Conclusion Comfort = f(contention) Disk and Memory surprisingly easy to borrow, CPU borrowing also liberal Set of empirical CDFs provided Depends of other factors: Resource, Context and Users 67

Current and Future Work User feedback for interactive VM Scheduling Real time Scheduling to give exact resources to VMs Minimal resources required to keep the user happy 68

Acknowledgements http: //comfort. cs. northwestern. edu Special Thanks to: Don Norman Andrew Ortony Ben Watson Jack Tumblin Leslie Lamport and All our beta testers Study participants 69

The placebo effect • Users express discomfort even when no testcase is running • Can never guarantee that the user will never express feedback 70

Related Work 71

Questions and Answers • • • Other types of irritation Bursty natures of Desktop Computing, bursts indicate sudden action-response tasks Large number of factors for irritation Since there is usually a power law for process run times, why do we need this model ? (CPU is idle most of the time) Why can’t we just run the bg app at a lower priority ? It may benefit much more than just running at a fixed contention. Memory exerciser: What does contention mean ? How do you find out free physical memory ? How do you avoid thrashing ? Why do you avoid thrashing ? How does the CPU exerciser work ? Why do you disable buffering in the Disk exerciser ? The aggregated graphs are not useful. Memory usage may be much more dynamic then in your tests due to user multitasking etc. So this may not represent reality ? – Our study is a start in this direction and we realize that there are limitations to the data we have collected. But we expect to collect and analyze more results from the Internet Wide area study and generalize our results to more real environments. There are no controls in the internet wide area study. The control study was meant to specifically answer the questions we have posed, like the affect of other factors in user discomfort and resource borrowing. 72

• Lower priority ? – Disk and memory ? – There is a limit (max) possible amount of cycles stolen. Our goal is to s=estimate that. Seperately, there are many possible sched mech that would get closer to that max. Lower priority is an example. Linger longer is an example. Screen saver is an example. – Quake example – Don’t argue for/against a particular scheduling scheme – It is a step in the right direction… (for results) 73