Remote Function Invocation Running a Single Program On

Thesis • Single Program • Single Computer • Serial Processing

Antithesis • Multiple Programs • Multiple Computers • Cooperate on Single Problem via Messages

Synthesis • Single Program • Multiple Computers

Raises Questions • Existing mechanisms? • Message-passing necessary? • Comparison of Metrics? • Anything

Objectives • Develop the mechanism • Benchmark RFI • Demonstrate: – Running existing program

Programs • Native Code • Virtual Platform RFI exists

Developing Rexel • Remote Execution Environment for Linux

Node Setup • Simple Comms – Store libfile. so – Register function – Call

Function Calling • Few major problems • Trade-offs acceptable • Uses dlopen() and dlsym()

Usage • Usage (mostly) automatic • Needs runtime linkage • librexelmaster. so – Configured

Test Setup • Benchmarked against MPI • Setup carefully controlled • System representative of

Summary Rexel MPI • Quicker at lots of smaller problems • Slower when number

Future? Rexel MPI • Performance improved towards the end • Consistency improved towards the

Running SQLite (1/2) • SQLite: – Small standalone database – Has c/line interface –

SQLite (2/2) • Run in LD_PRELOAD environment • Unaware of Rexel • Mixed results

Partial Shared-Memory • Expressions represented as trees • Leaf nodes are operands • Branch

Solving Expressions • Solved recursively • Independently

Research Answers • Mechanisms exist • Replacement possible for both shared-mem and MPI •

Future Opportunities • Can have an impact on program design • Quorum Solving •

Slides: 26

Download presentation

Remote Function Invocation Running a Single Program On Multiple Computers

Thesis • Single Program • Single Computer • Serial Processing

Antithesis • Multiple Programs • Multiple Computers • Cooperate on Single Problem via Messages

Synthesis • Single Program • Multiple Computers

Raises Questions • Existing mechanisms? • Message-passing necessary? • Comparison of Metrics? • Anything we can exploit? Use?

Objectives • Develop the mechanism • Benchmark RFI • Demonstrate: – Running existing program on cluster – Exploitation of new opportunities presented by RFI

Programs • Native Code • Virtual Platform RFI exists

Developing Rexel • Remote Execution Environment for Linux

Node Setup • Simple Comms – Store libfile. so – Register function – Call function

Function Calling • Few major problems • Trade-offs acceptable • Uses dlopen() and dlsym() • Uses Libffi for calling convention

Usage • Usage (mostly) automatic • Needs runtime linkage • librexelmaster. so – Configured exclusively from config files

Test Setup • Benchmarked against MPI • Setup carefully controlled • System representative of cloud providers • Problem-space exhaustion

Speed (1/2)

Speed (2/2)

Speed Overall Predictability

Consistency (1/2)

Consistency (2/2)

Summary Rexel MPI • Quicker at lots of smaller problems • Slower when number of repeated calls increase • Predictable and consistent throughout • Slower when problem is divided into tiny chunks • Faster as problem-size of each chunk grows • Unpredictable with tested criteria

Future? Rexel MPI • Performance improved towards the end • Consistency improved towards the end

Running SQLite (1/2) • SQLite: – Small standalone database – Has c/line interface – Stores all data in a single file

SQLite (2/2) • Run in LD_PRELOAD environment • Unaware of Rexel • Mixed results – Successfully wrote files – Nodes could not return errors

Partial Shared-Memory • Expressions represented as trees • Leaf nodes are operands • Branch nodes are operators

Solving Expressions • Solved recursively • Independently

Concurrent Solving

Research Answers • Mechanisms exist • Replacement possible for both shared-mem and MPI • Large effects on software design • Multiple opportunities to exploit

Future Opportunities • Can have an impact on program design • Quorum Solving • Fine-grained Unittesting • More? ? ?