Multithreading Design Patterns Surmounting complexity by raising abstraction

Multithreading Design Patterns Surmounting complexity by raising abstraction Gaël Fraiteur Post. Sharp Technologies Founder & Principal Engineer @gfraiteur

Hello! My name is GAEL. No, I don’t think my accent is funny. my twitter @gfraiteur

My commitment to you: to seed a vision, not to sell a tool. @gfraiteur

The vision: Code at the right level of abstraction, with compiler-supported design patterns @gfraiteur

back in 1951 @gfraiteur

hardware was fun but quite simple to use @gfraiteur

Univac Assembly Language @gfraiteur

FORTRAN (1955) • 1955 - FORTRAN I • Global Variables • Arrays • 1958 - FORTRAN II • Procedural Programming (no recursion) • Modules @gfraiteur

COBOL (1959) • Data Structures @gfraiteur

LISP (1959) • Variable Scopes (Stack) • Garbage Collection @gfraiteur

“ The new invention caught quickly, no wonder, programs computing nuclear power reactor parameters took now HOURS INSTEAD OF WEEKS to write, and required much LESS PROGRAMMING SKILL @gfraiteur

1. The Memory Management Revolution Section 2. Models and Patterns 3. Defining Threading Models 4. Designing with Threading Models 5. A Few Threading Models 6. Q&A 7. Summary @gfraiteur

How do programming languages increase productivity? @gfraiteur

What you may think the compiler does for you. Instruction Generation Code Validation @gfraiteur

Compilers translate code FROM HIGH TO LOW ABSTRACTION language @gfraiteur

Languages let us express ourselves against a model Thing ∞ ∞ abstracted into 1 World abstracted into 1 Concept 1 1 expressed with Word ∞ ∞ 1 1 Model expressed with Language @gfraiteur

Good Models are Easy • Allow for succinct expression of intent (semantics), with less focus on implementation details • less work • fewer things to think about • Allow for extensive validation of source code against the model • early detection of errors • Allow for better locality and separation of concerns • “everything is related to everything” no more • Are deterministic • no random error @gfraiteur

Good Models are Friendly • to human mind structure • cope with limited human cognitive abilities • to social organization • cope with skillset differences, division of labor, time dimension, changes in requirements @gfraiteur

The Classic Programming Model Quality Realization Succinct semantics • Concept of subroutine • Unified concept of variable (global, local, parameters, fields) Validation • Syntactic (spelling) • Semantic (type analysis) • Advanced (data flow analysis) Locality • Information hiding (member visibility) • Modularity Determinism • Total (uninterrupted single-threaded program) @gfraiteur

1. The Memory Management Revolution 2. Models and Patterns Section 3. Defining Threading Models 4. Designing with Threading Models 5. A Few Threading Models 6. Q&A 7. Summary @gfraiteur

Why we need threading models • Multithreading is way too hard – today • Too many things to think about • Too many side effects • Random data races • Your colleagues won’t get smarter • Increased demand – end of free lunch @gfraiteur

@gfraiteur

The Root of All Evil Access to Shared Mutable State • Memory ordering • Atomicity / transactions • Typical damage: data races • Corruption of data structures • Invalid states @gfraiteur

Locks alone are not the solution @gfraiteur

@gfraiteur

Locks alone are not the solution • Easy to forget • Difficult to test • Deadlocks @gfraiteur

"Problems cannot be solved by the same level of thinking that created them. " (and you’re not a man if you haven’t cited Albert Einstein) @gfraiteur

Threading Models Avoid Shared State Reader-Writer Sync’ed Pessimistic Transactions Thread Exclusive Optimistic Transactions Actor Avoid Shared Mutable State Transactional Lock-Based Immutable Avoid Mutable State Lock-Free @gfraiteur

Threading Models are… Models Design Patterns • Abstraction • Named solution • Automation • Best practice • Validation • Documented • Determinism @gfraiteur

1. The Memory Management Revolution 2. Models and Patterns 3. Defining Threading Models Section 4. Designing with Threading Models 5. A Few Threading Models 6. Q&A 7. Summary @gfraiteur

Golden rule: all shared state must be thread-safe. @gfraiteur

1. Assign threading models to types 1. Every class must be assigned a threading model. 2. Define aggregates with appropriate granularity. @gfraiteur

@gfraiteur

2. Ensure only thread-safe types are shared 1. Arguments of cross-thread methods 2. All static fields must be read-only and of thread-safe type. @gfraiteur

1. The Memory Management Revolution 2. Models and Patterns 3. Defining Threading Models 4. Designing with Threading Models Section 5. A Few Threading Models 6. Q&A 7. Summary @gfraiteur

Immutable Pattern Never changed after creation. Make a copy if you want to modify it. Issue: multi-step object initialization (e. g. deserialization) @gfraiteur

Freezable Pattern 1. Implement interface IFreezable Freeze() must propagate to children objects. 2. Before any non-const method, throw exception if object is frozen. 3. Call the Freeze method after any initialization. @gfraiteur

Thread Exclusive Promises never to be involved with multithreading. Ever. @gfraiteur

Thread Exclusivity Strategies • Exclusivity Policies: • Thread Affinity (e. g. GUI objects) • Instance-Level Exclusivity (e. g. most. NET objects) • Type-Level Exclusivity • Concurrency Behavior: • Fail with exception • Wait with a lock @gfraiteur

Implementing Thread Affine Objects • Remember current thread in constructor • Verify current thread in any public method • Prevent field access from outside current instance • Only private/protected fields • Only accessed in “this. field” @gfraiteur

Implementing Thread Excluse Objects • Acquire lock before any public method • Wait/throw if it cannot be acquired • Prevent field access from outside current instance • Only private/protected fields • Only accessed in “this. field” • Note: “Throw” behavior is non-deterministic @gfraiteur

Stricter coding constraints increase code verifiability. @gfraiteur

Lab: Checking Thread Exclusivity @gfraiteur

Actor Model Bound to a single thread (at a time) @gfraiteur

Message Queue Single Worker Actors: No Mutable Shared State Private Mutable State @gfraiteur

Composing Actors @gfraiteur

Actor Sequence Diagram @gfraiteur

Actor implementations • with compiler support: • Erlang, • without compiler support: • F# Mailbox • NAct • Post. Sharp • Actor. FX @gfraiteur

Verifying the Actor Model • Public methods will be made async • Parameters and return values of public methods must be thread-safe • Methods with return value must be async • Prevent field access from outside current instance • Only private/protected fields • Only accessed in “this. field” @gfraiteur

Lab: Implementing Actors @gfraiteur

Actors Benefits Limitations • Race free (no shared mutable state) • Performance • Lock free (no waiting, no deadlock) • Scales across processes, machines • Theoretical Latency: min 20 ns (L 3 double-trip) • Practical throughput (ring buffer): max 6 MT/s per core • Difficult to write without proper compiler support • Reentrance issues with waiting points (await) @gfraiteur

Reader-Writer Synchronized Everybody can read unless someone writes. @gfraiteur

Reader-Writer Synchronized • 1 lock per [group of] objects • e. g. invoice and invoice lines share the same lock • Most public methods require locks: Method Required Lock Level Reads 1 piece of state None Reads 2 pieces of state Read Writes 1 piece of state Write Reads large amount of state, then writes state Upgradeable Read, then Write @gfraiteur

Lab: Implementing RWS @gfraiteur

Transactions • Isolate shared state into thread-specific storage • Commit/Rollback semantics • Platform ensures ACID properties: • Atomicity, Consistency, Isolation, (Durability) • Type of concurrency policies • Pessimistic (lock-based: several “isolation levels” available) • Optimistic (lock-free, retry-based) “ As far as I'm concerned, I prefer silent vice to ostentatious virtue. Albert Einstein @gfraiteur

Q&A Gael Fraiteur gael@postsharp. net @gfraiteur

Summary • Repeat the memory management success with the multicore issue. • Models decrease complexity to a cognitively bearable level. • We need compilers that allow us to use our own models and patterns. @gfraiteur BETTER SOFTWARE THROUGH SIMPLER CODE