Foundations of Shared Memory Nir Shavit Multiprocessor Synchronization

Foundations of. Shared Memory Nir Shavit Multiprocessor Synchronization Spring 2003 3/11/2021 © 2003 Herlihy & Shavit 2

Fundamentals • What is the weakest form of communication that supports mutual exclusion? • What is the weakest shared object that allows asynchronous computation? © 2003 Herlihy & Shavit

Shared Memory • Modern multiprocessors provide rich set of primitive operations – Read (sometimes called load) – Write (sometimes called store) – Complex atomic operations • Swap, test-and-set, compare-and-swap, etc. Practical Theoretical algorithms bounds live here (3) © 2003 Herlihy & Shavit

Pons Asinorum • Literally: – The “Asses’ Bridge”: Theorem 5, Book 1 of Euclid's Elements • Figuratively: – You will not make use of it directly, but don’t leave home without it … • Asynchronous Computability: The pons asinorum of concurrent programming © 2003 Herlihy & Shavit

Asynchronous Computability • Why do we need this? Lets write code man… • Do you know this guy? • Alan Turing: helped us understand what is and is not computable on a sequential machine. 3/11/2021 © 2003 Herlihy & Shavit 6

Turing Computability T-Machine f r #a r a zrw 4 p t l r s r 0 1 g a k 0 3 0 • A mathematical model of computation • Defines what is and is not computable • Complexity irrelevant to real machines © 2003 Herlihy & Shavit

Asynchronous Computability 10011 Shared Memory • A model of concurrent computation • What is/is not asynchronously computable • Complexity irrelevant to real machines © 2003 Herlihy & Shavit

Foundations of Shared Memory To understand the computational power of hardware and software on real machines, lets see what we can build given the weakest type of shared concurrent object 10011 © 2003 Herlihy & Shavit

Register Holds a (binary) value 10011 © 2003 Herlihy & Shavit

Register Can be read 10011 © 2003 Herlihy & Shavit

Register Can be written 01100 10011 © 2003 Herlihy & Shavit

Register public interface Register { public int read(); public void write(int v); } © 2003 Herlihy & Shavit

Single-Writer/Single-Reader Register 10011 01100 10011 © 2003 Herlihy & Shavit

Single-Writer/Multi-Reader Register 10011 01100 10011 © 2003 Herlihy & Shavit

Multi-Writer/Multi-Reader Register mumble 10011 01010 10011 11011 © 2003 Herlihy & Shavit

Safe Register OK if reads and writes don’t overlap write(1001) read(1001) (2) © 2003 Herlihy & Shavit

Safe Register write(1001) Some valid value if reads and writes do overlap read(? ? ) 0000 1001 © 2003 Herlihy & Shavit $*&v 1111

Regular Register write(0) write(1) read(0) Safe + Concurrent read returns either old or new value (4) © 2003 Herlihy & Shavit

Regular ≠ Linearizable write(0) write(1) read(0) explain this! write(1) already happened (5) © 2003 Herlihy & Shavit

Atomic Register write(1001) write(1010) read(1001) read(1010) Linearizable to sequential safe register (5) © 2003 Herlihy & Shavit

Register Space MRMW MRSW Safe 3/11/2021 k-valued boolean Regular Atomic © 2003 Herlihy & Shavit 22

Register Names Readers/writers property bool or what? public class MRMW_Safe_Bool implements Register { public int read(); public void write(int x); } (3) © 2003 Herlihy & Shavit

Weakest Register 1 Single writer 0 Single reader 1 Safe Boolean register © 2003 Herlihy & Shavit

Results • From SRSW safe boolean register – All the other registers – Mutual exclusion • But not everything! Foundations of the field – Consensus hierarchy Non-blocking concurrent data structures (2) © 2003 Herlihy & Shavit

Locking within Registers • Cannot rely on mutual exclusion in register constructions • Can’t use mutual exclusion to implement itself! © 2003 Herlihy & Shavit

Wait-Free Implementations Definition: An object implementation is wait-free if every thread completes a method in a finite number of steps No mutual exclusion – Thread could halt in critical section – Build mutual exclusion from registers © 2003 Herlihy & Shavit

Road Map • • SRSW safe boolean MRSW regular omitted MRSW atomic MRMW atomic Atomic snapshot © 2003 Herlihy & Shavit

SRSW Safe MRSW safe public class MRSW_Safe implements Register { SRSW_Safe[n] register; public void write(int x) { for (int j=0; j<n; j++) register[i]. write(x); } Write each component public int read() { int i = thread. my. Index(); return register[i]. read(); }} (2) © 2003 Herlihy & Shavit Read my component

MRSW Safe Bool MRSW Regular Bool public class MRSW_Rglr implements Register { private bit old; private MRSW_Safe value; Last bit we wrote public void write(bit x) { if (old != x) { value. write(x); old = x; }} public bit read() { return value. read(); }} (2) © 2003 Herlihy & Shavit Actual value

MRSW Safe Bool MRSW Regular Bool public class MRSW_Rglr implements Register { private bit old; private MRSW_Safe value; Is new value different? public void write(bit x) { if (old != x) { value. write(x); old = x; }} public bit read() { return value. read(); }} (2) © 2003 Herlihy & Shavit Change it

MRSW Safe Bool MRSW Regular Bool public class MRSWReg. Reg implements Register { private bit old; private MRSWSafe. Reg value; • No overlap? No problem public void write(bit x) { • Overlap? if (old !=Either x) { Boolean value works value. write(x); old = x; }} public bit read() { return value. read(); }} (1) © 2003 Herlihy & Shavit

MRSW Regular Boolean MRSW Regular M-valued public class MRSW_Rglr implements Register { MRSW_Rglr_Bool[m] bit; public void write(int x) { this. bit[x]. write(1); for (int i=x-1; i>=0; i--) this. bit[i]. write(0); } Viewer discretion advised public (sorry, int read() { tuition is non-refundable) for (int i=0; i<M; i++) if (this. bit[i]. read()==1) return i; }} (1) © 2003 Herlihy & Shavit

MRSW Regular Boolean MRSW Regular M-valued public class MRSW_Rglr implements Register { MRSW_Rglr_Bool[m] bit; … } Unary representation: bit[i] means value i (1) © 2003 Herlihy & Shavit

Writing M-Valued public class MRSW_Rglr implements Register { Set bit x MRSW_Rglr_Bool[m] bit; public void write(int x) { this. bit[x]. write(1); for (int i=x-1; i>=0; i--) this. bit[i]. write(0); } … } Clear lower bits (2) © 2003 Herlihy & Shavit

Writing M-Valued Write “ 5” Initially “ 0” 01 0 0 0 1 01234567 © 2003 Herlihy & Shavit

Reading M-Valued public class MRSW_Rglr implements Register { MRSW_Rglr_Bool[m] bit; … public int read() { for (int i=0; i<M; i++) if (this. bit[i]. read()==1) return i; }} Find & return first bit set (1) © 2003 Herlihy & Shavit

Reading M-Valued Write “ 5” 0 01 0 0 0 1 57 0123456 © 2003 Herlihy & Shavit

MRSW Regular MRSW Atomic • Complicated and boring Of interest mostly to specialists • Sounds like a homework problem … © 2003 Herlihy & Shavit

MRSW Atomic MRMW Atomic 1101111010100010 Label: incremented on each write (2) © 2003 Herlihy & Shavit value

MRSW Atomic MRMW Atomic 1101111010100010 get. Label(…) new. Entry(…) (2) © 2003 Herlihy & Shavit get. Value(…)

MRSW Atomic MRMW Atomic public class MRMW_atomic { private MRSW_atomic[n] value; One MRSW void write(int x) { int max = 0; register/thread int i = Thread. my. Index(); for (int j=0; j<n; j++) max = Math. max(max, value[j]); value[i] = new. Entry(get. Label(max)+1, x); } Max in lexicographic order (4) New entry with higher label © 2003 Herlihy & Shavit

MRSW Atomic MRMW Atomic public class MRMW_atomic { private MRSW_atomic[n] value; … int read() { Find entry with highest int m = -1; label in lex order int max = 0; for (int j=0; j<n; j++) int v = value[j]. read(); if ((get. Label(v), j)>(get. Label(max), m)) { max = value; m = j; } return get. Value(max); } (3) (1) © 2003 Herlihy & Shavit

Atomic Snapshot update © 2003 Herlihy & Shavit scan

Atomic Snapshot • Array of SWMR atomic registers • Take instantaneous snapshot of all • Generalizes to MRMW registers … © 2003 Herlihy & Shavit

Snapshot Interface Write v to register i public interface Snapshot { public int update(int i, int v); public int[] scan(); } Instantaneous shapshot (2) © 2003 Herlihy & Shavit

Atomic Snapshot • Collect – Read values one at a time • Problem – Incompatible concurrent collects – Result not linearizable © 2003 Herlihy & Shavit

Clean Collects • Clean Collect – Collect during which nothing changed – Can we make it happen? – Can we detect it? © 2003 Herlihy & Shavit

Simple Snapshot • Put increasing labels on each entry • Collect twice • If both agree, – We’re done • Otherwise, – Try again Collect 2 Collect 1 1 1 22 1 7 13 18 12 © 2003 Herlihy & Shavit = 7 13 18 12

Simple Snapshot public Class Simple. Snapshot implements Snapshot{ private SRSW_atomic[] value; public int update(int i, int v){ int label = get. Label(value[i]); value[i] = new. Entry(label+1, v); }} Write with higher label (1) © 2003 Herlihy & Shavit

Simple Snapshot Keep trying public int[] scan(){ int[] copy = new int[N]; Collect: while(true){ for (int j=0; j<N; j++) copy[i]=this. value[i]; for (int j=0; j<N; j++) if(copy[i]!=this. value[i]) continue Collect; return copy; Collect }} (3) collect again, restart on mismatch © 2003 Herlihy & Shavit

Simple Snapshot • Linearizable • Update is wait-free – No unbounded loops • But Scan starve – If interrupted by concurrent update © 2003 Herlihy & Shavit

Wait-Free Snapshot • Add a scan before every update • Write resulting snapshot together with update value • If scan is continuously interrupted by updates, scan take the update’s snapshot © 2003 Herlihy & Shavit

Wait-free Snapshot If a scan reads that thread “moved” twice, that thread executed a complete update within the interval of the scan. Collect 2 Collect 1 Collect 4 Collect 3 Update 1 1 22 1 7 7 13 13 18 12 7 13 18 12 = © 2003 Herlihy & Shavit = 7 13 18 12

One Move is Not Enough 1 1 22 1 7 13 = 13 18 12 My scan update write scan Update by A © 2003 Herlihy & Shavit 7

Its Wait-free B scan update So some thread must have clean simple collect © 2003 Herlihy & Shavit

Expanded Array Entry 11011110101000101100… 00 Label: incremented on each write (3) value © 2003 Herlihy & Shavit Last snapshot

Wait-free Snapshot Take scan public class WFSnapshot implements Shapshot{ private SRSW_atomic[] value; public void update(int i, int v) { int[] snapshot = this. scan(); int label = this. value[i]. read(); this. value. write(i, new. Entry(label+1, v, snapshot); }} New label, value, snapshot (2) © 2003 Herlihy & Shavit

Collect method private int[] collect() { int[] copy = new int[N]; for (int i=0; i<n; i++) copy[i] = this. value[i]. read(); return copy; } © 2003 Herlihy & Shavit

Wait-free Snapshot Copy of array private int[] scan() { int[] copy; boolean moved = new boolean[N]; … } Did other thread move twice? (2) © 2003 Herlihy & Shavit

Wait-free Snapshot collect public int[] scan(){ … Collect: while(true){ copy = collect() for (int j=0; j<N; j++) if (copy[i]!=this. value[i]. read()) { if (moved[j]) { return get. Scan(this. value[j]. read()) } else { moved[j] = true; continue Collect; }} return copy; nd }} Take snapshot on 2 (2) © 2003 Herlihy & Shavit move

Observations • Uses unbounded counters – can be replaced with 2 bits • Assumes SWMR registers – for labels – Can be extended to MRMW © 2003 Herlihy & Shavit

Grand Challenge • Snapshot means – Write any one array element – Read multiple array elements © 2003 Herlihy & Shavit

Grand Challenge What about Multiple Locations? B 3/11/2021 A Writes to 0 and 1 Writes to 1 and 2 © 2003 Herlihy & Shavit Write many and Snapshot 64