Theoretical Foundations Clocks Logical clocks Lamport Partially ordered

Theoretical Foundations ▸ Clocks ▹ Logical clocks (Lamport) ▹ Partially ordered clocks (Fidge) ▸ Causally ordered message delivery ▹ Broadcast-based (Birman et al. ) ▹ Unicast-based (Raynal, Schiper & Toueg) ▸ System states ▹ Global snapshots (Chandy & Lamport) ▹ Termination detection (Huang) Theoretical Foundations 1

“Happened Before” Conditions ▸ Sequential Behavior: If events e and f occur in the same process p, and f occurs after e, then e → f. ▸ Process Creation: If event e and process q occur in the same process p, and event f occurs in q, and q begins after e, then e → f. ▸ Process Termination: If event e and process q occur in the same process p, and event f occurs in q, and e occurs after q terminates, then f → e. ▸ Synchronous Message-Passing: If event e is a synchronous send, and event f is the corresponding receive, and there is an event g such that e → g, then f → g. If there is an event h such that h → e, then h → f. ▸ Asynchronous Message-Passing: If event e is an asynchronous send, and event f is the corresponding receive, then e → f. ▸ Transitivity: If e → f and f → g, then e → g. Theoretical Foundations > Partially Ordered Clocks > Happened Before Relation 2

Implementation Rules ▸ A. Initialization: When the main process m begins, its clock is initialized to Cm = { }. ▸ B. Ticking: Whenever a process i performs an event, it increments Ci(i). ▸ C. Monotonically Increasing Counters: No counter is ever decreased. ▸ D. Process Creation: Whenever a process j is created by process i, Cj is set equal to Ci. ▸ E. Process Termination: Whenever a process j terminates, the clock of the parent process i is set to Ci = max(Ci, Cj). ▸ F. Synchronous Events: During a synchronous event, all processes involved maximize their local clocks (i. e. , ∀x: [j…r] Cx = max(Cj, …, Cr) ). ▸ G. Sending: Whenever a process i sends a message, it timestamps it the current value of Ci. ▸ H. Receiving: Upon receiving a message m, the clock of the receiving process i is set to Ci = max(Ci, Cm). ▸ I. Timestamping: Events are timestamped with the current clock value. Theoretical Foundations > Partially Ordered Clocks > Implementation Rules 3

Comparison Property ▸ Given two timestamps tei and tfj, event ei happened before event fj iff tfj has knowledge of process i as recent as the execution of ei but not vice versa ei fj (tei(i) ≤ tfj(i)) ∧ (tei(j) < tfj(j)) Theoretical Foundations > Partially Ordered Clocks > Comparison Property 4

Optimization: Vector Clocks ▸ Assume a static number of processes ▸ Use an array of counters instead of pairs ▸ Pros: ▹ Provide an upper bound on storage requirements for clock variables ▹ Clock variables are easier to read and manipulate ▸ Cons: ▹ The upper bound on storage requirements is also the lower bound Theoretical Foundations > Partially Ordered Clocks > Vector Clocks 5

Vector Clock Exercise [0 0 0] P Q p 1 = [1 0 0] Tm = [1 0 0] p 2 = [2 1 0] Tm = [0 1 0] q 1 = [0 1 0] q 2 = [1 2 0] Tm = [0 1 0] r 1 = [0 0 1] p 3 = [3 1 0] p 4 = [4 5 0] Tm = [1 5 0] q 4 = [1 4 0] q 6 = [1 6 0] q 7 = [1 7 2] q 3 = [1 3 0] T = [1 4 0] q 5 = [1 5 0] m Tm = [0 0 2] r 2 = [0 0 2] r 3 = [1 4 3] R ▸ Vector = [P Q R] ▸ r 2 q 4 ? No, (2 ≤/ 0) ^ (0 < 4) r 4 = [1 4 4] [0 0 0] Theoretical Foundations > Partially Ordered Clocks > Vector Clocks 6

Theoretical Foundations ▸ Clocks ▹ Logical clocks (Lamport) ▹ Partially ordered clocks (Fidge) ▸ Causally ordered message delivery ▹ Broadcast-based (Birman et al. ) ▹ Unicast-based (Raynal, Schiper & Toueg) ▸ System states ▹ Global snapshots (Chandy & Lamport) ▹ Termination detection (Huang) Theoretical Foundations 7

Theoretical Foundations: Broadcast-based Causal Delivery CS/CE/TE 6378 Advanced Operating Systems

Ordering of Message Delivery ▸ Example: redundant, distributed database X Y R 1 create(R 1) Z Theoretical Foundations > Causal Delivery 9

Ordering of Message Delivery ▸ Example: redundant, distributed database R 1 X Y R 1 create(R 1) Z Theoretical Foundations > Causal Delivery 10

Ordering of Message Delivery ▸ Example: redundant, distributed database R 1 X R 1 update(R 1) Y R 1 create(R 1) Z Theoretical Foundations > Causal Delivery 11

Ordering of Message Delivery ▸ Example: redundant, distributed database R 1 X R 1 update(R 1) Y R 1 create(R 1) Z ? Theoretical Foundations > Causal Delivery 12

Ordering of Message Delivery ▸ Example: redundant, distributed database R 1 X R 1 update(R 1) Y R 1 Problem create(R 1) R 1 create(R 1) Z ? R 1 Theoretical Foundations > Causal Delivery 13

Basics of Message Delivery ▸ sendp(m) ▹ Transmission of message m by process p to a set of destinations denoted destinations(m) ▸ receiveq(m) ▹ Reception of message m by process q ▹ sendp(m) receiveq(m) ▸ deliverq(m) ▹ Delivery of message m to process q ▹ receiveq(m) deliverq(m) Theoretical Foundations > Causal Delivery 14

Causal Delivery ▸ Causal Delivery: ▹ If sendi(w) sendj(m), and q ∈ destinations(w) and q ∈ destinations(m), then deliverq(w) deliverq(m) ▸ What’s an example of a causal delivery protocol? ▹ Transmission Control Protocol (TCP) Theoretical Foundations > Causal Delivery 15

Causal Delivery Example ▸ Violation of casual delivery? Yes R 1 X R 1 update(R 1) Y R 1 create(R 1) Z ? R 1 Theoretical Foundations > Causal Delivery 16

Causal Delivery Example ▸ send. Y(create) send. X(update)? Yes R 1 X R 1 update(R 1) Y R 1 create(R 1) Z ? R 1 Theoretical Foundations > Causal Delivery 17

Causal Delivery Example ▸ Z ∈ both destinations()? Yes R 1 X R 1 update(R 1) Y R 1 create(R 1) Z ? R 1 Theoretical Foundations > Causal Delivery 18

Causal Delivery Example ▸ deliver. Z(create) deliver. Z(update)? No R 1 X R 1 update(R 1) Y R 1 create(R 1) Z ? R 1 Theoretical Foundations > Causal Delivery 19

Causal Delivery Example ▸ Violation of casual delivery? No R 1 X Y R 1 update(R 1) Z R 1 Theoretical Foundations > Causal Delivery 20

Causal Delivery Example ▸ send. Y(update) send. X(update)? No R 1 X Y R 1 update(R 1) Z R 1 Theoretical Foundations > Causal Delivery 21

Quiz Question ▸ Which of the following is true? ▹ The message sent by r 4 will be delivered at q 5 Theoretical Foundations > Causal Delivery 22

Broadcast-based Causal Delivery ▸ Broadcast: ▹ when a process sends a message, it sends the message to every process in the system ▹ Called a multicast when sending to a defined group Theoretical Foundations > Causal Delivery > Broadcast-based 23

BSS Protocol ▸ BSS: ▹ Birman, Schiper & Stephenson ▸ A causal delivery protocol ▹ Uses vector clocks to track messages sent ▸ Broadcast-based ▹ Assumes an external service implements group abstraction ▹ sendp(m) broadcasts message m to the entire group in a single action Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 24

BSS Protocol ▸ Assumes a lossless network ▹ Any message broadcast will eventually be received unless the sender or destination fail ▸ Assumes a failure detection mechanism exists ▹ Will remove failed processes from the group ▹ Will flush broadcasts at the time of a failure Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 25

BSS Protocol Rules ▸ Rule 1 Before sending message m, process i increments Ci[i] and timestamps m. Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 26

BSS Protocol Example ▸ Rule 1: i increments Ci[i] and timestamps m [0 0 0] X Y R 1 create(R 1) [0 0 0] Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 27

BSS Protocol Example ▸ Rule 1: i increments Ci[i] and timestamps m [0 0 0] [0 1 0] X Y R 1 [0 1 0] create(R 1) [0 0 0] [0 1 0] Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 28

BSS Protocol Rules ▸ Rule 2 On reception of message m sent by process i and timestamped tm, process j (≠ i) delays delivery of m until: ∀k: [1…n] tm[k] = Cj[k] + 1; if k = i tm[k] ≤ Cj[k]; if k ≠ i Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 29

BSS Protocol Example ▸ Rule 2: i = 1; j = 0; k = 0; tm[k] ≤ Cj[k]? Yes, 0 ≤ 0 [0 0 0] [0 1 0] X Y R 1 [0 1 0] create(R 1) [0 0 0] [0 1 0] Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 30

BSS Protocol Example ▸ Rule 2: i = 1; j = 0; k = 1; tm[k] = Cj[k] + 1? Yes, 1 = 0+1 [0 0 0] X [0 1 0] Y R 1 [0 1 0] create(R 1) [0 0 0] [0 1 0] Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 31

BSS Protocol Example ▸ Rule 2: i = 1; j = 0; k = 2; tm[k] ≤ Cj[k]? Yes, 0 ≤ 0 [0 0 0] [0 1 0] X Y R 1 [0 1 0] create(R 1) [0 0 0] [0 1 0] Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 32

BSS Protocol Example ▸ Rule 2: Deliver the message [0 0 0] [0 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] [0 1 0] Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 33

BSS Protocol Rules ▸ Rule 3 When a message m is delivered, Cj is updated to max(Cj, tm). Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 34

BSS Protocol Example ▸ Rule 3: Cj is updated to max(Cj, tm) [0 0 0] [0 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] [0 1 0] Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 35

BSS Protocol Example ▸ Rule 3: Cj is updated to max(Cj, tm) [0 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] [0 1 0] Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 36

BSS Protocol Example ▸ Rule 1: i increments Ci[i] and timestamps m [0 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] R 1 update(R 1) [0 1 0] Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 37

BSS Protocol Example ▸ Rule 1: i increments Ci[i] and timestamps m [1 1 0] [0 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] R 1 [1 1 0] update(R 1) [0 1 0] Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 38

BSS Protocol Example ▸ Rule 2: i = 0; j = 1; k = 0; tm[k] = Cj[k] + 1? Yes, 1 = 0+1 R R [1 1 0] X 1 [0 1 0] Y R 1 [0 1 0] create(R 1) [0 0 0] 1 [1 1 0] update(R 1) [0 1 0] Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 39

BSS Protocol Example ▸ Rule 2: i = 0; j = 1; k = 1; tm[k] ≤ Cj[k]? Yes, 1 ≤ 1 [1 1 0] [0 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] R 1 [1 1 0] update(R 1) [0 1 0] Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 40

BSS Protocol Example ▸ Rule 2: i = 0; j = 1; k = 2; tm[k] ≤ Cj[k]? Yes, 0 ≤ 0 [1 1 0] [0 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] R 1 [1 1 0] update(R 1) [0 1 0] Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 41

BSS Protocol Example ▸ Rule 2: Deliver the message [1 1 0] [0 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] R 1 [1 1 0] update(R 1) [0 1 0] R 1 Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 42

BSS Protocol Example ▸ Rule 3: Cj is updated to max(Cj, tm) [1 1 0] [0 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] R 1 [1 1 0] update(R 1) [0 1 0] R 1 Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 43

BSS Protocol Example ▸ Rule 3: Cj is updated to max(Cj, tm) [1 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] R 1 [1 1 0] update(R 1) [0 1 0] R 1 Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 44

BSS Protocol Example ▸ Rule 2: i = 0; j = 2; k = 0; tm[k] = Cj[k] + 1? Yes, 1 = 0+1 R R [1 1 0] X 1 [1 1 0] Y R 1 [0 1 0] create(R 1) [0 0 0] 1 [1 1 0] update(R 1) [0 1 0] R 1 Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 45

BSS Protocol Example ▸ Rule 2: i = 0; j = 2; k = 1; tm[k] ≤ Cj[k]? No, 1 ≤/ 0 [1 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] R 1 [1 1 0] update(R 1) [0 1 0] R 1 Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 46

BSS Protocol Example ▸ Rule 2: Delay delivering the message [1 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] R 1 [1 1 0] update(R 1) [0 1 0] R 1 Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 47

BSS Protocol Example ▸ Rule 2: i = 1; j = 2; k = 0; tm[k] ≤ Cj[k]? Yes, 0 ≤ 0 [1 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] R 1 [1 1 0] update(R 1) [0 1 0] R 1 Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 48

BSS Protocol Example ▸ Rule 2: i = 1; j = 2; k = 1; tm[k] = Cj[k] + 1? Yes, 1 = 0+1 R R [1 1 0] X 1 [1 1 0] Y R 1 [0 1 0] create(R 1) [0 0 0] 1 [1 1 0] update(R 1) [0 1 0] R 1 Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 49

BSS Protocol Example ▸ Rule 2: i = 1; j = 2; k = 2; tm[k] ≤ Cj[k]? Yes, 0 ≤ 0 [1 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] R 1 [1 1 0] update(R 1) [0 1 0] R 1 Z Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 50

BSS Protocol Example ▸ Rule 2: Deliver the message [1 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] R 1 [1 1 0] update(R 1) [0 1 0] Z R 1 Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 51

BSS Protocol Example ▸ Rule 3: Cj is updated to max(Cj, tm) [1 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 0 0] R 1 [1 1 0] update(R 1) [0 1 0] Z R 1 Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 52

BSS Protocol Example ▸ Rule 3: Cj is updated to max(Cj, tm) [1 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 1 0] R 1 [1 1 0] update(R 1) [0 1 0] Z R 1 Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 53

BSS Protocol Example ▸ Rule 2: i = 0; j = 2; k = 0; tm[k] = Cj[k] + 1? Yes, 1 = 0+1 R R [1 1 0] X 1 [1 1 0] Y R 1 [0 1 0] create(R 1) [0 1 0] 1 [1 1 0] update(R 1) [0 1 0] Z R 1 Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 54

BSS Protocol Example ▸ Rule 2: i = 0; j = 2; k = 1; tm[k] ≤ Cj[k]? Yes, 1 ≤ 1 [1 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 1 0] R 1 [1 1 0] update(R 1) [0 1 0] Z R 1 Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 55

BSS Protocol Example ▸ Rule 2: i = 0; j = 2; k = 2; tm[k] ≤ Cj[k]? Yes, 0 ≤ 0 [1 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 1 0] R 1 [1 1 0] update(R 1) [0 1 0] Z R 1 Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 56

BSS Protocol Example ▸ Rule 2: Deliver the message [1 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 1 0] R 1 [1 1 0] update(R 1) [0 1 0] Z R 1 R 1 Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 57

BSS Protocol Example ▸ Rule 3: Cj is updated to max(Cj, tm) [1 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [0 1 0] R 1 [1 1 0] update(R 1) [0 1 0] Z R 1 R 1 Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 58

BSS Protocol Example ▸ Rule 3: Cj is updated to max(Cj, tm) [1 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [1 1 0] R 1 [1 1 0] update(R 1) [0 1 0] Z R 1 R 1 Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 59

BSS Protocol Example 2 ▸ Rule 1: i increments Ci[i] and timestamps m [0 0 0] P [0 0 0] Q [0 0 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 60

BSS Protocol Example 2 ▸ Rule 1: i increments Ci[i] and timestamps m [1 0 0] P [1 0 0] [0 0 0] Q [0 0 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 61

BSS Protocol Example 2 ▸ Rule 2: i = 0; j = 1; k = 0; tm[k] = Cj[k] + 1? Yes, 1 = 0+1 [1 0 0] P [1 0 0] [0 0 0] Q [0 0 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 62

BSS Protocol Example 2 ▸ Rule 2: i = 0; j = 1; k = 1; tm[k] ≤ Cj[k]? Yes, 0 ≤ 0 [1 0 0] P [1 0 0] [0 0 0] Q [0 0 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 63

BSS Protocol Example 2 ▸ Rule 2: i = 0; j = 1; k = 2; tm[k] ≤ Cj[k]? Yes, 0 ≤ 0 [1 0 0] P [1 0 0] [0 0 0] Q [0 0 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 64

BSS Protocol Example 2 ▸ Rule 2: Deliver the message [1 0 0] P [1 0 0] [0 0 0] Q [0 0 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 65

BSS Protocol Example 2 ▸ Rule 3: Cj is updated to max(Cj, tm) [1 0 0] P [1 0 0] [0 0 0] Q [0 0 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 66

BSS Protocol Example 2 ▸ Rule 3: Cj is updated to max(Cj, tm) [1 0 0] P [1 0 0] Q [0 0 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 67

BSS Protocol Example 2 ▸ Rule 1: i increments Ci[i] and timestamps m [1 0 0] P [1 0 0] Q [0 0 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 68

BSS Protocol Example 2 ▸ Rule 1: i increments Ci[i] and timestamps m [2 0 0] P [1 0 0] Q [0 0 0] R [2 0 0] Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 69

BSS Protocol Example 2 ▸ Rule 2: i = 0; j = 2; k = 0; tm[k] = Cj[k] + 1? No, 2 ≠ 0+1 [2 0 0] P [1 0 0] Q [0 0 0] R [2 0 0] Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 70

BSS Protocol Example 2 ▸ Rule 2: Delay delivering the message [2 0 0] P [1 0 0] Q [0 0 0] R [2 0 0] Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 71

BSS Protocol Example 2 ▸ Rule 1: i increments Ci[i] and timestamps m [2 0 0] P [1 0 0] Q [0 0 0] R [2 0 0] Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 72

BSS Protocol Example 2 ▸ Rule 1: i increments Ci[i] and timestamps m [2 0 0] P [1 0 0] [1 1 0] [0 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 73

BSS Protocol Example 2 ▸ Rule 2: i = 0; j = 2; k = 0; tm[k] = Cj[k] + 1? Yes, 1 = 0+1 [2 0 0] P [1 0 0] [1 1 0] [0 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 74

BSS Protocol Example 2 ▸ Rule 2: i = 0; j = 2; k = 1; tm[k] ≤ Cj[k]? Yes, 0 ≤ 0 [2 0 0] P [1 0 0] [1 1 0] [0 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 75

BSS Protocol Example 2 ▸ Rule 2: i = 0; j = 2; k = 2; tm[k] ≤ Cj[k]? Yes, 0 ≤ 0 [2 0 0] P [1 0 0] [1 1 0] [0 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 76

BSS Protocol Example 2 ▸ Rule 2: Deliver the message [2 0 0] P [1 0 0] [1 1 0] [0 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 77

BSS Protocol Example 2 ▸ Rule 3: Cj is updated to max(Cj, tm) [2 0 0] P [1 0 0] [1 1 0] [0 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 78

BSS Protocol Example 2 ▸ Rule 3: Cj is updated to max(Cj, tm) [2 0 0] P [1 0 0] [1 1 0] [1 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 79

BSS Protocol Example 2 ▸ Rule 2: i = 0; j = 2; k = 0; tm[k] = Cj[k] + 1? Yes, 2 = 1+1 [2 0 0] P [1 0 0] [1 1 0] [1 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 80

BSS Protocol Example 2 ▸ Rule 2: i = 0; j = 2; k = 1; tm[k] ≤ Cj[k]? Yes, 0 ≤ 0 [2 0 0] P [1 0 0] [1 1 0] [1 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 81

BSS Protocol Example 2 ▸ Rule 2: i = 0; j = 2; k = 2; tm[k] ≤ Cj[k]? Yes, 0 ≤ 0 [2 0 0] P [1 0 0] [1 1 0] [1 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 82

BSS Protocol Example 2 ▸ Rule 2: Deliver the message [2 0 0] P [1 0 0] [1 1 0] [1 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 83

BSS Protocol Example 2 ▸ Rule 3: Cj is updated to max(Cj, tm) [2 0 0] P [1 0 0] [1 1 0] [1 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 84

BSS Protocol Example 2 ▸ Rule 3: Cj is updated to max(Cj, tm) [2 0 0] P [1 0 0] [1 1 0] [2 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 85

BSS Protocol Example 2 ▸ Rule 2: i = 0; j = 1; k = 0; tm[k] = Cj[k] + 1? Yes, 2 = 1+1 [2 0 0] P [1 0 0] [1 1 0] [2 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 86

BSS Protocol Example 2 ▸ Rule 2: i = 0; j = 1; k = 1; tm[k] ≤ Cj[k]? Yes, 0 ≤ 1 [2 0 0] P [1 0 0] [1 1 0] [2 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 87

BSS Protocol Example 2 ▸ Rule 2: i = 0; j = 1; k = 2; tm[k] ≤ Cj[k]? Yes, 0 ≤ 0 [2 0 0] P [1 0 0] [1 1 0] [2 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 88

BSS Protocol Example 2 ▸ Rule 2: Deliver the message [2 0 0] P [1 0 0] [1 1 0] [2 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 89

BSS Protocol Example 2 ▸ Rule 3: Cj is updated to max(Cj, tm) [2 0 0] P [1 0 0] [1 1 0] [2 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 90

BSS Protocol Example 2 ▸ Rule 3: Cj is updated to max(Cj, tm) [2 0 0] P [1 0 0] [2 1 0] [2 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 91

BSS Protocol Example 2 [2 0 0] P [1 0 0] [2 1 0] [2 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 92

BSS Protocol Example 2 ▸ Rule 2: Deliver the message [2 0 0] P [1 0 0] [2 1 0] [2 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 93

BSS Protocol Example 2 ▸ Rule 3: Cj is updated to max(Cj, tm) [2 1 0] P [1 0 0] [2 1 0] [2 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 94

BSS Protocol Example 2 ▸ Rule 2: Deliver the message [2 1 0] P [1 0 0] [2 1 0] [2 0 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 95

BSS Protocol Example 2 ▸ Rule 3: Cj is updated to max(Cj, tm) [2 1 0] P [1 0 0] [2 1 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 96

Proof of BSS Safety ▸ Safety: ▹ messages are always delivered in causal order ▹ Causal delivery is never violated ▸ Must prove ▹ If send(m 1) send(m 2) then deliveri(m 1) deliveri(m 2) ▹ Two cases: 1. Same process: sendp(m 1) sendp(m 2) 2. Different processes: sendp(m 1) sendq(m 2) Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 97

Proof of BSS Safety: Case 1 ▸ If sendp(m 1) sendp(m 2) then deliveri(m 1) deliveri(m 2) [2 1 0] P [1 0 0] [2 1 0] Q [2 0 0] [1 1 0] R Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 98

Proof of BSS Safety: Case 2 ▸ If sendp(m 1) sendq(m 2) then deliveri(m 1) deliveri(m 2) [1 1 0] R 1 X Y R 1 [0 1 0] create(R 1) [1 1 0] R 1 [1 1 0] update(R 1) [0 1 0] Z R 1 R 1 Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 99

Quiz Question ▸ Assume sent. Q(m) sent. Q(n), process P ∈ dests(m), and P ∈ dests(n). If deliver. P(n) deliver. P(m), then safety is violated. ▹ True Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 100

Proof of BSS Liveness ▸ Liveness: ▹ a message will never be indefinitely delayed ▹ Every message will be delivered eventually ▸ Must prove ▹ If sendi(m) and receivej(m) then deliverj(m) ▹ Two counterexamples: 1. If k = i then tm[k] will never equal Cj[k] + 1 2. If k ≠ i then tm[k] will always be greater than Cj[k] Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 101

Proof of BSS Liveness: Counterexample 1 ▸ If k = i then tm[k] will never equal Cj[k] + 1 ▸ tm[k] is not less than Cj[k] + 1 ▹ Process i sent message m ▹ Hence, Cj[i] < Ci[i] when m was timestamped ▸ If tm[k] is more than Cj[k] + 1 ▹ Process i sent other messages prior to m ▸ BSS assumes a lossless network ▹ Process j will eventually receive the other messages from i ▹ Hence, tm[k] will eventually equal Cj[k] + 1 ▸ Proves if sendi(m) and receivej(m) then deliverj(m) Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 102

Proof of BSS Liveness: Counterexample 2 ▸ If k ≠ i then tm[k] will always be greater than Cj[k] ▸ tm[k] is greater than Cj[k] ▹ Before sending message m to process j, process i received messages from process k that process j has not received ▸ BSS is broadcast-based ▹ Process k also sent these messages to process j ▸ BSS assumes a lossless network ▹ Process j will eventually receive the messages from k ▹ Hence, tm[k] will eventually be less than or equal to Cj[k] ▸ Proves if sendi(m) and receivej(m) then deliverj(m) Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 103

Quiz Question ▸ Assume sent. Q(m) sent. Q(n), process P ∈ dests(m), and P ∈ dests(n). If deliver. P(m) never occurs, then liveness is violated. ▹ True Theoretical Foundations > Causal Delivery > Broadcast-based > BSS Protocol 104

Upcoming Classes ▸ Next ▹ Unicast-based Causal Ordering ▹ Global States 105