Totally Ordered Broadcast Algorithms for Distributed System Semester

Totally Ordered Broadcast Algorithms for Distributed System Semester Project (Nam Thoai)

Totally Ordered Broadcast • Broadcast Service Qualities: – ordering : single-source FIFO, totally ordered, causally ordered – reliability • Implementing a broadcast service • Algorithm for Totally Ordered Broadcast • Modified Algorithm for Totally Ordered Broadcast

Broadcast • Point-to-Point links: provides One-to-One communication – one processor sends a message on an incident link to a single other processor M • Broadcast(multicast): provides one-to-all (one-to-many) communication – a processor sends a message to all processors M • Broadcast (multicast) can be simulated by sending a number of pointto-point messages

The Basic Service Specification (broadcast) • bc-sendi(m, qos): An input event of processor pi, which sends a message m to all processors, containing an indication of the sender; qos is a parameter describing the quality of service required. • bc-recvi(m, j, qos): An output event in which processor pi receives the message m previously broadcast by pj; qos, as above, describes the quality of service provided. • Basic broadcast service: each bc-recvi(m, j, qos) event is mapped to an earlier bc-sendi(m, qos) event, every message received was previously sent (Integrity), and every message that is sent is received once (Liveness) and only once (No Duplicates) at every processor.

Broadcast Service Qualities Broadcast properties can be organized along two axes: • Ordering: Do processors see all messages in the same order or just see the messages from a single processor in the order they were sent ? Does the order in which messages are received by the broadcast service preserve the happens-before relation ? • Reliability: Do all processors see the same set of messages even if failures occur in the underlying system? Do all processors see all the messages broadcast by a nonfaulty processor?

Ordering • Single-Source FIFO: For all messages m 1 and m 2 and all processors pi and pj, if pi sends m 1 before sends m 2, then m 2 is not received at pj before m 1 is. • Totally Ordered: For all messages m 1 and m 2 and all processors pi and pj, if m 1 is received at pi before m 2 is, then m 2 is not received at pj before m 1 is. • Causally Ordered: For all messages m 1 and m 2 and every processor pi, if m 1 happens before m 2, then m 2 is not received at pi before m 1 is. Note: message m 1 is said to happen before message m 2 if either: - the bc-recv event for m 1 happens before the bc-send event for m 2, or - m 1 and m 2 are sent by the same processor and m 1 is sent before m 2.

Ordering • Causally Ordered implies Single-Source FIFO a a b a, b b, a b (1) causally ordered (totally ordered) a b, a b a, b (1) single-source FIFO (totally ordered, causally ordered) (2) totally ordered (causally ordered, single-source ordered)

Reliability In the presence of faulty processors, the Liveness property needs to be weakened. There at most f faulty processors and the mapping from bc-recv(m) events to bc-send(m) events. • Integrity: For each processor pi, 0 i n-1, the restriction of to bcrecvi events is well-defined. That is, every message received was previously sent - no message is received ‘out of thin air’. • No Duplicates: For each processor pi, 0 i n-1, the restriction of to bc-recvi events is one-to-one. That is, no message is received more than once at any single processor.

Reliability • Nonfaulty Liveness: When restricted to bc-send and bc-recv events at nonfaulty processors, is onto. That is, all messages broadcast by a nonfaulty processor are eventually received by all nonfaulty processors. • Faulty Liveness: If one nonfaulty processor has a bc-recv event that maps to a particular bc-send event of a faulty processor, then every nonfaulty processor has a bc-recv event that maps to the same bc-send event. That is, every messages sent by a faulty processor is either received by all nonfaulty processors or by none of them. Pn-1 m 1 m 1 m 1 . . . P 0 P 1 m 1. . . Pn-1 -f P 0 P 1 Pn-1 -f Atomic broadcast is a reliable broadcast with total ordering; atomic broadcast is also called total broadcast. FIFO atomic broadcast is an atomic broadcast with single-source FIFO ordering. Causal atomic broadcast is an atomic broadcast with preserves causality.

Implementing a Broadcast Service • Basic Broadcast Service Using the underlying point-to-point message system to send a copy of m to all processors. • Single-Source FIFO Ordering N N = N+1; bc-send(data, N) message(data, N) Message(data, n) is received if all message(i) (0 i < n) are received

Totally Ordered Broadcast • An Asymmetric Algorithm - Processor pi sends m using the basic broadcast service to a unique central site at processor pc. - Processor pc assigns a sequence number to each message and then sends it to all processors using the basic broadcast service. - Message(k) is received if all messages(i) (0 i < k) are received a(1) b(2) c(3) c a b b(2) a(1) c(3) a(1) b(2) c(3) b(2) a(1) b(2) c(3)

Totally Ordered Broadcast • A Symmetric Algorithm - Based on assigning timestamps to messages. - Assume that the underlying communication system provides singlesource FIFO broadcast. - Each processor maintains an increasing counter (timestamp), which is tagged on messages. - Each processor also maintains a vector with estimates of the timestamps of all other processes. Processor pi updates its entry for pj using the tags on messages received from pj and using special ‘timestamp update’ message sent by pj.

Totally Ordered Broadcast (Code for pi, 0 i n-1) Initially ts[j]=0, 0 j n-1, and pending is empty 1: when bc-sendi(m, to) occurs: // quality of service to means totally ordered 2: ts[i] : = ts[i]+1 3: add m, ts[i], i to pending 4: enable bc-sendi( m, ts[i] , ssf) // quality of service to means single-source FIFO 5: when bc-recvi( m, T , j, ssf), j i, occurs: // j indicates sender; ignore messages from self 6: ts[j] : = T 7: add m, T, j to pending 8: if T > ts[i] then 9: ts[i] : = T 10: enable bc-sendi( ts-up, T , ssf) // bcast timestamp update message 11: when bc-recvi( ts-up, T , j, ssf), j i, occurs: // ignore message from self 12: ts[j] : = T 13: enable bc-recvi(m, j, to) when 14: m, T, j is the entry in pending with the smallest (T, j) 15: T ts[k] for all k 16: result: remove m, T, j from pending

Totally Ordered Broadcast (implementation) (Code for pi, 0 i n-1) Initially ts[j]=0, tn[j]=0 (0 j n-1), N=0, and pending is empty 1: when bc-sendi(m, to) occurs: // quality of service to means totally ordered 2: ts[i] : = ts[i]+1 N = N+1 3: add m, ts[i], N, i to pending 4: enable bc-sendi( m, ts[i], N , basic) // quality of service to means basic 5: when bc-recvi( m, T, N , j, basic), j i, occurs: // j indicates sender; ignore messages from self 6: ts[j] : = T 7: add m, T, N, j to pending 8: if T > ts[i] then 9: ts[i] : = T 10: enable bc-sendi( ts-up, T , basic) // bcast timestamp update message 11: when bc-recvi( ts-up, T , j, basic), j i, occurs: // ignore message from self 12: ts[j] : = T 13: enable bc-recvi(m, j, to) when 14: m, T, N, j is the entry in pending with the smallest (T, j) 15: (T ts[k] for all k) and (N = tn[j]+1) 16: result: remove m, T, j from pending, and tn[j] : = tn[j] + 1