Chapter 6 Consistency and Replication 6 1 Introduction

Chapter 6 Consistency and Replication 6. 1 Introduction: replication and scalability 6. 2 Data-Centric Consistency Models 6. 3 Client-Centric Consistency Models 6. 4 Distribution Protocols: replica placement, update propagation 6. 5 Consistency Protocols

6. 1 Introduction • Reasons for replication – To increase the reliability of a system – When the distributed system needs to scale in numbers and geographical area, replication is important for performance • Reasons to against replication – Consistency problem • When and how the modifications need to be carried out determines the price of replication 2

6. 1. 2 Object Replication Organization of a distributed remote object shared by two different clients. Need to solve the problem of how to protect the object against simultaneous access by multiple clients first. 3

Two solutions for handling multiple accesses a) b) A remote object capable of handling concurrent invocations on its own. Use mechanisms like synchronized in Java. A remote object for which an object adapter is required to handle 4 concurrent invocations

Methods for Replicating a Shared Object (a) A distributed system for replication-aware distributed objects. Advantage: it is possible to adopt object-specific replication strategies, which is often needed inn solving scalability problems. (b) A distributed system responsible for replica management. The distributed system ensures that concurrent invocations are passed to the various replicas in the correct order. Advantage: 5 simple for application developers.

6. 1. 3 Replication as Scaling Technique • Keeping copies up to date may require more network bandwidth. If the access-to-update ratio is very low, no need to install a local replica. • Keeping multiple copies consistent itself is subject to serious scalability problems. – Implementing atomicity with a large number of replicas across a large-scale network is inherently difficult when operations are also required to complete quickly. Difficulties come from the need to synchronize all replicas. – In many cases, the real solution is to loosen the consistency constraints. • To what extent consistency can be loosened depends on the access and update patterns of the replicated data, as well as on the purpose for which those data are used. 6

6. 2 Data-Centric Consistency Models read or write The general organization of a logical data store, physically distributed and replicated across multiple processes. A consistency model is a contract between processes and the data store. It says that if processes agree to obey certain rules, the store promises to work correctly. Normally, the process expects a read operation to return the result of the last write operation on the same 7 data.

Strict Consistency Definition: Any read on a data item returns a value corresponding to the result of the most recent write on it. Behavior of two processes, operating on the same data item. • A strictly consistent store. • A store that is not strictly consistent. Problem: 1. Rely on absolute global time. 2. Multiple operations may take place within an interval. 8

Linearizability and Sequential Consistency (1) Sequential consistency: The result of any execution is the same as if the operations by all processes on the data store were executed in some sequential order and the operations of each individual process appear in this sequence in the order specified by its program. a) b) A sequentially consistent data store. A data store that is not sequentially consistent. 9

Linearizability and Sequential Consistency (2) Linearizability: conditions for sequential consistency, plus if , then OP 1(x) should precede OP 2(y) in this sequence. It is used to assist formal verification of concurrent algorithms. Process P 1 Process P 2 Process P 3 x = 1; print ( y, z); y = 1; print (x, z); z = 1; print (x, y); Three concurrently executing processes, where x, y, and z are initially 0. 10

Linearizability and Sequential Consistency (3) x = 1; print ((y, z); y = 1; print (x, z); z = 1; print (x, y); output of P 1, P 2, P 3 x = 1; y = 1; print (x, z); print(y, z); z = 1; print (x, y); y = 1; z = 1; print (x, y); print (x, z); x = 1; print (y, z); y = 1; x = 1; z = 1; print (x, z); print (y, z); print (x, y); Prints: 001011 Prints: 101011 Prints: 010111 Prints: 111111 Signature: 001011 (a) Signature: 110101 (c) Signature: 111111 (d) Signature: 101011 (b) Four valid execution sequences for the processes of the previous slide. The vertical axis is time. 11

Casual Consistency (1) Definition: Writes that are potentially casually related must be seen by all processes in the same order. Concurrent writes may be seen in a different order on different machines. 12

Casual Consistency (2) concurrent This sequence is allowed with a casually-consistent store, but not with sequentially or strictly consistent store. 13

Casual Consistency (3) a) b) A violation of a casually-consistent store. A correct sequence of events in a casually-consistent store. 14

FIFO Consistency (1) Definition: Writes done by a single process are seen by all other processes in the order in which they were issued, but writes from different processes may be seen in a different order by different processes. All writes from different processes are concurrent. 15

FIFO Consistency (2) A valid sequence of events of FIFO consistency, but not causally consistent. 16

FIFO Consistency (3) x = 1; print (y, z); y = 1; print(x, z); z = 1; print (x, y); x = 1; y = 1; print(x, z); print ( y, z); z = 1; print (x, y); y = 1; print (x, z); z = 1; print (x, y); x = 1; print (y, z); Prints: 00 Prints: 10 Prints: 01 (a) (b) (c) Statement execution as seen by the three processes from Fig 6 -7, where (a) is P 1’s view, (b) is P 2’s view, and (c) is P 3’s view. The statements in bold are the ones that generate the output shown. The signature 001001 is impossible with sequential consistency. 17

FIFO Consistency (4) Process P 1 Process P 2 x = 1; if (y == 0) kill (P 2); y = 1; if (x == 0) kill (P 1); Two concurrent processes. Could both P 1 and P 2 be killed? (1) With sequential consistency? (2) With FIFO consistency? 18

Weak Consistency • In FIFO, writes originating in a single process are requires to be seen everywhere in order. However, not all applications require seeing all writes. • Weak consistency only places restrictions on synchronization variables. 19

Weak Consistency (1) Properties: 1. Accesses to synchronization variables associated with a data store are sequentially consistent 2. No operation on a synchronization variable is allowed to be performed until all previous writes have been completed everywhere 3. No read or write operation on data items are allowed to be performed until all previous operations to synchronization variables have been performed. Limit the time when consistency holds, not the form of the consistency. 20

Weak Consistency (2) int a, b, c, d, e, x, y; int *p, *q; int f( int *p, int *q); /* variables */ /* pointers */ /* function prototype */ a = x * x; b = y * y; c = a*a*a + b*b + a * b; d = a * c; p = &a; q = &b e = f(p, q) /* a stored in register */ /* b as well */ /* used later */ /* p gets address of a */ /* q gets address of b */ /* function call */ A program fragment in which some variables may be kept in registers, not updating their memory locations. A second process might get inconsistent values of a, b, c. 21

Weak Consistency (3) a) A valid sequence of events for weak consistency. b) An invalid sequence for weak consistency. 22

Release Consistency (1) In weak consistency, when a synchronization variable is accessed, the data store does not know it is after write or before read. To provide this information, two kinds of synchronization variables are needed. In release consistency, acquire and release operations are provided for grouping operations to prevent interleaving. A valid event sequence for release consistency. 23

Release Consistency (2) Rules: – Before a read or write operation on shared data is performed, all previous acquires done by the process must have completed successfully. – Before a release is allowed to be performed, all previous reads and writes by the process must have completed – Accesses to synchronization variables are FIFO consistent (sequential consistency is not required). Variation: lazy release consistency: at the time of release, nothing is sent. 24

Entry Consistency (1) Each synchronization variable has a current owner, the process that last acquired it. Conditions: • An acquire access of a synchronization variable is not allowed to perform with respect to a process until all updates to the guarded shared data have been performed with respect to that process. • Before an exclusive mode access to a synchronization variable by a process is allowed to perform with respect to that process, no other process may hold the synchronization variable, not even in nonexclusive mode. • After an exclusive mode access to a synchronization variable has been performed, any other process's next nonexclusive mode access to that synchronization variable may not be performed until it has performed with respect to that variable's 25 owner.

Entry Consistency (2) A valid event sequence for entry consistency. Programming problem with entry consistency: associating data with synchronization variables. One approach is the use of distributed shared object. 26

Summary of Consistency Models Consistency Description Strict Absolute time ordering of all shared accesses matters. Linearizability All processes must see all shared accesses in the same order. Accesses are furthermore ordered according to a (nonunique) global timestamp Sequential All processes see all shared accesses in the same order. Accesses are not ordered in time Causal All processes see causally-related shared accesses in the same order. FIFO All processes see writes from each other in the order they were used. Writes from different processes may not always be seen in that order (a) Consistency Description Weak Shared data can be counted on to be consistent only after a synchronization is done Release Shared data are made consistent when a critical region is exited Entry Shared data pertaining to a critical region are made consistent when a critical region is entered. (b) a) b) Consistency models not using synchronization operations. Models with synchronization operations. 27

6. 3 Client-Centric Consistency Models • The consistency models in the previous section provide a system-wide consistent view on a data store. • This section consider data stores characterized by the lack of simultaneous updates, or when such updates happen, they could be easily resolved. 28

Eventual Consistency (data-centric) The principle of a mobile user accessing different replicas of a distributed database. All replicas will gradually become consistent. 29

Client-Centric Consistency Models • Client-centric consistency provides guarantees for a single client concerning the consistency of accesses to a data store by that client. • Notations: – – : the version of data x at local copy at time t : the set of a series of write operations at that took place since initialization to obtain – : If operations in have also been performed at at a later time 30

Monotonic Reads Definition: If a process reads the value of x, any successive read operation on x by that process will always return the same or a more recent value. The read operations performed by a single process P at two different local copies of the same data store. a) A monotonic-read consistent data store b) A data store that does not provide monotonic reads. 31

Monotonic Writes Definition: A write operation by a process on x is completed before any successive write operation on x by the same process. The write operations performed by a single process P at two different local copies of the same data store a) A monotonic-write consistent data store. 32 b) A data store that does not provide monotonic-write consistency.

Read Your Writes Definition: The effect of a write operation by a process on x will always be seen by a successive read operation on x by the same process. a) b) A data store that provides read-your-writes consistency. A data store that does not. 33

Writes Follow Reads Definition: A write operation by a process on x following a previous read on x by the same process, is guaranteed to take place on the same or a more recent value of x that was read. a) b) A writes-follow-reads consistent data store A data store that does not provide writes-follow-reads consistency 34