Outline Distributed shared memory continued 3112021 COP 5611

Outline • Distributed shared memory – continued 3/11/2021 COP 5611 - Operating Systems 1

Distributed Shared Memory • Distributed computing is mainly based on the message passing model – Client/server model – Remote procedure calls • Distributed shared memory – is a resource management component that implements the shared memory model in distributed systems, where there is no physically shared memory 3/11/2021 COP 5611 - Operating Systems 2

Memory Hierarchies 3/11/2021 COP 5611 - Operating Systems 3

Virtual memory 3/11/2021 COP 5611 - Operating Systems 4

Page Table Mapping 3/11/2021 COP 5611 - Operating Systems 5

Distributed Shared Memory – cont. • This is a further extension of the virtual memory management on a single computer – When a process accesses data in the shared address space, a mapping manager maps the shared memory address to the physical memory, which can be local or remote 3/11/2021 COP 5611 - Operating Systems 6

Distributed Shared Memory – cont. 3/11/2021 COP 5611 - Operating Systems 7

Distributed Shared Memory – cont. A mapping manager is a layer of software to map addresses in user space into the physical memory addresses 3/11/2021 COP 5611 - Operating Systems 8

Distributed Shared Memory – cont. • With DSM, application programs can access data in the shared space as they access data in traditional virtual memory – Mapping manager can move data among the local main memory, local disk, and another nodes 3/11/2021 COP 5611 - Operating Systems 9

Distributed Shared Memory – cont. • Advantages of DSM – It is easier to design and develop algorithms with DSM than message passing models – DSM allows complex structures to be passed by reference, simplifying the distributed application development – DSM can cut down the overhead of communication by exploiting locality in programs – DSM can overcome some the architectural limitations of shared memory machines 3/11/2021 COP 5611 - Operating Systems 10

Distributed Shared Memory – cont. • Central implementation issues in DSM – How to keep track of the location of remote data – How to overcome the communication delays and high overhead associated with communication protocols – How to improve the system performance 3/11/2021 COP 5611 - Operating Systems 11

The Central-Server Algorithm • A central server maintains all the shared data – It serves the read requests from other nodes or clients by returning the data items to them – It updates the data on write requests by clients 3/11/2021 COP 5611 - Operating Systems 12

The Central-Server Algorithm – cont. 3/11/2021 COP 5611 - Operating Systems 13

The Migration Algorithm • In contrast to the central-server algorithm, data in the migration algorithm is shipped to the location of data access request – Subsequent accesses can then be performed locally • Thrashing can be a problem 3/11/2021 COP 5611 - Operating Systems 14

The Migration Algorithm – cont. 3/11/2021 COP 5611 - Operating Systems 15

The Migration Algorithm – cont. 3/11/2021 COP 5611 - Operating Systems 16

The Read-Replication Algorithm • The read-replication algorithm allows multiple node to have read access or one node to have read-write access 3/11/2021 COP 5611 - Operating Systems 17

The Read-Replication Algorithm 3/11/2021 COP 5611 - Operating Systems – cont. 18

The Read-Replication Algorithm 3/11/2021 COP 5611 - Operating Systems – cont. 19

The Full-Replication Algorithm • This is a further extension of the readreplication algorithm – It allows multiple nodes to have both read and write access to shared data blocks • Data consistency issue 3/11/2021 COP 5611 - Operating Systems 20

The Full-Replication Algorithm 3/11/2021 COP 5611 - Operating Systems – cont. 21

Memory Coherence • Memory consistency models – Many consistency models have been proposed – These models differ in • • 3/11/2021 Restrictiveness Implementation complexity Ease of programming Performance COP 5611 - Operating Systems 22

Memory Coherence – cont. • Strict consistency – Any read on a data item x returns a value corresponding to the result of the most recent write on x • Sequential consistency – The result of any execution is the same as if the operations by all processes on the data store were executed in some sequential order and the operations of each individual process appear in this sequence in the order specified by its program 3/11/2021 COP 5611 - Operating Systems 23

Memory Coherence – cont. 3/11/2021 COP 5611 - Operating Systems 24

Memory Coherence – cont. Process P 1 Process P 2 Process P 3 x = 1; print ( y, z); y = 1; print (x, z); z = 1; print (x, y); 3/11/2021 COP 5611 - Operating Systems 25

Memory Coherence – cont. x = 1; print ((y, z); y = 1; print (x, z); z = 1; print (x, y); x = 1; y = 1; print (x, z); print(y, z); z = 1; print (x, y); y = 1; z = 1; print (x, y); print (x, z); x = 1; print (y, z); y = 1; x = 1; z = 1; print (x, z); print (y, z); print (x, y); Prints: 001011 Prints: 101011 Prints: 010111 Prints: 111111 (a) 3/11/2021 (b) (c) COP 5611 - Operating Systems (d) 26

Memory Coherence – cont. • General consistency – All the copies of a memory location eventually contain the same data when all the writes issued by every process have completed • Causal consistency – Writes that are causally related must be seen by all the processes in the same order; concurrent writes may be seen in a different order on different machines – Processor consistency is related to causal consistency but with one more relaxation 3/11/2021 COP 5611 - Operating Systems 27

Memory Coherence – cont. • Weak consistency – Accesses to synchronization variables are sequentially consistent – Before a synchronization access, all previous regular data accesses must be completed – Before a regular data access, all previous synchronization accesses must be completed 3/11/2021 COP 5611 - Operating Systems 28

Memory Coherence – cont. 3/11/2021 COP 5611 - Operating Systems 29

Memory Coherence – cont. • Release consistency – Synchronization operations are broken down into acquire and release operations – Before a read or write operation on shared data is performed, all previous acquires done by the process must have completed successfully – Before a release is allowed, all previous reads and write done by the process must have been completed 3/11/2021 COP 5611 - Operating Systems 30

Memory Coherence – cont. • Coherence protocols – A protocol to keep replicas coherent • Write-invalidate protocol – A write to a shared data causes the invalidation of all copies except the one where the write occurs • Write-update protocol – A write to a shared data causes all copies of that data to be updated 3/11/2021 COP 5611 - Operating Systems 31

Memory Coherence – cont. 3/11/2021 COP 5611 - Operating Systems 32

Memory Coherence – cont. • Type-specific memory coherence – Exploiting application specific semantics information – The system uses different kinds of coherence mechanisms for different kinds of classes • Write-once objects • Write-many objects • Read-mostly objects 3/11/2021 COP 5611 - Operating Systems 33

Design Issues • Granularity – The size of the shared memory unit – Advantages of large sizes • A page size that is a multiple of the size provided by the underlying memory management system allows for the integration of DSM and the memory management system • Better utilization of locality of reference – Disadvantages • The greater the chance for contention • False sharing 3/11/2021 COP 5611 - Operating Systems 34

False Sharing 3/11/2021 COP 5611 - Operating Systems 35

Design Issues – cont. • Page replacement – Traditional methods such as LRU can not be used directly – Page access modes must be taken into consideration • For example, private pages may be replaced before shared pages 3/11/2021 COP 5611 - Operating Systems 36

Case Studies • IVY – Integrated Shared Virtual Memory at Yale – The granularity is a page – The address space is divided into shared virtual memory address space and private space – It supports strict consistency using writeinvalidation protocol 3/11/2021 COP 5611 - Operating Systems 37

IVY – cont. • The coherence protocol – Write fault handling • The owner is identified first • Then the owner sends the page and its copy-set to the requested processor • The faulting processor sends out invalidation messages to all the processors contained in the copy-set – Read fault handing • The owner is identified first • The owner sends the page and adds the requested site in the copy-set of the page 3/11/2021 COP 5611 - Operating Systems 38

IVY – cont. • Coherence protocol – The centralized manager scheme – Fixed distributed manager scheme – Dynamic distributed manager scheme 3/11/2021 COP 5611 - Operating Systems 39

IVY – cont. 3/11/2021 COP 5611 - Operating Systems 40

IVY – cont. • Double fault – A double fault occurs if a page not available locally is read and written successively • The page is transferred first due to the read fault • Then the same page is transferred again due to the write fault • Memory allocation • Process synchronization 3/11/2021 COP 5611 - Operating Systems 41

Case Studies – cont. • Mirage – Thrashing control • Clouds – The RA kernel • Segmentation with paging (The size of a segment is a multiple of the physical page size) – Distributed shared memory controller • Four modes: read-only, read-write, weak-read, and none 3/11/2021 COP 5611 - Operating Systems 42

Summary • Distributed shared memory tries to provide an easy-to-use interface for distributed applications – Distributed programs access data in the shared address space as they access data in local memory • However, the performance is a critical issue – Replication is used to reduce the high cost of communications • Memory coherence is however difficult and expensive to achieve 3/11/2021 COP 5611 - Operating Systems 43