NET 422 Nouf Al Batati Course outline Characterization

NET 422 Nouf Al Batati

Course outline � Characterization of distributed systems � Introduction � Examples � Resource sharing and the web � Challenges � Summary � system models � Introduction � Architectural models � Fundamentals models � Summery � Interprocess communication � External data representation and marshalling � Clint-server communication � Group communication � Case study: interprocess communication in UNIX � summery

� Distributed objects and remote invocation � introduction � Remote procedure call � Events and notifications � Case study : Java RMI � Summery � Distributed file systems � Introduction � File service architecture � Case study: sun network file system � Summery � Name services � Introduction � Name services and the DNS � Case study of the X. 500 Directory Service � summery

� PEER-TO-PEER systems � Introduction � Peer-to-peer middleware � Time and global states � Introduction � Clocks events and process states � Synchronizing physical clocks � Logical time and logical clocks � Summery � Coordination and agreement � Introduction � Distributed mutual exclusion � Elections � Multicast communication � Consensus and related problems � Summery

� Replication � Introduction � System models and group communication � Fault-tolerant services � Summery � Web services � Introduction � Service descriptions and IDL for web services � Case study: the grid � summery

Chapter 1: Characterization of distributed systems �Introduction �Examples �Resource sharing and the web �Challenges �Summary

Introduction �The aim of the course: �Explain the characteristics of networked computers �Present the main concepts and techniques that have been developed.

�What is Distributed Systems? It is one in which hardware or software components located at networked computers communicate and coordinate their actions only by passing messages �Concurrency �No global clock �Independent failure

Advantages of Distributed Computing � Reliability The important advantage of distributed computing system is reliability. It is more reliable than a single system. If one machine from system crashes, the rest of the computers remain unaffected and the system can survive as a whole. � Incremental Growth In distributed computing the computer power can be added in small increments i. e. new machines can be added incrementally as per requirements on processing power grow. � Sharing of Resources Shared data is required to many applications such as banking, reservation system and computer-supported cooperative work. As data or resources are shared in distributed system, it is essential for various applications. � Flexibility As the system is very flexible, it is very easy to install, implement and debug new services. Each service is equally accessible to every client

Advantages of Distributed Computing � Speed A distributed computing system can have more computing power than a mainframe. Its speed makes it different than other systems. � Open system As it is open system, it can communicate with other systems at anytime. Because of an open system it has an advantage over selfcontained system as well as closed system. � Performance It is yet another advantage of distributed computing system. The collection of processors in the system can provide higher performance than a centralized computer. http: //www. platformusers. net/distributedcomputing/distributed-computing-advantages-anddisadvantages. html

Disadvantages of Distributed Computing � Troubleshooting and diagnosing problems are the most important disadvantages of distributed computing system. The analysis may require connecting to remote nodes or checking communication between nodes. � Software Less software support is the main disadvantage of distributed computing system. Because of more software components that comprise a system there is a chance of error occurring. � Networking The underlying network in distributed computing system can cause several problems such as transmission problem, overloading, loss of messages. Hence, the problems created by network infrastructure are the disadvantages of distributed computing. � Security The easy distributed access in distributed computing system http: //www. platformusers. net/distributedwhich increases the risk of security. The sharing of data creates thecomputing/distributed-computing-advantages-andproblem of data security. disadvantages. html

Examples of distributed systems �Internet �Intranets �File services �Firewalls �The cost issue �Mobile and ubiquitous computing

Figure 1. 1 A typical portion of the Internet intranet % ISP % % % backbone satellite link desktop computer: server: network link: Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Figure 1. 2 A typical intranet Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Figure 1. 3 Portable and handheld devices in a distributed system Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Resource sharing and the web �Hardware and software sharing �Service : the part of computer system that manages a collection of related resources and presents their functionality to users and applications �We access shared files through file service �We send document to printer through a printing service. �The service provide a set of operations �Resources in a distributed system are physically encapsulated within computers and can only be accessed from other computers by communication.

Resource sharing and the web �Server : is the running program on a networked computers that accepts requests from programs running on other computers to perform a service and responds appropriately �Remote invocation: is a complete interaction between a client and a server �Client and server refer to processes

Client server example �The world Wide Web (WWW) �The web is open system: Its operation is based on communication standards and document standards that are freely published and widely implemented 2. The web is open with respect to the types of resources that can be published and shared on it. 1.

� The web is based on three main standard technological components: �HTML : Hyper Text Markup language �URL : Uniform Resource Locator �http: //servername[: port][/path. Name][? query][#fragm ent] �http: //www. w 3. org/protocol/activity. htm#intro �http: //www. google. com /search? q=kindberg �HTTP : Hyper Text Transfer Protocol �Request replay �Content types �One resource per request �Simple access control

Figure 1. 4 Web servers and web browsers www. google. com http: //www. google. comlsearch? q=kindberg Browsers Web servers www. cdk 3. net Internet http: //www. cdk 3. net/ www. w 3 c. org File system of www. w 3 c. org Protocols http: //www. w 3 c. org/Protocols/Activity. html Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Challenges �Heterogeneity �Networks �Computer hardware �Operating systems �Programming languages �Implementations by different developers

Challenges ‘Continue’ �Openness �The ability of adding new resource sharing services and making them available for use by variety of client programs. �Open distributed systems are extensible : �Hardware level �Software level

Challenges ‘Continue’ �Security �Denial of service attacks: bombarding the service with such a large number of pointless requests that the serious user are unable to use it. �Security of mobile code: the possible effect of running mobile code ‘ attachment’ are unpredictable.

Challenges ‘Continue’ �Scalability �Controlling the cost of physical resources �Controlling the performance loss �Preventing software resources running out �Avoiding performance bottlenecks

Figure 1. 5 Computers in the Internet Date 1979, Dec. 1989, July 1999, July 2003, Jan. Computers 188 130, 000 56, 218, 000 171, 638, 297 Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Web servers 0 0 5, 560, 866 35, 424, 956

Figure 1. 6 Computers vs. Web servers in the Internet Date 1993, July 1995, July 1997, July 1999, July 2001, July Computers 1, 776, 000 6, 642, 000 19, 540, 000 56, 218, 000 125, 888, 197 Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Web servers Percentage 130 0. 008 23, 500 1, 203, 096 6, 598, 697 31, 299, 592 42, 298, 371 0. 4 6 12 25

Challenges ‘Continue’ �Failure �Detecting failures �Masking failures �By hiding it or make it less severe: 1. Re-transmission 2. Using backups �Tolerating failures �Recovery from failures: ‘rolled back’ �Redundancy �Two different routes between any two routers �Every name table is replicated in at least two different servers , in DNS �Database may be replicated

Challenges ‘Continue’ �Concurrency � There is a possibility that several clients will attempt to a access a shared resources at the same time. � The process that manages a shared resource could take one client request at a time. But that approach limits throughput.

Challenges ‘Continue’ • Transparency Access transparency: enables local and remote resources to be accessed using identical operations. Location transparency: enables resources to be accessed without knowledge of their physical or network location (for example, which building or IP address). Concurrency transparency: enables several processes to operate concurrently using shared resources without interference between them. Replication transparency: enables multiple instances of resources to be used to increase reliability and performance without knowledge of the replicas by users or application programmers.

Challenges: ‘Continue’ Transparency ( continue) Failure transparency: enables the concealment of faults, allowing users and application programs to complete their tasks despite the failure of hardware or software components. Mobility transparency: allows the movement of resources and clients within a system without affecting the operation of users or programs. Performance transparency: allows the system to be reconfigured to improve performance as loads vary. Scaling transparency: allows the system and applications to expand in scale without change to the system structure or the application algorithms. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Small quiz

http: //www. proprofs. com/quiz-school/quizshow. php? title=tsn 30305 -distributedsystems&quesnum=10&show. Next. Q=no �Question 1 In a distributed system, the existence of multiple autonomous computers is transparent to the users or appears to the users as a single computer. �A. True �B. False

http: //www. proprofs. com/quiz-school/quizshow. php? title=tsn 30305 -distributedsystems&quesnum=10&show. Next. Q=no �Question 2 For a distributed system, users must explicitly log on to a machine, explicitly know what the machine can do, explicitly submit data to the correct location, and explicitly tell the machine how to return their results (e. g. , give their own logical addresses to the machine). �A. True �B. False

http: //www. proprofs. com/quiz-school/quizshow. php? title=tsn 30305 -distributedsystems&quesnum=10&show. Next. Q=no �Question 3 There are four requirements in the design of a distributed system. Choose the correct combination from the list below. �A. Network integrity; Quality of Software (Qo. S); Caching and alteration; Dependability issues �B. Network dependency; Quantity of Service (Qo. S); Cookies and replication; Dependability issues. �C. Network performance; Quality of Service (Qo. S); Caching and replication; Dependability issues. �D. Network Accessibility; Quality of hardware (Qo. H); Caching and replication; Dependability issues

http: //www. proprofs. com/quiz-school/quizshow. php? title=tsn 30305 -distributedsystems&quesnum=10&show. Next. Q=no �Question 4 Peer processes model is a distributed application based on peer processes. Sometimes, some processes play similar roles, interacting cooperatively as peers to perform a task. This model is usually used because of the nature of the application, e. g. , group communication. �A. True �B. False

Chapter 2: System models �Introduction �Architectural model �Fundamental model

Introduction In this chapter we bring out the common properties and design issue for DS in the form of descriptive models. Each model is intended to provide an abstract, simplified but consistent description of a relevant aspect of distributed system design

Problems that designers face: �Widely varying modes of use �Wide range of system environment �Internal problems: �Non-synchronized clocks �Conflicting data updates �Many modes of HW and SW failure involving the individual components of a system �External threats: Attacks and denial services

System models Architectural model Fundamental model Layered structure Interaction model Main architectural models Failure model Variant of client-server model Security model features Design requirement

Architectural model Major concerns: to make the system: �Reliable �Manageable �Adaptable �Cost-effective The architectural model of a distributed system simplifies and abstracts the functions of the individual components of the DS and then it considers: �The replacement of the components across a network of computers �The interrelationships between the components

Architectural model System simplification: Classifying processes to identify the responsibilities which helps us to assess their workloads and determine the impact of failures in each process. Processes classes: �Server processes �Client processes �Peer processes

Architectural model Dynamic systems: �Moving code to reduce access delay and minimize communication traffic �Adding new computers and allowing them to discover the services and offer their services to others

Architectural model Software layers: � Software architecture: �Platform ( hardware + OS) �Middleware � Services: � Remote method invocation � Communication between a group of processes � Notification of events ………. . etc � Examples: � CORBA: Common Object Request Broker Architecture � Java RMI : Java Remote Method Invocation � Web services � DCOM: distributed component object model � The ISO/ITU-T : International Organization for Standardization /International telecommunication union � Limitation of middleware

Figure 2. 1 Software and hardware service layers in distributed systems Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Figure 2. 2 Clients invoke individual servers Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Architectural model System architecture 1. Client- server �Web server is a client of a local file server that manages the files in which the web pages are stored �Web servers and most other internet services are clients of the DNS services. �Search engines What are crawlers? Programs run in the background at a search engine site using HTTP request to access web servers throughout the internet v Simple v Scale poorly: Q: The search engine is both a server and a client , explain. Q: A typical search engine would normally include many concurrent threads of execution, what do they do? and what do they need?

Architectural model 2. Peer-to-peer �All process in a task or activity play the same roles �It enables hundreds of thousand computers to provide access to data and other resources �E. g : Napster application for sharing digital music files �A large number of data object are shared �Each computer holds only a small part of the application database and storage �This architecture is more complex �Replication is provided to distribute the load and provide resilience

Figure 2. 3 A distributed application based on peer processes Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Architectural model 3. Variations �The use of multiple servers and caches �The use of mobile code and mobile agents �Users need for low-cost computers with limited hardware resources that are simple to manage �The requirement to add and remove mobile devices in a convenient manner

Figure 2. 4 A service provided by multiple servers Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Architectural model �Services provided by multiple servers �Objects partitioning and distribution by the server �Objects replication : Sun NIS �Cluster : used for highly scalable web services such as search engines and online stores �Proxy servers and caches �Caches: �Collocated with each client �Located in a proxy server �Proxy server to increase availability and performance �Provides a shared cache �Used to access remote web servers through a ……………….

Figure 2. 5 Web proxy server Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Architectural model �Mobile code �Applets are example of mobile code �The user running browser select a link to an applet whose code is stored on a web server then the code is downloaded to the browser and runs there �Mobile agents �A moving program ( code + data) from one computer to another carrying on task on someone's behalf returning with a result

�Network computers: �Network computer downloads its operating system and any application software needed by the user from a remote file server. �Applications are run locally but the files are managed by remote file server. �Thin clients: �It refers to a software layer that supports window- based user interface on a computer that is local to the user while executing application programs on a remote computer. �Instead of downloading the code of applications into the user’s computer, it runs them on a computer server.

Figure 2. 6 Web applets Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Figure 2. 7 Thin clients and compute servers Compute server Network computer or PC Thin Client network Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Application Process

Architectural model � Mobile devices and spontaneous interoperation �Mobile devices are HW computing components that move between physical locations carrying software component with them �Connect to wireless network via: � 3 G and GSM � Wi. Fi (IEEE 802. 11) � Bluetooth �Mobile servers �Three main issues: � Mobility transparency � Variable connectivity � Spontaneous interoperation What is service discovery?

Architectural model �Interfaces and objects �Modula �C++ �Java

Architectural model Design requirement for distributed architectures 1. Performance issues �Responsiveness � Users require fast and consistent response � The speed is affected by the load, performance and delays in all software components involved � To achieve a good response time, systems must be composed of relatively few SW layers and the quantity of transferred data must be small �Throughput � The measure of performance � Affected by processing speed and data transfer rate �Balancing computational loads � Applet � Several computers

Architectural model 2. Quality of service �It is affected by � Reliability � Security � Performance � adaptability 3. Use of caching and replication �Web-caching protocol 4. Dependability issue �Fault tolerance � Redundancy � Retransmission �security

Fundamental model The system model has to address the questions : �What are the main entities in the system? �How do they interact? �What are the characteristics that affect their individual and collective behavior? The purpose of a model: �To make explicit all the relevant assumptions about the system �To make generalization concerning what is possible or impossible, given those assumptions. �General –purpose �Guarantees

Fundamental model 1. Interaction model � Multiple server processes may cooperate with one another to provide a service � A set of beer processes may cooperate with one another to achieve a common goal Distribution algorithm: the steps to be taken by each of the process of which the system is composed � Massages are transmitted to: � Transfer information � Coordinate activities It is difficult to describe the rate because of the failure Interacting processes perform all of the activity in a distributed system Each process has its own private state Interacting process is affected by: � Communication performance � Timing issue

Fundamental model Performance of communication channels �Latency: is the delay between the start of a message’s transmission from one process and its receipt by another and it includes: �Transmission time through the network �Time needed to reach the network which increases with the increase of the network load �The time taken by the operating system communication services �The bandwidth of the computer network �jitter

�Computer clock and timing events Clock drift rate: the relative amount that a computer clock differs from a perfect reference clock Approach to correcting the times on computer clocks: �Radio receivers to get time readings from the GPS

Fundamental model Two variants of the interaction model �Synchronous DS �The time to execute each step of a process has known lower and upper bounds �Each message transmitted over a channel is received within a known bounded time �Each process has a local clock whose drift rate from real time has a known bound �Asynchronous DS �No bounds on process execution �No bounds on message transmission delay �No bounds on clock drift rate

Fundamental model Event ordering �Knowing whether an event at one process occurred before, after or concurrently with another event at another process. �Logical time can be used to provide an ordering among the events at process running in different computers in a distributed system

Figure 2. 8 Real-time ordering of events Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Fundamental model Item From Subject 23 Z Re: Meeting 24 X Meeting 25 Y Re: Meeting

Fundamental model 2. Failure model 1. Omission failures �Process omission failure �Communication omission failure � Caused by lack of buffer space � Network transmission error �Send-omission failure �Receive-omission failure �Channel omission failute 2. Arbitrary failures �The process may set wrong values in its data items �The process my return a wrong value in response to an invocation

Figure 2. 10 Omission and arbitrary failures Class of failure Affects Description Fail-stop Process halts and remains halted. Other processes detect this state. Crash Process halts and remains halted. Other processes not be able to detect this state. Omission Channel A message inserted in an outgoing message buffer n arrives at the other end’s incoming message buffer. Send-omission Process A process completes send, a but the message is not put in its outgoing message buffer. Receive-omission Process A message is put in a process’s incoming message buffer, but that process does not receive it. Arbitrary Process or Process/channel exhibits arbitrary behaviour: it may (Byzantine) channel send/transmit arbitrary messages at arbitrary times, commit omissions; a process may stop or take an incorrect step. Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Figure 2. 9 Processes and channels Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Fundamental model 3. Timing failures Class of Failure Affects Clock Process Performance Channel Description Process’s local clock exceeds the bounds o rate of drift from real time. Process exceeds the bounds on the interva between two steps. A message’s transmission takes longer tha stated bound. Figure 2. 11: Timing failures Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Fundamental model �Masking failures �Multiple servers �Knowledge of the failure characteristics of component � a service masks a failure by: � Hiding it � Converting it into a more acceptable type �Reliability of one-to-one communication �Reliable communication is defined in terms of validity and integrity as follows: �Validity: any message is delivered �Integrity : received message = sent message with no duplication

Fundamental model �The threats com from: �Any protocol does not reject a message that arrives twice �Malicious users

Fundamental model 3. Security model �The security of a DS can be achieved by securing the processes and the channel used and by protecting the objects that they encapsulate against unauthorized access. �Protecting objects �Using access rights �Securing processes and their interactions �The enemy �Threats to processes �Threats to communication channels

Figure 2. 12 Objects and principals Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Fundamental model �Defeating security threats 1. Cryptography and shared secrets � Cryptography is the science of keeping messages secure � Encryption is the process of scrambling a message to hide its contents. 2. Authentication � It is to include in a message an encrypted portion that contains enough of the contents of the message to guarantee its authenticity, for example: o a representation of the requesting principal’s identity o The identity of the file o The date and time of the request …….

3. Secure channels Secure channel has the following properties: • Each process knows the identity of the principal on whose behalf the other process is executing • A secure channel ensures the privacy and integrity of the transmitted data • Each message include a physical or logical time stamp

�Other possible threats from an enemy �Denial of service �Mobile code �The uses of security models

Figure 2. 13 The enemy Copy ofm The enemy Processp m’ m Processq Communication channel Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005

Figure 2. 14 Secure channels Principal. B Principal. A Processp Secure channel Instructor’s Guide for Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edn. 4 © Pearson Education 2005 Processq