Distributed Systems Middleware Prof Nalini Venkatasubramanian Dept of
Distributed Systems Middleware Prof. Nalini Venkatasubramanian Dept. of Information & Computer Science University of California, Irvine 1
CS 237 - Distributed Systems Middleware – Spring 2013 Lecture 1 - Introduction to Distributed Systems Middleware Tuesdays, Thursdays 3: 30 -4: 50 p. m. Prof. Nalini Venkatasubramanian nalini@ics. uci. edu Intro to Distributed Systems Middleware 2
Course logistics and details z Course Web page yhttp: //www. ics. uci. edu/~cs 237 z Lectures – Tu. Th 3: 30 – 4: 50 p. m z Reading List x. Technical papers and reports x. Reference Books Intro to Distributed Systems Middleware 3
Course logistics and details z Homeworks y. Paper summaries y. Survey paper z Course Presentation z Course Project y. Maybe done individually or in groups y. Potential projects will be available on webpage Intro to Distributed Systems Middleware 4
Comp. Sci 237 Grading Policy z Homeworks - 30% • 1 paper summary due every week • (3 randomly selected each worth 10% of the final grade). z Survey Paper - 10% z Class Presentation - 10% z Class Project - 50% of the final grade z Final assignment of grades will be based on a curve. Intro to Distributed Systems Middleware 5
Lecture Schedule y. Weeks 1, 2: Distributed Computing Fundamentals • • Middleware Concepts Distributed Operating Systems Messaging, Communication in Distributed Systems Naming , Directory Services, Distributed File. Systems y. Weeks 3, 4, 5, 6, 7: Middleware Frameworks x. Distributed/Cluster Computing Frameworks – DCE, Hadoop x. Object-based Middleware –CORBA, COM, DCOM x. Java Based Technologies – Java RMI, JINI, J 2 EE, EJB x. Messaging Technologies • XML Based Middleware, Publish/Subscribe Technologies x. Service Oriented Architectures • . NET, Web Services, SOAP, REST, Service Gateways x. Database access and integration middleware (ODBC, JDBC, mediators) x. Cloud Computing Platforms and Technologies • Amazon EC 2, Amazon S 3, Microsoft Azure, Google App Engine • VMWare, Hypervisor and Virtualization technologies Intro to Distributed Systems Middleware 6
Course Schedule y. Weeks 8, 9 and 10: Middleware for Distributed Application Environments x. Real-time and Qo. S-enabled middleware x. Middleware for Mobile/Wireless networks and applications x. Middleware for Sensor Networks and Pervasive Environments x. Middleware for P 2 P architectures x. Middleware for Resilient/Fault tolerant applications x. Middleware for Secure applications Intro to Distributed Systems Middleware 7
What is Middleware? z Middleware is the software between the application programs and the operating System/base networking z Integration Fabric that knits together applications, devices, systems software, data z Middleware provides a comprehensive set of higher-level distributed computing capabilities and a set of interfaces to access the capabilities of the system. Intro to Distributed Systems Middleware 8
The Evergrowing Alphabet Soup Distributed Computing Environment (DCE) Orbix IOP IIOP GIOP WSDL WS-BPEL WSIL Java Transaction API (JTA) JNDI LDAP JMS BPEL BEA Tuxedo® Object Request Broker (ORB) EAI RTCORBA SOAP Message Queuing (MSMQ) Distributed Component XQuery Object Model (DCOM) opal. ORB XPath Remote Method Invocation TM INI ORBlite Encina/9000 (RMI) Rendezvous Enterprise BEA Web. Logic® Java. Beans Remote Procedure Call Technology (RPC) (EJB) Extensible Markup Language (XML) ZEN IDL J Borland® Visi. Broker®
More Views of Middleware z Software technologies to help manage complexity and heterogeneity inherent to the development of distributed systems, distributed applications, and information systems z Higher-level programming abstraction for developing distributed applications z Higher than “lower” level abstractions, such as sockets, monitors provided by the operating system y a socket is a communication end-point from which data can be read or onto which data can be written From Arno Jacobsen lectures, Univ. of Toronto
Middleware Systems – more views z Aims at reducing the burden of developing distributed applications for the developer y informally called “plumbing”, i. e. , like pipes that connect entities for communication y often called “glue code”, i. e. , it glues independent systems together and makes them work together z Masks the heterogeneity programmers of distributed applications have to deal with y network & hardware y operating system & programming language y different middleware platforms y location, access, failure, concurrency, mobility, . . . z often also referred to as transparency mechanisms y network transparency, location transparency From Arno Jacobsen lectures, Univ. of Toronto
Middleware Systems Views z An operating system is “the software that makes the hardware usable” z Similarly, a middleware system makes the distributed system programmable and manageable z Bare computer without OS could be programmed, y programs could be written in assembly, but higher-level languages are far more productive for this purpose z Distributed application be developed without middleware y But far more cumbersome From Arno Jacobsen lectures, Univ. of Toronto
Distributed Systems Multiple independent computers that appear as one y. Lamport’s Definition x“ You know you have one when the crash of a computer you have never heard of stops you from getting any work done. ” y“A number of interconnected autonomous computers that provide services to meet the information processing needs of modern enterprises. ” Intro to Distributed Systems Middleware 14
Examples of Distributed Systems z Banking systems z Communication - email z Distributed information systems y. WWW y. Federated Databases z Manufacturing and process control z Inventory systems z General purpose (university, office automation) Intro to Distributed Systems Middleware 15
Characterizing Distributed Systems z Multiple Computers yeach consisting of CPU’s, local memory, stable storage, I/O paths connecting to the environment z Interconnections ysome I/O paths interconnect computers that talk to each other z Shared State ysystems cooperate to maintain shared state ymaintaining global invariants requires correct and coordinated operation of multiple computers. Intro to Distributed Systems Middleware 16
Why Distributed Computing? z Inherent distribution y. Bridge customers, suppliers, and companies at different sites. z Speedup - improved performance z Fault tolerance z Resource Sharing y. Exploitation of special hardware z Scalability z Flexibility Intro to Distributed Systems Middleware 17
Why are Distributed Systems Hard? z Scale ynumeric, geographic, administrative z Loss of control over parts of the system z Unreliability of message passing yunreliable communication, insecure communication, costly communication z Failure y. Parts of the system are down or inaccessible y. Independent failure is desirable Intro to Distributed Systems Middleware 18
Design goals of a distributed system z Sharing y. HW, SW, services, applications z Openness(extensibility) yuse of standard interfaces, advertise services, microkernels z Concurrency ycompete vs. cooperate z Scalability yavoids centralization z Fault tolerance/availability z Transparency ylocation, migration, replication, failure, concurrency Intro to Distributed Systems Middleware 19
• Personalized Environment • Predictable Response • Location Independence • Platform Independence • Flexibility • Code Reusability • Real-Time Access • Increased • Interoperability to information Complexity • Portability • Lack of Mgmt. • Scalability • Reduced Tools • Faster Developmt. Complexity And deployment of • Changing Business Solutions Technology ORGANIZATION Intro to Distributed Systems Middleware System Administrator Application Developer END-USER [Khanna 94] 20
Application Systems: support enterprise systems Distributed Computing Platform • Application Support Services (OS, DB support, Directories, RPC) • Communication Network Services (Network protocols, Physical devices) • Hardware Intro to Distributed Systems Middleware Interoperability Portability Integration Management and Support Network Management Enterprise Systems: Perform enterprise activities 21
Application Systems: User Processing Data files & Interfaces programs Databases Distributed Computing Platform • Application Support Services Dist. Data Distributed C/S Support Trans. Mgmt. OS Interoperability Portability Integration Management and Support Network Management Enterprise Systems: • Engineering systems • Manufacturing • Office systems • Business systems Common Network Services • Network protocols & interconnectivity OSI TCP/IP protocols Intro to Distributed Systems Middleware 22
An Event-driven Architecture for a Real-time Enterprise The Enterprise Services Bus
Distributed Systems & Middleware Research at UC Irvine z Safe and Adaptive Middleware y Comp. OSE|Q - Safe composability of m/w services and protocols x Security, fault tolerance, reliability, QOS, mobility y Contessa – Context Sensitive System Adaptation (formal methods based) x Adaptive Data Collection – wireless and instrumented sensor networks x Adaptive Communication -- groupware on MANETS, mesh networks z Mobile Multimedia Systems and Applications y Mobile Crowdsourcing; Multimedia over Hybrid Networks y MAPCLOUD –Cloud Computing for Mobile Applications y FORGE /x. Tune – Cross-Layer Adaptation and Tuning (OS, Device, Network, Application) Techniques for mobile embedded systems; On-the-fly formal methods for cross-layer adaptation z Pervasive/Cyber. Physical Systems and Applications y Irvine Sensorium – A Next Generation Pervasive Computing Testbed y CYPRESS – Middleware for Resilient Cyber. Physical Spaces y SATWARE – Stream Acquisition and Transformation Middleware z Application Focused Distributed Middleware y RESCUE, SAFIRE : Improving Information Flow in Crises, Situational Awareness for Firefighters y Crisis. Alert, SIGNAL: Socio. Technical Alerting and Notification Systems 24
Mobile Middleware 25
Dynamo: Power Aware Mobile Middleware To build a power-cognizant distributed middleware framework that can o exploit global changes (network congestion, system loads, mobility patterns) o co-ordinate power management strategies at different levels (application, middleware, OS, architecture) o maximize the utility (application Qo. S, power savings) of a low-power device. o study and evaluate cross layer adaptation techniques for performance vs. quality vs. power tradeoffs for mobile handheld devices. Network Infrastructure Caching Compress Encryption Decryption Compositing Transcode Execute Remote Tasks Low-power mobile device Wide Area Network Wireless Network proxy Use a Proxy-Based Architecture 26
Middleware for Pervasive Systems UCI I-Sensorium Infrastructure Campus-wide infrastructure to instrument, experiments, monitor, disaster drills & to validate technologies sensing, communicating, storage & computing infrastructure Software for real-time collection, analysis, and processing of sensor information used to create real time information awareness & post-drill analysis 27 27
SATware: A semantic middleware for multisensor applications Abstraction - makes programming easy - hides heterogeneity, failures, concurrency Provides core services across sensors - alerts, triggers, storage, queries Mediates app needs and resource constraints - networking, computation, device 28
SATWARE – Semantic Middleware for Cyberphysical Spaces SAFIRE- situational Calit 2 Recycling Monitor awareness System Indoor Localization Privacy Preserving Surveillance ne ar by se ns ors ! Occupancy Forecasting System Evac. Pack 29
SAFIRE : Situational Awareness for Firefighters SAFIREStreams Sensor Data Ingest Unit Sp. CO, light, inertial, RFID, heart rate, . . Temperature humidity, visibility Sensor Data Collectio n Sensor Stream Processing Module Virtual Sensors for Media Level events (Alerts, Queries, Replay, Triggers) Firefighter Status Dashboard Sensor Fusion Multisensor Event Extraction Multimedi a Data Collection Visualization & Decision Support Services Available GIS layers Mapping and Localization Ambient CO Image/Video Audio/speech Receive /display alert messages. Weather Raw Data DB Raw Sensor data (sensors, speech, video) Event DB Semantically Enriched Event Data Sensor/Incident Storage& Archival Floor plans HAZMAT CAD Systems Demographics Occupancy Ebox External Data Access Goal: Reliable Timely SA over Unpredictable Infrastructure
Next Generation Alerting and Warning Project Dissemination in the Large Delivery Layer Research Wired Networks Wireless Networks Content Layer Research Efficient Publish Subscribe Content Customization 31 Systems and Deployments Crisis. Alert Disaster. Portal
Classifying Distributed Systems z Based on degree of synchrony y. Synchronous y. Asynchronous z Based on communication medium y. Message Passing y. Shared Memory z Fault model y. Crash failures y. Byzantine failures Intro to Distributed Systems Middleware 32
Computation in distributed systems z Asynchronous system y no assumptions about process execution speeds and message delivery delays z Synchronous system y make assumptions about relative speeds of processes and delays associated with communication channels y constrains implementation of processes and communication z Models of concurrency y Communicating sequential processes, Pi calculus y Functions, Logical clauses, Passive Objects y Active objects - Actors, Agents y Transactions - ACID properties (Atomicity, Consistency, Isolation, Durability); relaxed models z General correctness of distributed computation y Safety, Liveness Intro to Distributed Systems Middleware 33
Flynn’s Taxonomy for Parallel Computing Single (SD) Multiple (MD) Data Instructions Single (SI) Multiple (MI) SISD MISD Single-threaded process Pipeline architecture SIMD MIMD Vector Processing Multi-threaded Programming Parallelism – A Practical Realization of Concurrency
SISD (Single Instruction Single Data Stream) Processor D D D D Instructions A sequential computer which exploits no parallelism in either the instruction or data streams. Examples of SISD architecture are the traditional uniprocessor machines (currently manufactured PCs have multiple processors) or old mainframes.
SIMD Processor D 0 D 0 D 1 D 1 D 2 D 2 D 3 D 3 D 4 D 4 … … … … Dn Dn Instructions A computer which exploits multiple data streams against a single instruction stream to perform operations which may be naturally parallelized. For example, an array processor or GPU.
MISD (Multiple Instruction Single Data) D Instructions Multiple instructions operate on a single data stream. Uncommon architecture which is generally used for fault tolerance. Heterogeneous systems operate on the same data stream and aim to agree on the result. Examples include the Space Shuttle flight control computer. Intro to Distributed Systems Middleware 37
MIMD Processor D D D D D Instructions Processor D D Instructions Multiple autonomous processors simultaneously executing different instructions on different data. Distributed systems are generally recognized to be MIMD architectures; either exploiting a single shared memory space or a distributed memory space.
Communication in Distributed Systems z Provide support for entities to communicate among themselves y. Centralized (traditional) OS’s - local communication support y. Distributed systems - communication across machine boundaries (WAN, LAN). z 2 paradigms y. Message Passing x. Processes communicate by sharing messages y. Distributed Shared Memory (DSM) x. Communication through a virtual shared memory. Intro to Distributed Systems Middleware 39
Message Passing z Basic communication primitives y Send message y Receive message z Modes of communication y Synchronous xatomic action requiring the participation of the sender and receiver. x. Blocking send: blocks until message is transmitted out of the system send queue x. Blocking receive: blocks until message arrives in receive queue y Asynchronous x. Non-blocking send: sending process continues after message is sent x. Blocking or non-blocking receive: Blocking receive implemented by timeout or threads. Non-blocking receive proceeds while waiting for message. Message is queued(BUFFERED) upon arrival. Intro to Distributed Systems Middleware 40
Reliability issues z Unreliable communication y. Best effort, No ACK’s or retransmissions y. Application programmer designs own reliability mechanism z Reliable communication y. Different degrees of reliability y. Processes have some guarantee that messages will be delivered. y. Reliability mechanisms - ACKs, NACKs. Intro to Distributed Systems Middleware 41
Reliability issues z Unreliable communication y. Best effort, No ACK’s or retransmissions y. Application programmer designs own reliability mechanism z Reliable communication y. Different degrees of reliability y. Processes have some guarantee that messages will be delivered. y. Reliability mechanisms - ACKs, NACKs. Intro to Distributed Systems Middleware 42
Distributed Shared Memory z Abstraction used for processes on machines that do not share memory y. Motivated by shared memory multiprocessors that do share memory z Processes read and write from virtual shared memory. y. Primitives - read and write y. OS ensures that all processes see all updates z Caching on local node for efficiency y. Issue - cache consistency Intro to Distributed Systems Middleware 43
Remote Procedure Call z Builds on message passing y extend traditional procedure call to perform transfer of control and data across network y Easy to use - fits well with the client/server model. y Helps programmer focus on the application instead of the communication protocol. y Server is a collection of exported procedures on some shared resource y Variety of RPC semantics x“maybe call” x“at least once call” x“at most once call” Intro to Distributed Systems Middleware 44
Fault Models in Distributed Systems z Crash failures y. A processor experiences a crash failure when it ceases to operate at some point without any warning. Failure may not be detectable by other processors. x. Failstop - processor fails by halting; detectable by other processors. z Byzantine failures ycompletely unconstrained failures yconservative, worst-case assumption for behavior of hardware and software ycovers the possibility of intelligent (human) intrusion. Intro to Distributed Systems Middleware 45
Other Fault Models in Distributed Systems z Dealing with message loss y. Crash + Link x. Processor fails by halting. Link fails by losing messages but does not delay, duplicate or corrupt messages. y. Receive Omission xprocessor receives only a subset of messages sent to it. y. Send Omission xprocessor fails by transmitting only a subset of the messages it actually attempts to send. y. General Omission x. Receive and/or send omission Intro to Distributed Systems Middleware 46
Other distributed system issues z Concurrency and Synchronization z Distributed Deadlocks z Time in distributed systems z Naming z Replication yimprove availability and performance z Migration yof processes and data z Security yeavesdropping, masquerading, message tampering, replaying Intro to Distributed Systems Middleware 47
Traditional Systems Client/Server Computing z Allocates application processing between client and server processes. z A typical application has three basic components: y Presentation logic, Application logic, Data management logic z Three different C/S models for distributing functions: y Presentation logic module running on the client system and the other two modules running on one or more servers. y Presentation logic and application logic modules running on the client system and the data management logic module running on one or more servers. y Presentation logic and a part of application logic module running on the client system and the other part(s) of the application logic module and data management module running on one or more servers Intro to Distributed Systems Middleware 48
Typical Middleware View z Enables the modular interconnection of distributed software (typically via services) x abstract over low level mechanisms used to implement resource management services. x Support separation of concerns and reuse of services z Customizable and Composable x Provide for dynamic invocation/revocation/installation of services. x Concurrent execution of multiple distributed systems policies. Application Program API Middleware Service 1 API Middleware Service 2 API Middleware Service 3 Intro to Distributed Systems Middleware 49
Useful Middleware Services y. Naming and Directory Service y. State Capture Service y. Event Service y. Transaction Service y. Fault Detection Service y. Trading Service y. Replication Service y. Migration Service Intro to Distributed Systems Middleware 50
Types of Middleware z Integrated Sets of Services -- DCE z Domain Specific Integration frameworks y Transactions, Workflows, Network Management z Distributed Object Frameworks y Uses object models – Passive objects, Java objects/threads, Actors, Agents y OMG’s CORBA, Microsoft COM, Jini, z Component services and frameworks y Provide a specific function - Presentation, Communication, Control, Information Services, computation services etc. z Web-Service Based Frameworks Intro to Distributed Systems Middleware 51
- Slides: 50