The Software Matrix Simplifying Software Salvage Riddhiman Ghosh

The Software Matrix Simplifying Software Salvage Riddhiman Ghosh Advisor: Dr. James Fawcett

Introduction l Approximately 100 billion lines of source code at work in the world today. l Large fractions of code in systems are functionally equivalent. l However systems are often constructed without leveraging existing code bases – less than 15% of new code serves an original purpose. l “Reinvention of the wheel” in the software industry. 2

Introduction l Software construction is needlessly error-prone and expensive. – We’re maintaining multiple copies of essentially the same software. – Time and cost of developing, testing and documenting a piece of software is multiplied by the number of equivalent copies in existence. 3

Software Reuse l Study of reuse – recycling of software assets – has been an important branch in the software engineering discipline. l Effective reuse promises – – – l Reduced development and maintenance costs Gains in development schedule and quicker time-tomarket Increased robustness and quality Not a new idea. 4

Software Reuse l Systematic Reuse: “an institutionalized organizational approach to product development in which software assets are intentionally created or acquired to be reusable” l However, few organizations practice systematic reuse, in spite of a recognition of its potential benefits. l Reuse is hard! 5

Gap between “theory” and “practice” in Reuse “ Not all the “theoretical oriented” and “sophisticated” solutions presented by researcher[s] have. . . thrilled the practitioners. . . – Zand, M. . the reuse community has worked on complex technologies and methods with high ceremony, yet most of the software community seems to be looking for simpler solutions. . . High ceremony methods require an organization with high process maturity to achieve success. – Griss, M. 6

Software Salvage l What we seen in the industry is not software reuse, but rather “software salvage” l Salvage: lifting of a significant block of existing systems and inserting them into a newly developed system. l Radar Systems Department, General Electric Company (Syracuse, NY) routinely attempted salvage in the building of a new radar. 7

Reuse vs. Salvage l We make a distinction between the terms software reuse and software salvage. l Reuse connotes immutability – l the individual pieces meant for reuse cannot be modified; they’re designed and implemented to be adaptable but with no intent to change even a single character of source code. Salvage makes no guarantee of immutability – very often source code of individual pieces being salvaged is modified. 8

Problem l Effective salvage is difficult to accomplish – The large pieces we want to recycle often have many dependencies on parts we don’t want. – These dependencies usually require expensive changes to the part being salvaged. – Pulling out pieces from different systems, and getting these extricated pieces to work together is challenging. – Think of salvage as analogous to transplanting the heart from one living organism to another! Similar problems, when we pull out a part from a software system due to the connectedness inherent in typical software. 9

Problem It is difficult to salvage existing parts systems and build new systems from them, with ease. 10

Goal l Enable leveraging of major parts of systems l Eliminate or reduce changes required by salvage so that salvage moves closer to the reuse model (immutability), without requiring high-process organization. l Simplify software salvage to make it a useful paradigm. 11

Prior Approaches l Two major (and fairly recent) approaches towards encouraging reuse have been – object-oriented reuse – component-oriented reuse. 12

Object-Oriented Reuse l Reuse has been one of the classic motivations of object-orientation l Object oriented theory – – mandated discipline in writing code through bestpractice guidelines such as separating interface from implementation Encouraged reuse through inheritance, composition, parameterization 13

Object-Oriented Reuse l However OO technologies didn’t quite engender the reuse revolution that was hoped for. – – Perhaps the only OO libraries widely used userinterface frameworks (such as MFC) and libraries for data structures. While OO techniques have made compiler libraries an effective means of reuse, business organizations have had a harder time getting leverage from their pre-existing software assets only through the creation and consumption of OO libraries. 14

Object-Oriented Reuse l The definition of an object is purely “technical” – – l Defined as an encapsulation of state and behavior No direct mapping to the physical unit that is actually deployed, versioned and potentially reused. Objects don’t fit the “salvage” model well – – In terms of granularity, they are of an inappropriate size to be mixed and matched. They are the size of grains of sand, while what salvagers are typically looking for in building systems, are bricks. 15

Component-Oriented Systems l The logical next step was the notion of components – – a coherent package of software implementation that can be independently deployed and composed with other components. of coarser granularity than objects independent and deployable implies executable or loadable code The defining feature of component technology is that a system, which is composed of components, can be repaired or made better by updating components, without rebuilding the whole system. 16

Component-Oriented Systems l There are several component technology offerings – COM, CORBA, Java. Beans/EJB, . NET Components l Adopting a component-oriented approach (rather than only concentrating on programming language as in OO theory) is beneficial. l But the existence of these component technologies has not solved the problems that plague software salvage, discussed earlier l Packaging techniques alone are not enough 17

Our Approach l We are of the view that components are useful only if there is a framework that actively supports and promotes their reuse. l . NET and J 2 EE have framework support, however it is focused on generic industry problems (network communication, web publishing, application security, etc. ) We seek to support vertical, perhaps proprietary, applications, where building a huge framework is impractical from a return on investment point of view. 18 l

Our Approach l Make salvage easier by – – l viewing applications as compositions of different pieces having a framework of collaborating pieces from which applications can be composed dynamically Try to achieve benefits of the “Software-IC” model— a plug-and-play approach to software construction. 19

Our Approach l We are limiting the hard problem of general reuse to a smaller domain—of salvage within an organization, to be used in the construction of modest-sized systems (about 200, 000 lines of code). l To address this domain would be to provide solutions of value to the small and medium-sized software shops that have been resistant to adopt systematic reuse. 20

Software Matrix l The Software Matrix is a framework that actively supports and promotes the salvage of components. l We focus on the reuse of major blocks of code rather than low-level components l By employing message-passing and mediator structures and by supporting the discovery of needed types, we’ve built a pluggable architecture that can gracefully adapt to salvage operations. 21

Software Matrix l In particular, the Matrix is a runtime infrastructure that acts as a substrate into which individual pieces of an application – different blocks of code – can plug. 22

Software Matrix From these “plugged-in” individual pieces the Matrix dynamically composes applications. 23

Software Matrix l This infrastructure was named the Software Matrix in order to connote a very structured pattern of building software. l System elements are embedded in this substrate – a matrix /collection – through which applications are dynamically composed. l The image of endless banks of incubators of humans in a matrix, as from “The Matrix” motion picture, isn’t entirely unintentional, if somewhat flippant. 24

Cells • The individual blocks of code that plug-in to the Matrix are called “Cells” and are the building blocks out of which applications are built. 25

Cells • Cells represent the unit of composition and reuse in our system. 26

Cells l An application is built through the collaboration of Cells, and all Cells communicate with each other strictly through messages (we use XML encoded messages) 27

Cells l One of the problems of salvaging: extracting parts of monolithic (or very tightly-coupled) applications. l By enforcing a separation of concern between Cells by inserting a message bus between them we are insisting on loose-coupling between different pieces of an application, which makes salvaging these pieces for use in future applications much easier. 28

A closer look at Cells… l Every Cell contains: – a message queue: holds request and response messages during collaboration with other cells. 29

A closer look at Cells… l Every Cell contains: – a capability list: used to advertise the capabilities of a cell to other cells, via the Matrix. (E. g. SU. Math. Convolution) The capability list is used by the Matrix in order to discover the right cells for system construction. 30

A closer look at Cells… l Every Cell contains: – a globally unique identifier (GUID): Cells can be uniquely identified using a GUID. This is used by the Matrix for several operations such as discovery and registration. 31

A closer look at Cells… l Every Cell contains: – functionality: Cells also contain the functionality that allows them to be considered as software assets with potential for reuse. 32

A closer look at Cells… l All cells subscribe to a common protocol (ICell) that specifies how to: – – register and un-register with the Matrix advertise capabilities send accept messages collaborate with other cells (could be synchronous, asynchronous or one-way) ICell My. Cell 33

A closer look at Cells… l Every cell also has an entry-point (start), and is given a chance to execute once it is plugged-in. l This entry-point (empty/non-empty) decides whether a cell will be only a passive ‘server’, or itself actively seek collaboration from other cells. 34

Example l Sample application: needs to read data samples from an input file, perform a signal processing operation on the a data (e. g. , filtering), plot the results of the operations on the system display, and log the results to a file. l The major pieces of the application would be responsible for: – – – file operations signal processing graphical plotting 35

Example l l These would be written as cells and plugged-in to the Matrix. The Matrix would then assume the responsibility of constructing the application from these individual cells. In order to perform its task a cell may need services of another cell. But cells do not explicitly bind to other cells – they only specify what message type they need handled, and the Matrix discovers cells capable of handling that message. If no suitable cell found, a “not supported” message is generated. 36

Example 37

Example Sequence of events in the construction of the sample application 38

Example l So what we see here is dynamic composition – we are building a system from pieces that exist on the Matrix at runtime. The compositional aspects (as opposed to only computational aspects) of software being taken care of by the Matrix. l It automatically connects the right pieces at run-time without having to bind to anything explicitly at compiletime. l This method of software construction, with its dynamic nature and loosely coupled building blocks, is amenable to salvage operations. The very same Cells could be used to build other applications. 39

How do cells “plug-in”? l Cells are implemented as “plug-in” modules, and are realized using. NET components. l The Matrix uses the ‘reflection’ and ‘late-binding’ (Fusion) features of the. NET framework to discover and register plug-ins. l Given an. NET assembly, the Matrix will reflect over the contained types to determine which of them implement the ICell interface in order to recognize valid plug-ins. 40

How do cells “plug-in”? l Throughout its lifetime, the Matrix runtime infrastructure monitors a known location – the plug-in directory, in order to discover if any new cells are present. l On detecting a valid cell, the Matrix ‘registers’ the cell with itself. This newly registered cell (along with previously registered cells) is then available for system construction. 41

How do cells “plug-in”? At periodic intervals the Software Matrix interrogates a known location in order to discover if any new Cells are present, and if so registers them. 42

Steps to take advantage of the Matrix l In order to enable salvage, pieces of an application (at the time of writing it, or existing pieces) are wrapped in a Cell. l Create a wrapper that inherits from ICell (e. g. we wish to create File. Manager Cell responsible for common file operations ) File. Manager : ICell {. . . } 43

Steps to take advantage of the Matrix l. The Capability List of this File. Manager Cell is then populated to indicate the types of messages it is capable of handling. Capability. List. Add(“SU. File. Manager. Files. Read”); Capability. List. Add(“SU. File. Manager. Files. Write”); Capability. List. Add(“SU. File. Manager. Files. Search”); Capability. List. Add(“SU. File. Manager. Files. Compression”); 44

Steps to take advantage of the Matrix l Override the “process” method to add appropriate message processing – Basically you specify what is to be done in response to a particular message type – delegate calls to your implementation. 45

Steps to take advantage of the Matrix • Compile, and copy the resulting binary into the “plug-in” directory of the Software Matrix. The Matrix will automatically detect and register the cell, and it will be available for composition. 46

Steps to take advantage of the Matrix • If a cell wishes to use other cells (or is a program executive for instance), it will probably say: . . . Result = sync. Send( "SU. File. Manager. Files. Search", Params); . . . • Here it is trying to locate a cell that can handle the named message type. 47

Features l Fine-grained message-passing – – Many problems associated with software salvage arise from extracting one of several tightly-coupled pieces of a system. We are using message passing at a much finer level of granularity than is normally seen The Matrix requires messages be the only mode of collaboration between different parts of an application, thereby decreasing the degree of coupling. This loose coupling is critical to the success of salvage operations. 48

Features l Dynamic Composition – – The Matrix takes care of the compositional aspects of software by automatically discovering and connecting the right pieces and building an application at runtime By adding a few more cells into the Matrix if needed, we can build new applications by reusing existing cells. 49

Features l Support for System Evolution – – Since the appropriate cell to serve a particular message type is selected at runtime, the Matrix supports evolution of software systems. Evolving requirements can be accommodated easily by unplugging or modifying only those cells that represent the affected part of the system. Easy to field-replace cells – all one has to copy the new cell over the old cell in the plug-in directory. Effective way to support program maintainability, bug fixes/upgrades. 50

Features l Simplicity – – – The Matrix is a supporting infrastructure that is lightweight (the bare infrastructure is roughly only 2000 lines of code) Simple to use. The infrastructure comes with documented full source (if needed) and sample applications. The underlying technology used is the. NET component model which is considered superior and less complex as compared to other models such as COM. 51

Assessment l l Built applications in the “regular” style and using the Software Matrix. Working on an assessment model with quantitative and qualitative criteria for effective comparison. How easy is salvage? – – SLOC Complexity Changes necessitated, cognitive distance Performance 52

Comparison with prior and related work l There exists a combination of middleware and component technologies such as COM and CORBA that have reusability as one of their goals. – COM, CORBA are considered highly complex, overspecified and require heavy-weight supporting infrastructures. – The Matrix was meant to address the specific problem of salvage and is lightweight (approx. 2000 SLOC) and simple. 53

Comparison with prior and related work – Most middleware technologies use the procedure-call model of collaboration. Having components bind to exact function signatures is not flexible enough – leads to the tightly-coupled systems that make salvage operations difficult. – The Matrix uses message-passing as the mode of collaboration between components. This leads to looser coupling of system elements resulting in potentially easier salvage in the future. 54

Comparison with prior and related work – Message Oriented Middleware (MOM) and to a certain extent Web Services also use message-passing (most web services use XML-RPC, though they support a message-passing interface) – The Matrix differs from them in the granularity and scale of message-passing, since we are focused on local compositions of cells (components) to build applications, rather than accessing remote functionality over the network. 55

Conclusion l The Software Matrix is a runtime substrate with a plug-in architecture that enables simpler software salvage. Enforces loose coupling of system elements l Helps in the discovery and collaboration of components. l Easy to dynamically compose new applications from existing cells. l Provides support for product evolution and maintenance – important in large systems. l Simplifying software salvage is a worthy goal and will have a positive impact on productivity in the software industry. l 56

End of Presentation 57