Foundational Software Systems A Brief Overview Creative Commons

Foundational Software Systems A Brief Overview Creative Commons License – Curt Hill.

Introduction • Certain classes of software need to be understood by all technical people • Operating Systems • Servers • Databases and Data Warehouses • Enterprise Resource Planning • Customer Relationship Management Creative Commons License – Curt Hill.

Outroduction • • • What this presentation is not about: Virtualization Cloud computing Service Oriented Architecture Software as a Service These are important topics to be considered at another time Creative Commons License – Curt Hill.

OS • Foundation of the foundation • Every general purpose computer has one – Many special purpose computers as well • Applications are why we buy the computer, but the operating system is required to run the applications Creative Commons License – Curt Hill.

What does an OS do? • • • Execute application programs Provide services to these programs Establish and manage a file system Utility functions Many functions that are used by most programs will eventually be incorporated into the OS Creative Commons License – Curt Hill.

Execute application programs • Interpret commands – Interpret typed commands or mouse actions • Load and execute programs • Provide services to the running programs – The exact description of these services distinguishes different OSs Creative Commons License – Curt Hill.

Provide services to the program • Memory management – Memory allocation – Virtual or fixed memory • I/O management – Manage file activity – Often the most visible and important • Task Scheduler – Inter-process communication – Managing priorities Creative Commons License – Curt Hill.

Establish and manage a file system • Determine how things are stored on disk – Do we have a directory or just search? – Are things stored in a contiguous fashion or scattered? – How are things named? – How are things accessed? • Interrupt processing • Utility functions – Prepare a disk for use – Copy/move/delete files – Show contents of a file or disk Creative Commons License – Curt Hill.

How are files accessed? • Access to files is a program service – Program says that it wants access to a file of some filename – System finds it – Gets it ready – Gives it to the program one chunk at a time • Several things should be transparent to program • Buffering • Devices Creative Commons License – Curt Hill.

Utility functions • Protection – Files from deletion by someone other than owner – Memory of one program from another • Hardware manipulation – Mount disks and tapes • Miscellaneous – Error detection – Multitasking – Accounting Creative Commons License – Curt Hill.

Pieces of the operating system • • CPU management Memory management Process management I/O management User management File management Others Creative Commons License – Curt Hill.

OS Layers • Most OSs are onion shaped – For historical and maintenance reasons – Historically the innermost was all that was available • Each layer then communicates with adjacent layers through well defined paths – Well defined in terms of this particular OS • Layers: Kernel, Service, Command Creative Commons License – Curt Hill.

Kernel is the innermost layer • Low level activities only – – Load and/or execute a program Device allocation and access Process management Low level file access • This is the lowest level interface to the hardware and the only one that touches the hardware directly • In DOS there is the BIOS which amounts to a kernel • The kernel is usually always resident in the memory • Basic process management is usually here, since we need to make each program we load a process Creative Commons License – Curt Hill.

Service layer • Set of routines that provide services – Use the kernel – This is where the traditional file manipulation commands are executed, such as opening and closing files, reading and writing records – Entire file copying is usually done at the next higher level • The service layer represents the services that are available to a program • The service layer is also designed to generalize the interface between devices – A CRT, disk and tape are wildly different, yet we would like their interface to look similar Creative Commons License – Curt Hill.

Service Layer • In the assembly language interface each of the service calls becomes a macro or subroutine call • In a HLL it is usually built into the command part of the language or is in a library of some sort • Some process management is here as well – Inter-process communication Creative Commons License – Curt Hill.

Command layer • The interface that allows the user to interface with the system directly • This includes command interpretation • This is Command. com in DOS or program manager in Windows • Many utility functions are placed in this layer – The DOS Format is a command in this layer • It performs an essential function, but is its own program Creative Commons License – Curt Hill.

Familiar OSs • Windows – Several versions from personal to server • UNIX – Bell Labs and BSD • Mac OS – OS 10 is a BSD UNIX with specialized Mac GUI on top • LINUX – UNIX like alternative to UNIX Creative Commons License – Curt Hill.

LINUX Again • LINUX is open source so there are many distributions of it: – Debian – Ubuntu – Fedora – SUSE – Red Hat Enterprise • There are very many others Creative Commons License – Curt Hill.

Servers • Client server computing is the basis of the Internet – Among other things • Requires two computers – A client – A server • Communication medium • The illusion is that each client computer thinks it is has exclusive access to the server Creative Commons License – Curt Hill.

The Paradigm • • • Server provides one service Waits for requests for that service Client initiates a request Server replies to the request Information may flow in both directions Many clients may make requests with just the one server Creative Commons License – Curt Hill.

Why Client - Server? • Specialization – The client handles the user interface using local computational ability – The server manages the data and communication • Sharing – One server may support many clients – Services provided are effectively shared – Data may be maintained in only one location Creative Commons License – Curt Hill.

Clients • An application program that runs on the local computer • Able to use the local computer for doing work • When invoked by a user, it initiates communication with a server • Handles all the user interface issues • Only used for a single session Creative Commons License – Curt Hill.

Example • • • The ubiquitous client is the web browser Several functions: Communicate with server Display HTML/XML Interpret client-side languages, such as Java. Script • The trend is for the web browser to be the universal interface to every kind of server on the web Creative Commons License – Curt Hill.

Servers • Specialized program that typically provides only one service • May handle multiple client requests simultaneously (or pseudo simultaneously) • Interacts with users in many sessions – Both simultaneously and over long time periods • Waits for a request and then produces the reply Creative Commons License – Curt Hill.

Example • The web server • Waits for requests using Hyper Text Transmission Protocol • Produces results in HTML/XML • May handle many clients simultaneously Creative Commons License – Curt Hill.

Connection Diagrammed Server Request Client Reply Request Client Creative Commons License – Curt Hill. Reply Client

Interaction Diagrammed Time Client Processing Request Server Wait Reply Wait • The server is not waiting if it is processing other requests • The client may also perform other processing while waiting for the reply Creative Commons License – Curt Hill.

Multiple tiers • The classic client-server is a two tier system • The client is one tier and the server another • Three or more tiers are also possible • The middle server functions as a – Server to the client – Client to other servers – Known as middleware • Often used to distribute load Creative Commons License – Curt Hill.

A Three Tier Approach Server Application Server Client Creative Commons License – Curt Hill. Client

Examples of Multi-Tier • Authentication server – Maintains a connection to verify user – Funnels all requests to appropriate server • Shopping cart – Records purchases – The product lookup/purchase handled by other servers • Distributed Database Management System Creative Commons License – Curt Hill.

Horizontal Distribution • Multi-Tier is a vertical distribution • A Horizontal distribution may also be effective • A single server owns the web address: www. x. com • It then distributes all further requests to a rotation of identical web servers – www 1. x. com – www 2. x. com –… – www. N. x. com Creative Commons License – Curt Hill.

Horizontal Distribution www. x First Request www 1. x www 2. x … www. N. x Subsequent Request Client Client Creative Commons License – Curt Hill.

Database • The web server may be the most common server type, but the database server is the workhorse • It is the foundation for many other important classes of programs Creative Commons License – Curt Hill.

Historical Perspective • Actions we might want on any file of fixed records: – – Create Update (add, remove, change records) Sort Generate any of several reports • Each of these would be a program for an overworked programming staff – Typically a COBOL program Creative Commons License – Curt Hill.

Database • The key – All these programs are about the same – All that changes is the underlying file • The solution – Describe the file in a general way • This is the metadata – data that describes data – Generate a program that handles the file based on the description Creative Commons License – Curt Hill.

How to describe a file • A file is a collection of records • Each record is a collection of fields – Typically one type of record in a file • Each field is described by a: – Name – Type • For example numeric, string, boolean etc. – Length • Booleans have a predefined length, others require specification Creative Commons License – Curt Hill.

Database Advantages 1 • Data independence – Application program no longer need some or all of the files – Do not know or care how data is stored, aka abstraction – Simplifies application development • Efficient access – The DBMS employs sophisticated access techniques seldom used by normal programmers Creative Commons License – Curt Hill.

Database Advantages 2 • Integrity constraints – The DBMS may check data in a way seldom done in normal file processing – Eg. Account validity • Security – A DBMS may enforce requirements on who can access the data and in what way Creative Commons License – Curt Hill.

Database Advantages 3 • Administration – Minimize redundancy – Manage sharing of the data – Optimize for the enterprise, not a small group – Easier to backup the data • Concurrent access – Manages the simultaneous update problem Creative Commons License – Curt Hill.

Functions of a Database • • • Data storage and retrieval User-accessible catalog Transaction support Concurrency support Recovery services Authorization services Support for data communication Integrity services Data independence services Creative Commons License – Curt Hill.

Data storage and retrieval • While hiding the file structure, users may: – Add – Delete – Update in place • Most relational databases support a common language – Structured Query Language Creative Commons License – Curt Hill.

SQL • A language where a question is asked of the database and it provides an answer: – Pronounced Sequel or spelled out – Usually state what is wanted rather than how to get it • The Data Manipulation Language usually includes both query and update facilities Creative Commons License – Curt Hill.

Transaction Support • The unit of update is a transaction – All or nothing – No partial updates are allowed • The DBMS must guarantee the integrity of a transaction, even in the face of: – Program errors – Incorrect input – System crashes Creative Commons License – Curt Hill.

Transactions: ACID • Atomic – Transaction perceived to be indivisible • Consistent – Transforms database from one consistent state to another • Isolated – Understandable without regards to any other agents • Durable – Once committed permanence is guaranteed even with system crashes Creative Commons License – Curt Hill.

Types of database • A large organization often have two distinct types of databases • A transactional or operational database • An analytical database or data warehouse • No. SQL • These have separate uses Creative Commons License – Curt Hill.

Database uses • The database is about transactions – Maintaining current knowledge • The data warehouse is about decision support – Spotting trends • No. SQL is much harder to classify, any of these – Massive data – Unique organizations – Distributed – Among many others Creative Commons License – Curt Hill.

Characteristics • Operational database have: – Strict performance requirements – Predictable workloads – Small units of work – High utilization • Data warehouses are contrary in every respect Creative Commons License – Curt Hill.

Warehouses • A data warehouse – Subject oriented, integrated, non-volatile, time-variant collection of data in support of management decsions • Data warehouses support – OLAP (On. Line Analytical Processing) – DSS/EIS (Decision Support Systems or Executive Information Systems) – Data mining Creative Commons License – Curt Hill.

What is Data Mining? • Exploratory data analysis based on a data warehouse – Knowledge Discovery in Databases (KDD) • Data Mining extracts previously unknown and potentially useful information – Rules, constraints, correlations, patterns, signatures and irregularities • The goal is to automate the methods for finding these in the data Creative Commons License – Curt Hill.

Goals of Data Mining • Prediction of future behaviors – Seasonal or non-seasonal trends – How will consumers respond to discounts? – Allows the enterprise to be ready • Identification of item, event or activity – Intruders may be identified by the files they access or programs they use Creative Commons License – Curt Hill.

Goals Again • Classification of categories of users or products – Shoppers may be categorized as: • • Discount seeking Rush Regular Attached to certain brand names – The store may be made more friendly to such • Optimize the use of time, space, materials and money Creative Commons License – Curt Hill.

Knowledge Discovery • There are several types of discoverable knowledge: • Association Rules • Classification hierarchies • Sequential patterns • Time series patterns • Clustering Creative Commons License – Curt Hill.

Databases • The two types of databases are the transactional and analytical – The day to day transactional feeds the analytical • We do not want knowledge discovery, data mining and the like to occur on the transactional database – Could seriously impede performance • Either might be used in the next two applications – Transactional is more likely Creative Commons License – Curt Hill.

• Data Some Definitions – Small measurable pieces of information – Foundation of knowledge • Information – Organized data – Putting data into meaningful patterns • Knowledge – A higher aggregation of information and data that enables interpretation – Ability to use information – Sets of rules and relationships Creative Commons License – Curt Hill.

Example • Any numbers or text is data – Such as: 324 3. 83 • It becomes information when we know its context – Such as these are the number of items in stock and how much we paid per unit • Knowledge is when we take this information and combine it with processes and rules – Computing our price for selling something made out of these Creative Commons License – Curt Hill.

Data Mart • A data warehouse restricted to a single subject – Such as a group or line of business within a larger organization • A dependent data mart obtains its data from the central organizational data warehouse • An independent data mart obtains data from other sources – No overlap with the data warehouse Creative Commons License – Curt Hill.

Knowledge Warehouse • An architectural integration of knowledge management, decision support, artificial intelligence and data warehousing • Built on top of a data warehouse • Must also encode the rules and processes of the business – These rules used to only be in the minds of employees – Next captured on documents Creative Commons License – Curt Hill.

ERP • Enterprise Resource Planning • The key idea behind database technology is that the process of manipulating files is basically the same – The only thing that changes is the file format • The key to ERP is that many business processes are very similar – Payroll is just payroll • Took off because of the Euro and Y 2 K threat Creative Commons License – Curt Hill.

ERP Areas • • Financial Accounting Management Accounting Human Resources Manufacturing Supply Chain Management Project Management Customer Relationship Management These were all separate programs at one time Creative Commons License – Curt Hill.

Configuration • Most systems need to be customized by programmers modifying the code • Most ERPs may be configured to customize to a particular situation • This configuration is usually much easier than code modification • The program typically handles best practices, but the configuration can take into account the idiosyncrasies of this business Creative Commons License – Curt Hill.

Simple Example • The form of a product number drastically varies between companies – Usually groups of letters and digits • Configuring the system to store and validate the type used by a particular company needs to be a simple configuration Creative Commons License – Curt Hill.

• • CRM Customer Relationship Management The notion of a call center A past, present or future customer calls A representative, with no previous knowledge of this customer, has to deal with them • CRM puts all the information necessary to help in front of the representative, regardless of the purpose of the call Creative Commons License – Curt Hill.

Types of calls • Sales – Availability, delivery, costs and discounts • Support after the sale – Warranty, replacements, returns • Information and marketing – Request for information • Appointments for sales force • Analysis of customer or products Creative Commons License – Curt Hill.

Implementation • The leading CRMs, like ERPs are highly configurable • Also like ERPs they are backed by a transactional database • Both generate large quantities of SQL on the fly • ERPs are starting to absorb CRMs Creative Commons License – Curt Hill.

Finally • The type or size of businesses that would be architected will likely have all of these • They will also have serious customizations to these products • Many other specialized applications also exist – EG. EA tools Creative Commons License – Curt Hill.
- Slides: 65