Data Warehouse Fundamentals Chapter 4 Data Warehouse Project

  • Slides: 44
Download presentation
Data Warehouse Fundamentals Chapter 4 Data Warehouse Project Planning & Management Paul K Chen

Data Warehouse Fundamentals Chapter 4 Data Warehouse Project Planning & Management Paul K Chen 1

Chapter 4 - Objectives u u u u Review types of development models Review

Chapter 4 - Objectives u u u u Review types of development models Review the essentials of system development life cycle and project management functions Discuss project team organization, roles, and responsibilities Review data warehouse project scope document Consider the warning signs and success factors Distinguish between data warehouse projects and OLTP system projects Discuss Data Warehouse deployment

Types of Development Models u The Waterfall Development Model u The Spiral Model u

Types of Development Models u The Waterfall Development Model u The Spiral Model u The Iterative Development Model

The Waterfall Development Model Characteristics: u Encouraging to gather and define system requirements. u

The Waterfall Development Model Characteristics: u Encouraging to gather and define system requirements. u Breaking the complex mission of development into several logical steps (, analysis, design, code, test, and so forth) – Divide and conquer approach. u Ensuring each step is executed properly with good quality deliverable, validation, entry, and exit criteria for each step.

The Waterfall Development Model Advantages: u Enabling tracking of project progress more accurately and

The Waterfall Development Model Advantages: u Enabling tracking of project progress more accurately and uncovering possible slippages early. u Focusing the organization that develops the software system to be more structured and manageable. Disadvantages: u The process could become too rigid to be efficient and effective.

The Spiral Model Developed by Boehm in 1988 Characteristics: u Relying heavily on prototyping

The Spiral Model Developed by Boehm in 1988 Characteristics: u Relying heavily on prototyping and risk management vs. the document-driven approach of the waterfall approach. u Foe each portion of the project and for each of its levels of elaboration, the same sequence of steps (cycle) is involved. For instance, the concept of software requirements, to design, and implementation, each involves a spiral cycle.

The Spiral Model Approach: u The first step of each cycle is to identify

The Spiral Model Approach: u The first step of each cycle is to identify the objective of the portion of the product being elaborated, the alternative means of implementation of the portion of the product, and the constraints imposed on the application of the alternatives. u The next step is to evaluate the alternatives relative to the objectives and constraints and to identify the associated risks and resolve them. u In addition to prototyping for risk analysis, the spiral model also simulations, models, and benchmarks in order to reach the beat alternatives.

Waterfall Approach vs. Spiral Approach Structured Development: Analysis, design and coding take place in

Waterfall Approach vs. Spiral Approach Structured Development: Analysis, design and coding take place in The traditional waterfall way. Each step is isolated from the other. (Waterfall Approach) A D P Object-oriented development: One multifaceted model is used from Concept to code. Because one underlying model is used, teams apply Analysis, design And programming Concurrently. (Spiral Approach) A D P

The Iterative Development Model Characteristics: u u Begin with a subset of the requirements

The Iterative Development Model Characteristics: u u Begin with a subset of the requirements and develop a subset of the product that satisfies the essential needs of the users. Based on the analysis of each immediate product, the requirements and design are modified over a series of iterations to provide a system to the user that meets evolving customer needs with improved design based on feedback and testing. Combine with prototyping with the strength of the classical waterfall model. Supporting the iterative development was the small team approach in which each team assumed the full responsibility of the system.

System Development Life Cycle – A brief overview It is a systematic approach to

System Development Life Cycle – A brief overview It is a systematic approach to solving business problem. It’s divided into seven phases: u u u u Identifying problems, opportunities, and objectives Determining system requirements Analyzing system needs Designing the recommended systems Developing and documenting software Testing and maintaining the system Implementing and evaluating the systems

System Development Life Cycle – A Brief Overview Why should a system development project

System Development Life Cycle – A Brief Overview Why should a system development project be segmented in phases? Project Management– easier to understand manage its deliverables and track its progress u Resources – Better utilize the resources related to technology, skills, and time u Risk –Minimize commitment and cost in case the project restarts. u

Project Management Functions u Initiate project u Project planning u Establishing project u Organization

Project Management Functions u Initiate project u Project planning u Establishing project u Organization Start the project by assessing the opportunity Determining tasks, schedule, and allocating resources Defining project charter and issuing The statement of work Organizing staff by function, tools and environment

Project Management Functions (cont’d) u u u Administration On-going project reporting And administrative work

Project Management Functions (cont’d) u u u Administration On-going project reporting And administrative work Evaluation and control Monitor project progress by cost, product, and schedule Termination Wrapping up the task by doing project summary and archives

Five Major Project Fundamentals For System Analysts The five project fundamentals the system analysts

Five Major Project Fundamentals For System Analysts The five project fundamentals the system analysts must handle are: u Project initiation u Determining project feasibility u Project scheduling u Activity planning and control u Managing system analysis team members

Project Initiation Projects are initiated for two broad reasons: – Problems that lend themselves

Project Initiation Projects are initiated for two broad reasons: – Problems that lend themselves to systems solutions – Opportunities for improvement through » Upgrading systems » Altering systems » Installing new systems

Project Feasibility u A feasibility study assesses the operational, technical, and economic merits of

Project Feasibility u A feasibility study assesses the operational, technical, and economic merits of the proposed project u There are three types of feasibility: – Technical feasibility – Economic feasibility – Operational feasibility

Technical Feasibility u Technical feasibility assesses whether the current technical resources are sufficient for

Technical Feasibility u Technical feasibility assesses whether the current technical resources are sufficient for the new system u If they are not available, can they be upgraded to provide the level of technology necessary for the new system

Economic Feasibility u Economic feasibility determines whether the time and money are available to

Economic Feasibility u Economic feasibility determines whether the time and money are available to develop the system u Includes the purchase of – New equipment – Hardware – Software

Operational Feasibility u Operational feasibility determines if the human resources are available to operate

Operational Feasibility u Operational feasibility determines if the human resources are available to operate the system once it has been installed u Users that do not want a new system may prevent it from becoming operationally feasible

Determining Project Feasibility (Key Issues) u Value and Expectations u Risk Assessment u Top-down

Determining Project Feasibility (Key Issues) u Value and Expectations u Risk Assessment u Top-down or Bottom-up u Build or Buy u Single Vendor or Best-of-Breed

Tools for Planning & Scheduling Activities u Gantt Chart & PERT (Program Evaluation and

Tools for Planning & Scheduling Activities u Gantt Chart & PERT (Program Evaluation and Review Techniques) diagram; Spreadsheet u Computer-based project scheduling Such as: Microsoft Project; Computer Associates’ CASuper Project

Gantt Chart

Gantt Chart

Gantt vs. PERT Diagram A B C D E Time 4 2 A 4

Gantt vs. PERT Diagram A B C D E Time 4 2 A 4 10 5 20 9 15 C, 5 40 B, 2 E, 6 50 D, 3 30 The longest path is called critical path. Circles called events

Activity Planning and Control Beginning to plan a project by breaking it these three

Activity Planning and Control Beginning to plan a project by breaking it these three major activities : u u u Analysis Design Implementation

Activity Planning and Control Refining the planning and scheduling of analysis activities by adding

Activity Planning and Control Refining the planning and scheduling of analysis activities by adding detailed tasks and establishing the following milestones: u u u Data Gathering Data Flow & Decision analysis Proposal Preparation

Data Warehouse Project Team: Roles and Responsibilities u u u Executive Sponsor – Direction,

Data Warehouse Project Team: Roles and Responsibilities u u u Executive Sponsor – Direction, support, arbitration Project Manager – Assignment, monitoring, control User Liaison Manager – Coordination with user group Lead Architect – Architecture Design Business Analyst – Requirement definition Data Modeler – Relational and Dimensional Modeling Data Warehouse Administrator –DBA Quality Assurance Analyst – Quality control for warehouse data Testing Coordinator – Program, system and tool testing End-user Application Specialist – Confirmation of data meanings/relationships Development Programmer – in-house programming and scripts

Steps in Ascertaining Hardware and Software Needs u u u u Inventory computer hardware

Steps in Ascertaining Hardware and Software Needs u u u u Inventory computer hardware already in the organization Estimate both current and projected workload for the system Evaluate the performance of hardware and software using some predetermined criteria Choose the vendor according to the evaluation Acquire the hardware and software from the selected vendor Acquire the hardware and software in conformance with your enterprise architecture The acquisition of the hardware and software must be justified by a business process required of either shortterm (tactical) or long-term(strategic) goals.

Data Warehouse Project Scope Document I Executive Summary -- Business needs II Project Background

Data Warehouse Project Scope Document I Executive Summary -- Business needs II Project Background -- How did the project start? -- Who is the sponsor? III Project Definition -- Project Objectives -- Project Organization -- Project Critical Success Factor -- Measurements of Success

Data Warehouse Project Scope Document IV Project Scope What’s in the Data Warehouse? What’s

Data Warehouse Project Scope Document IV Project Scope What’s in the Data Warehouse? What’s not in the Data Warehouse? Samples of Queries & Reports V Methodology and Approach Methodology Employed Techniques Employed

Data Warehouse Project Scope Document VI Project Cost/Benefits VII Project Schedule, Budget and Resources

Data Warehouse Project Scope Document VI Project Cost/Benefits VII Project Schedule, Budget and Resources -- The plan should include the following milestones: u Logical Data Modeling u Data Warehouse Physical Model u Source System of Record u Extraction/Transformation Program u Populated Data Warehouse u Populated Metadata u End User Access Application u End User Training u Ongoing Support Plan

Data Warehouse Project Scope Document VIII Project Planning Assumptions and Issues -- Project Assumptions

Data Warehouse Project Scope Document VIII Project Planning Assumptions and Issues -- Project Assumptions -- Project Risks -- Project Contingencies IX Expected Follow-on Projects

Summary Project management consists of these four essential elements: Planning (an iterative process) Determining

Summary Project management consists of these four essential elements: Planning (an iterative process) Determining the deliverables Estimating efforts and cost Projecting the resources u Organizing Assembling the team Defining and establishing the structure of the team Creating a productive environment u

Summary Controlling the project Monitoring the progress Reporting performance and variables Adjusting resources u

Summary Controlling the project Monitoring the progress Reporting performance and variables Adjusting resources u Leading the project Emphasizing human factors—motivation; Team spirit; Delegation u

Consider the warning signs and success factors Warning Sign Indication Action The Data Requirements

Consider the warning signs and success factors Warning Sign Indication Action The Data Requirements definition phase is well the target date. Suffering from “analysis paralysis”. Stop the capturing of unwanted inf. Remove any problems by meeting with users. Set firm final target date. Need to write too many inhouse programs. Users not cooperating to provide details of data. Selected third party tools running out of steam. If there is time and budget, get different tools. Otherwise increase programming staff. Possible turf concerns over Work with executive data ownership. sponsor to resolve the issue.

Consider the warning signs and success factors (cont’d) Warning Sign Indication Users not comfortable

Consider the warning signs and success factors (cont’d) Warning Sign Indication Users not comfortable with the Users not trained adequately. query tools. Continuing problems with data brought over to the staging area. Data transformation and mapping not complete. Action First ensure that the selected query tool is appropriate. Then provide additional training. Revise all data transformation and integration routines. Ensure that no data is missing. Include the user representative in the verification process.

Data Warehouse Project Different From OLTP System Project Data Acquisition Data Storage Inf. Delivery

Data Warehouse Project Different From OLTP System Project Data Acquisition Data Storage Inf. Delivery Large Number of sources Many disparate sources Storage of large data volumes Rapid growth Several user types Queries stretched to limits Need for parallel processing Multiple query types Different computing platforms Web-enabled Data storage in staging area Outside sources Multidimensional analysis Multiple index types High initial load Ongoing data feeds Several index files OLAP functionality Metadata management

Data Warehouse Project Different From OLTP System Project Data Acquisition Data Storage Inf. Delivery

Data Warehouse Project Different From OLTP System Project Data Acquisition Data Storage Inf. Delivery Data replication considerations Difficult data integration Storage of newer data types Interfaces to DSS applications Archival of old data Feed into data mining Complex data transformations Data cleaning Compatibility with tools Multi-vendor tools RDBMS & MDDBMS

Major Deployment Activities Complete User Acceptance Finish final testing of all aspects of user

Major Deployment Activities Complete User Acceptance Finish final testing of all aspects of user interface including system performance. u Perform Initial Loads Load dimension tables followed by the fact tables. Create aggregate tables. u

Major Deployment Activities (cont’d) Get User Desktops Ready Install the needed desktop tools. Test

Major Deployment Activities (cont’d) Get User Desktops Ready Install the needed desktop tools. Test each client machine. u Complete Initial User Training Train the users on data warehouse concepts, relevant contents, and data access tools. u

Major Deployment Activities (cont’d) Institute Initial user Support Set up support to assist the

Major Deployment Activities (cont’d) Institute Initial user Support Set up support to assist the users in basic usage, answer questions, and hold hands. u Deploy in stages Divide the deployment into manageable stages in agreement with users. u

Deploy in Stages Top-down approach Deploy the overall enterprise data warehouse (E-R model) followed

Deploy in Stages Top-down approach Deploy the overall enterprise data warehouse (E-R model) followed by the dependent data marts, one by one. u Bottom-up approach Gather departmental requirements, plan and deploy the independent data marts, one by one. u Practical approach Deploy the subject data marts (dimensional model), one by one, with fully confirmed dimensions and facts, according to preplanned sequence. u

Considerations for A Pilot Types of pilot deployment: Proof-of Technology: Intended only to prove

Considerations for A Pilot Types of pilot deployment: Proof-of Technology: Intended only to prove new technology for IT. u Comprehensive Test: Only intended for IT to test all infrastructure/architecture. u Proof-of-concept: Small-scale, works with limited data, not suitable for integration u

Considerations for A Pilot (cont’d) u User tool appreciation: Only intended for users to

Considerations for A Pilot (cont’d) u User tool appreciation: Only intended for users to test and become familiar with tools. Broad Business: Early deliverable with broader scope, may be integrated. u Expandable Seed: Manageable and simple, but designed for integration. u

Tugas u Jelaskan langkah-langkah yang harus dilakukan dalam membangun datawarehouse? u Mengapa aspek studi

Tugas u Jelaskan langkah-langkah yang harus dilakukan dalam membangun datawarehouse? u Mengapa aspek studi kelayakan begitu penting dalam pembangunan datawarehouse? u Apa yang anda lakukan bila memiliki staff / rekan kerja yang tidak bisa diajak kerja sama dalam membangun datawarehouse?