Application Change Management Best Practices Siebel IT ISACA
Application Change Management Best Practices @ Siebel IT ISACA San Francisco Chapter Nov 17 th, 2005 Salil Durani Group Director, IT Applications Siebel Systems
Application Change Management Scope • Solution Development Life Cycle (SDLC) • Test Environment Deployment • Production Environment Deployment © 2005 Siebel Systems, Inc. Confidential. 2
Application Change Management En vir Tes on t m en Control t p Te lo ve Support © 2005 Siebel Systems, Inc. Confidential. 3 duc Pro y Dep lo ts men u i re Req esi g n &D Document Release Communicate Management Train Control Change Management tion st De
Siebel (Quality & Release) Application Change Management ARP Application Release Package Control Mgmt Create Patch Vendor IT Development User / IT Program Manager Production Approval Approved Requests IT Apps/BU Technical Design Development Complete Feature Requests Log Defect Release Operations © 2005 Siebel Systems, Inc. Confidential. Monitor Release Patch Request Mgmt IT Applications Verify Fix Migrate To Test Application Change Item Yes P 0 Criteria Release Criteria No No Application Release Package 4 Generate ARP Migrate To Production
60 -Day Drum Beat © 2005 Siebel Systems, Inc. Confidential. 5
Types of Application Change § Siebel Upgrade § Upgrade of the base Siebel Application platform § 60 -day IT Release § Functionality Release that covers prioritized, approved features and pending defects in all applications § Service Pack § 3 weeks after a 60 -Day Release § High priority/low impact defects § Production (P 0) Patch § P 0 Issue is defined as a business critical issue in production will no “acceptable” workaround § Business Justification and Impact should be approved by the Business Dept VP or BU Leader © 2005 Siebel Systems, Inc. Confidential. 6
Planned Release Dates Published © 2005 Siebel Systems, Inc. Confidential. 7
Change Management op Te el v De Support © 2005 Siebel Systems, Inc. Confidential. 8 Dep ts men u i re Req Desi g n & Release Document Management Communicate Train Change Control Management loy st
Requirements, Development Review All BU Projects IT Apps Mgmt BU Mgmt User Approved Project List Prioritized List Business Requirements IT Program Manager Confirm Project Timeline Create Functional Specs Tech Review IT Dev IT Applications Operations © 2005 Siebel Systems, Inc. Confidential. 9 Assign Level Of Effort Complete Dev Unit Test
Requirements & Design Phase § Business Requirements Document (BRD) § Functional Specification Document (FSD) § FSD review & BU signoff to meet deadline to be a release candidate § Technical Review § Technical feasibility of the FSD § Review States (Tags): § Every type of IT review is captured with an approval tag on Change Request (CR). § § FSD Review = PM Management Functional Review Dev Review = Development review Arch Review = Interfaces, Data Migration & Data Security App. Ops Review = Server/Component Set-Up, Process Scheduling, Environment Changes © 2005 Siebel Systems, Inc. Confidential. 10
Enhancement States § Approved § Requirements met all deadlines, CR/FSD PM reviewed, tech-reviewed and in scope for the release § Standby § Requirements met all deadlines, CR/FSD PM reviewed, tech-reviewed by all dev leads, low-risk/low-scope § Dev resources are on standby § Discovery § Requires further technical evaluation, high-priority project for the business § Dev resources are committed to detailed technical design and discovery efforts § Automatically moves to Standby status once discovery is complete © 2005 Siebel Systems, Inc. Confidential. 11
Test Phase op Dep ts men u i re Req Desi g n & Release Document Management Communicate Train Change Control Management loy Te st el v De Support Control © 2005 Siebel Systems, Inc. Confidential. 12
Test Phase Vendor Create Patch IT Development Fix Defects User / IT Program Manager Quality Update Test Plans Execute Test Plans Verify Fix Log Defect Update Test Status Load Test Plans Release Mgmt Patch Request Migrate To Test IT Applications Operations Siebel (Quality & Release) Control © 2005 Siebel Systems, Inc. Confidential. 13 Monitor Release Application Change Item
Quality Balance Business Requirements User Acceptance Test Functional Specs Business Process Test Logical Design Basic Integration Test Physical Design Unit Test Component Design Technical Walkthrough Configuration/Programming © 2005 Siebel Systems, Inc. Confidential. 14
Test Environments § Test Environment Characteristics: § Contains every hardware / software component that is present in production § Test data is a scrambled copy of production data § Restricted environments for non-scrambled testing § All integration and interface components present § Back-Office test systems included § Completely managed by IT Applications Operations team © 2005 Siebel Systems, Inc. Confidential. 15
Environment Management Environment Description Users Production All enterprise applications and interfaces to third party products All employees worldwide Development Used for development All IT developers Unit Test Scrambled copy of production All internal Developers User Acceptance Test (UAT) Scrambled copy of Production Anyone involved in testing User Test Scrambled Copy of Production • Users needing exclusive access • Production support Staging Used for content publishing IT Apps Operations Demo + Training Copy of UAT • Executives showcase products to external customers • End User Training PSR Testing Performance, Scalability & Reliability Used by IT PSR Team Read-Only Copy of Production used during upgrade All employees worldwide © 2005 Siebel Systems, Inc. Confidential. 16
Test Management using Siebel Analytics © 2005 Siebel Systems, Inc. Confidential. 17
Go-Live Criteria - Functional § § § § All Test Plans executed No Regressions Zero test plans in “Blocked” or “Defective” state No Open P 0 Defects § Business Unit Sign Offs © 2005 Siebel Systems, Inc. Confidential. 18
Go-Live Criteria – System & Operations § Performance Scalability Reliability (PSR) § PSR testing complete § Performance § Client Coverage KPI § Server Components : No degradation from previous release § Scalability § No degradation of foot-print from previous release § Reliability § Uninterrupted load tests for at least 48 hours with no crashes § No degradation on memory leaks § Systems & Operations § Sign Off from IT Applications Operations © 2005 Siebel Systems, Inc. Confidential. 19
Siebel Release & Analytics Components To Manage Test Plans © 2005 Siebel Systems, Inc. Confidential. 20
Deploy Phase p lo ve Te Support © 2005 Siebel Systems, Inc. Confidential. 21 Dep lo ts men u i re Req Desi g n & Release Document Management Communicate Train Change Control Management y st De
Siebel (Quality & Release) Deploy - Overview ARP Application Release Package Control Production Approval IT Apps Mgmt Production Approval IT App Operations IT Development User / BU Signoff IT Program Manager Yes Release Monitor Release Mgmt Verify Release Criteria Un-planned release Yes No Go-no-go No IT Applications Operations Migrate To Test © 2005 Siebel Systems, Inc. Confidential. Application Change Item Application Release Package Operations Signoff 22 Generate PRP ARP Migrate To Production
Production Deployment § Application Release Package (ARP) § ‘close-verified’ objects that have been approved for production roll-out are extracted into a ARP § ARP consists of Change Requests and Service Requests only. (no email, voicemail or word of mouth) § Production Deployment § IT Applications Operations will deploy only those objects into production that are listed in the ARP § A brief final validation test is conducted by IT PM and key business users § Positive validation test is a go-ahead for final go-live © 2005 Siebel Systems, Inc. Confidential. 23
Production Deployment § Rollback § In an event if serious post production issues/defects are encountered, the Release Council can decide to roll back the upgrade § Redundant business continuity enterprise stack is used for the rollback process to minimize downtime © 2005 Siebel Systems, Inc. Confidential. 24
Support Phase p lo ve Te Support © 2005 Siebel Systems, Inc. Confidential. 25 Dep lo ts men u i re Req Desi g n & Release Document Management Communicate Train Change Control Management y st De
Support Verify Problems IT PM Users Log Defects Log Problems Customer Monitor Problems Care Log Defects Quality Release Mgmt IT Applications Operations Analyze Problems Request Patches Monitor Operations Siebel (Quality & Release) Control © 2005 Siebel Systems, Inc. Confidential. 26 Monitor Performance Daily Report Card
Post Upgrade – Service Requests Upgrade © 2005 Siebel Systems, Inc. Confidential. 27
Defect Root Cause Analysis § Root cause of every defect is captured in CR § § § Product issues Configuration mistakes Incomplete or inaccurate Functional Specification Incomplete or inaccurate Business Requirements Incomplete testing Process and communication breakdowns § Root Cause Analysis § Process to evaluate the root cause for each defect reported § Intended to measure quality of unit, system and Acceptance testing § Used to identify process weaknesses, increase accountability and drive process improvements through all phases of the Apps Release Process © 2005 Siebel Systems, Inc. Confidential. 28
Root Cause Analysis © 2005 Siebel Systems, Inc. Confidential. 29
Test/Production Deployment En vir Tes on t m en op t Te el v De tion duc Pro Dep lo ts men u i re Req esi g n &D Release Document Management Communicate Train Change Control Management y st Control Support © 2005 Siebel Systems, Inc. Confidential. 30
Change Control Board Users Alert subscription Communications (email, portal) Operations Customer Care Network © 2005 Siebel Systems, Inc. Confidential. Data Center Applications Change Control Desktop Support IT Applications Operations Telecom Software Development Life Cycle 31 Approvals Notifications Server Infrastructure Security Rejections IT Change Control Board (ICCB)
High Priority - Defect Fix Process § P 0 Defect Fix (Patch) Process § for business critical defects to be deployed to production outside of the release cycle § Analysis § PM and Business Owner discuss business criticality, workaround, proximity to upcoming release and P 0 criteria § Business Owner obtains organization VP approval § Change Request & Approval § PM logs Change Request with the P 0 Business Justification § Release Manager (RM) obtains IT Apps approval for off-cycle release § Deployment § § Dev fixes in break-fix dev repository Ops moves from dev to UAT Business user tests and close-verifies CR Upon verification of IT Apps approval and testing, Release Manager follows the production release process to deploy the fix at the next maintenance window © 2005 Siebel Systems, Inc. Confidential. 32
Production Change Windows § Fridays 7: 00 pm PST to 9: 00 pm PST § IT 60 Day Release § Service Pack § Wednesdays & Fridays 7: 00 pm PST to 9: 00 pm PST § System maintenance § P 0 priority/approved changes § Saturdays (with business approval) § Major platform upgrades § Extended Maintenance § Business Continuity exercise © 2005 Siebel Systems, Inc. Confidential. 33
Production Blackout Window § Why § Compliance § Minimize risk during critical quarter-end activity § When § Starts 2 weeks before end of quarter § Ends 2 weeks after end of quarter § Changes permitted with approval § Critical system changes § Changes that impact end of quarter activity § Legal or statutory compliance related changes § Change Process § Approval from BU head and IT Head for low risk changes © 2005 Siebel Systems, Inc. Confidential. 34
ts n e m n e ir sig u q De e R & Control Solution Development Life Cycle (SDLC) ve lo p Drive Maintain Continuous World. Business Class IT Process Document Infrastructure Improvements Release ol tr Con p o rt t Tes Con trol Sup Communicate Management Train Provide Best-in-Class Control Customer Services l 35 ro nt Co C o n tro l Deploy © 2005 Siebel Systems, Inc. Confidential. De
Release Process - Control Documents § Business Requirements Document (BRD) Deadline § BRD Acceptance § Functional Specification Document (FSD) Deadline § Release Definition § Development § Mid-Cycle Review § User Acceptance Testing (UAT) § Go-Live § Post-Production Support § Root Cause Analysis © 2005 Siebel Systems, Inc. Confidential. 36
Release Definition § Change Requests (CRs) created for new enhancements § Business Requirements Document (BRD) reviewed by PM § approval tag added to indicate acceptance § IT PM to create Functional Specification Document (FSD) § All BRDs are stacked and ranked § Development teams to provide LOE for each enhancement § BRDs attached to CRs © 2005 Siebel Systems, Inc. Confidential. 37
Application Change Management - Recap En vir Tes on t m en Control t p Te o el Support © 2005 Siebel Systems, Inc. Confidential. 38 duc Pro y Dep lo ts men u i re Req Desi g n & Document Release Communicate Management Train Control Change Management tion st v De
Sarbanes-Oxley Information Technology Focus Areas © 2005 Siebel Systems, Inc. Confidential. 39
Change Management Controls (CM) – Controls provide reasonable assurance that all systems and system changes are appropriately requested, approved, tested, and validated by authorized personnel prior to implementation to the production environment. © 2005 Siebel Systems, Inc. Confidential. 40
Siebel App. Enables Change Management § Every object implemented as an Application object § Change Request (CR), Service Request (SR), Release, Test Plan, Test Pass, monitoring dashboards, Application Change Item, Application Release Package § Every state in Change Management is implemented using an appropriate Tag in Siebel § Appropriate approval states defined § All documents are attached to Siebel Objects § Complete audit trail is maintained § Access to Tags and Objects is restricted to authorized people only § Periodic audit of process conducted § Detailed Post Mortem exercise § All Quality, Release, Project metrics available in real-time Siebel Analytics Dashboards © 2005 Siebel Systems, Inc. Confidential. 41
Questions? Salil Durani salil. durani@siebel. com © 2005 Siebel Systems, Inc. Confidential. 42
- Slides: 42