EP CH Managing Outsourcing Contracts for System Administration

EP CH Managing Outsourcing Contracts for System Administration 97 Frédéric Hemmer CERN 1

Capacity explosion 14000 12000 10000 9000 8000 7000 6000 l 8000 4000 6000 3000 4000 2000 1997 (est. ) 1996 1995 1993 1992 1991 1990 1989 1988 n n l Cheaper and more affordable computing cycles Large data stores Services dedicated to experiments Large increase of computer nodes (20 in 1990, 300 now) Over 1000 disks îneed for more manpower to perform the system administration 97 Year 1994 1000 2000 Physics Data Processing capacity Benefits n EP 5000 Disk Storage - Giga. Bytes CH CPU Capacity - CERN Units l l l 2

Manpower evolution 25 l 20 15 Physics Data Processing manpower n n n EP n n l 10 system administration user support general administration planning etc. Total decreased by 25 % Sys. Admin. decreased by 40 % 97 Year 1996 1995 1994 1993 1992 1991 5 1997 (est. ) l 1990 man-years per year 30 Systems Administration CH Total 3

Outsourcing l Very wide definition l l l 97 l EP l CH l Total outsourcing of a service Total transfer of ownership (equipment, infrastructure and staff) Long term mega-deals Temporary labour contracts Selective sourcing (scope, size and duration) different activities lead to different management attention create constant competition more flexibility to adapt to changes îWe try to follow this approach 4

“Best Practices in Information Technology Sourcing” CH M. C. Lacity, L. P. Wilcocks EP Executive Research Briefing, Oxford Institute of Information Management, Templeton College, University of Oxford 1996 97 5

Our approach l l 97 l EP l CH l Short term contracts System administration only CERN retains planning, HW purchase/maintenance, user support, choice of technology and architecture SLA’s defining quality measures At least two contracts (offset in time) îallows for regular retendering (competition) 6

Our experience (1) Initially tried to use temporary labour l l Difficulty in finding experience candidates CERN policy evolved to service contracts EP l CH l Difficulty to express specifications of sys. admin. l l l Outsourced the task of writing the specifications Long iterative process Vendor and customer have different goals ! Contract covering 28 HP and 27 DEC starting Jan. 96 97 l 7

Our experience (2) l Documents l l System Administration tasks installation, configuration changes, reactive and proactive problem management, development of service quality and performance measurements 97 l l EP l Service Definition Document (Scope, communication and escalation procedures) Service Level Agreement (responsibilities, tasks and performance metrics) Specific Sys. Admin. documents for each system CH l Aggressive timescale l 2 months for sys. admin. guides, take over one month later 8

Our experience (3) l May review l l EP l Objectives met No degradation in service Documentation very good Improvements needed l Communication needs to be more formal l î Mastersoft’s OSM CH l l l 97 • Inventory, SLA, communication, reporting, quality measurement System managers showed reluctance to give away “their” systems (lack of confidence, faster to fix it now). Not solved yet. Decided to expand l l 24/hours day later expand to all DEC, HP, SUN and SGI equipment 9

Potential Problem areas l Call Management System Installed in Autumn, not used before March • No PC for our OPS manager • Lack of momentum in contract follow-up (increased responsibilities of the team leader, staffing problems) î no quality or performance measures l EP CH l Coverage and end of 96 shutdown l 97 Contract excluded this period l Period of intense activity as the computers are down îreturn of control to CERN staff l Contractor’s staff stability l î unable to test whether procedures and documentation are enough 10

Conclusions l Globally positive l l EP l Procedures well documented Very few problems/mistakes Not too many delays Clear cost attributable to system administration Scaling up no longer limited by CERN staffing level CH l Improvement needed l 97 l More involvement from CERN needed in contract follow-up The performance metrics need to be gathered analysed î We hope to assess performance by the summer and be ready for tendering at the end of the year. 11