Columbia Generating Station INPO AFI Recovery Dave Weber

Columbia Generating Station INPO AFI Recovery Dave Weber, Energy Northwest CMBG June, 2015

CGS - INPO Area For Improvement (AFIs) September 2014 ÷ AFI CM. 2. 1 - Design Calculation Backlog and Quality ÷ AFI CM. 3. 1 - Engineering Service Provider Design Quality 2

AFI CM. 2. 1 - Design Calculation Backlog and Quality

CGS - INPO Area For Improvement (AFI) September 2014 ÷ AFI CM. 2. 1 - Design Calculation Backlog and Quality. Some plant critical design bases calculations have not been maintained and some are of poor quality. This has resulted in some design bases without updated supporting calculations and technical evaluations, a large backlog of revisions to lowvoltage calculations, and documentation control issues with calculations. Contributing, design basis calculation update standards are not reinforced. 4

INPO 12 -013 Performance Objectives and Criteria OPERATIONAL CONFIGURATION CONTROL ÷ PERFORMANCE OBJECTIVE (CM. 2) - Plant operation, maintenance, and testing activities are conducted in a manner consistent with the licensing and design bases and maintain configuration control. ÷ CRITERIA - 1. Conditions that could result in structures, systems, or components being outside of operating and design requirements are evaluated and resolved on a schedule commensurate with the importance to safety and reliability. Special consideration is given to conditions that may challenge design bases or safety analyses. 5

Background History ÷ January 2014 ◘ Identified Design Calculations and > 5 Calculation Modification Records (CMRs) as a negative trend ◘ Created Excellence Plan Focus Area, “Calculation Backlog Improvement” ÷ January – September 2014 ◘ Worked on calculation backlog as level of effort ÷ July 2014 ◘ Created Calculation Backlog Project ÷ September 2014 ◘ AFI in CM. 2 -1, Design Calculation Backlog and Quality ◘ CR 315070 Severity Level A (RCE) on 9/27/2014 6

Root and Contributing Causes ÷ Root Cause ◘ RC 1: Design Engineering did not recognize the vulnerability to configuration management with regards to calculations. ÷ The AFI is based on three aspects: ◘ Four legacy calculation discrepancies were identified during the 2013 CDBI Inspection ◘ The design calculation backlog is higher than the industry standard ◘ Approximately 150 calculations were reported during the plant evaluation as missing, illegible, or incomplete. 7

Root and Contributing Causes ÷ Aspect one: Four legacy issues involving plant design documents without supporting calculations or technical evaluations. These issues were not self -identified, not corrected, and therefore challenge the design organization when addressing nonconforming conditions. ◘ CR 292758 – Independence requirements IAW industry standards (IE/Non-1 E Loads) ◘ CR 292764 – No technical basis for turbulent mixing of diesel combustion air ◘ CR 292760 – Operation of under voltage relays did not consider all design inputs ◘ CR 292762 – Emergency core cooling system pumps’ limits were not included in emergency operating procedures. 8

Root and Contributing Causes ÷ Aspect two: Three critical electrical calculations have a Calculation Modification Request (CMR) backlog of greater than 100: ◘ Calculation E/I-02 -95 -01 Calculation for Overcurrent Protective Device Settings and Coordination for 480 V Distribution System (146 CMRs), ◘ Calculation 2. 06. 20 Calculation for Cable Ampacity Verification for Scheduled Power Cables (144 CMRs), ◘ Calculation E/I-02 -90 -01 Calculation for Low Voltage System Loading (102 CMRs). ◘ At the time of initial investigation, there were 37 calculations that had greater than 5 CMRs (efforts during this RCE reduced this number to 29), 30 calculations that have 5 CMRs, and approximately 3425 calculations that have one to three CMRs. This large backlog challenges the retrieval of the design basis of the affected systems when addressing nonconforming or degraded plant conditions or the development of design changes. 9

Root and Contributing Causes ÷ Aspect three: Approximately 150 design bases calculations were identified as either missing in their entirety (hard and microfiche copies), incomplete (missing pages, appendices, or attachments) or illegible. During the evaluation, the missing calculations were located: however, the issue with missing pages, appendices, or attachments remains. ÷ The plant evaluation team stated “Some plant critical design bases calculations have not been maintained and some are of poor quality. ” Based on the examples given and the Design Engineering Manager’s discussions with the evaluator it was determined that ‘poor quality’ relates to the incomplete or illegible calculations listed on the spreadsheet and not technical quality. Therefore the poor quality piece is addressed under documentation control issues. Additionally technical quality issues related to design basis documents is being addressed under Root Cause Evaluation, CR 315069. 10

Root and Contributing Causes ÷ The team identified that there has been a lack of priority applied to upkeep of the design bases calculations ÷ With emphasis placed on issuing design modifications and/or emergent plant needs, configuration management suffers with a culture that has justified the Calculation Modification Request (CMR) process as meeting minimum standards. 11

Corrective Actions ÷ Immediate and Interim Corrective Actions ◘ Revised calculation procedure to clarify timeliness requirements of revising a calculation ◘ Developed a work down curve for the outstanding calculation backlog ÷ RC 1/CC 2/CC 3 corrective actions ◘ Dedicate Design Engineering resources that are focused on completing calculation updates ◘ Revised calculation procedure to clarify timeliness and corrective action requirements for revising a calculation 12

RCE Corrective Actions ÷ RC 1/CC 2/CC 3 corrective actions ◘ Create performance evaluation SMART goals for each supervisor ◘ Create calculation backlog project plan ◘ Create a calculation workdown curve performance indicator ◘ Create a schedule addendum that will include calculations and milestone due dates ◘ Drive the calculation backlog >5 CMRs to zero – due 12/31/2015 13

Effectiveness Review ÷ Perform three Semi-annual effectiveness reviews ◘ Review Supervisory observations, Condition reports and Performance indicators for validation – due 06/30/2016 Actions ÷ AE hired to revise calculations with greater than 5 plant implemented CMRs ◘ Weekly challenge calls with the AE to discuss calculation work focused on quality, cost and schedule 14

Design Calculation Backlog 15

Calculations > 5 CMRs PI 16

Sustainability ÷ Set standards and expectations ◘ Performance Appraisals ◘ Technical Conscience ÷ Procedure Improvements ◘ Time based calculation updates not to exceed 1 cycle ◘ Modifications developed to include calculation/ drawing update ◘ Condition reports generated when CMRs >5 ÷ Monitor Calculation Backlog with Performance Metrics ÷ Annual funding for Configuration Management maintenance ÷ Perform periodic effectiveness reviews 17

INPO Interviews - Drawing Backlog Background History ÷ January 2014 ◘ Drawing backlogs for those categorized as “Update Upon Request” identified as a negative trend ÷ January – September 2014 ◘ Backlog reduction efforts were worked as level of effort ÷ September 2014 ◘ Drawing Backlog entered into the Design Engineering Excellence Plan ◘ RCE for calculation backlog identified an extent of cause and condition issue regarding drawing backlog ◘ Condition Report generated 18

Actions Completed to Date ÷ Develop Project Plan for resolving the drawing backlog ÷ Funds obtained for FY 15 ÷ Project plan has been developed with schedule, resources and funding identified ÷ Work down curves have been established ÷ 7 contractors have been hired to work on drawing backlog ÷ Additional Auto. CAD and Raster Design licenses purchased to support new contract Drafters ÷ Improved Performance Indicator ÷ Funds obtained for FY 16 ÷ Monitor progress using work down curve and performance indicators metrics on a weekly and monthly basis 19

Current status ÷ As of May, 2015 ◘ 1886 of 10, 200 drawings have been drafted ◘ 1869 of 10, 200 drawings have been checked ◘ 1813 of 10, 200 drawings are approved ÷ Prioritization of System Drawings (Based on Maintenance Department) ◘ Heater Drain 442 total drawings 97% Comp ◘ Condensate 674 total drawings 95% Comp ◘ Bleed Steam 320 total drawings 83% Comp ◘ Seal Steam 111 total drawings 50% Comp ◘ Heater Vents 225 total drawings 29% Comp ◘ Misc Drains 175 total drawings 3% Comp 20

Drawing Backlog PI 21

AFI CM. 3. 1 – Engineering Service Provider Design Quality

CGS - INPO Area For Improvement (AFI) September 2014 ÷ AFI CM. 3. 1 - Engineering Service Provider Design Quality. From November 2012 to September 2014, engineering service provider design packages and products have contained technical errors such as use of incorrect or incomplete design inputs, specifying use of obsolete equipment or components, missing affected drawings and calculations, and frequently omitting critical construction and installation details. Contributing to this, engineering supervisors do not consistently hold engineering service providers accountable to department standards for quality and schedule adherence. 23

INPO 12 -013 Performance Objectives and Criteria DESIGN CHANGE PROCESSES ÷ PERFORMANCE OBJECTIVE (CM. 3) - Changes to plant configuration, design, and licensing bases are evaluated, controlled, tested, and implemented while consistency is maintained among the physical plant configuration, design and licensing requirements, and the documented plant configuration. ÷ CRITERIA - 1. The design authority is clearly identified and authorizes all plant design changes through a system of controls. Roles, responsibilities, and processes for developing, reviewing, and approving design and license bases changes are clearly defined and followed. 24

Background History ÷ 2005 - Decision to use AEs ÷ 2008 – AFI in CM. 3 -1, Weaknesses in oversight of external engineering products ÷ 2010 - Self Assessment, AE Oversight ÷ January 2014 ◘ Identified Engineering Change Product Quality as a negative trend ◘ Created Excellence Plan Focus Area, “Effectiveness of Design Engineering Changes” ÷ April 2014 - INPO IER L 1 -14 -20 ÷ July 2014 - Created Procedure Upgrade Project ÷ September 2014 ◘ AFI in CM. 3 -1, Engineering Service Provider Design Quality ◘ CR 315069 Severity Level A Root Cause (RCE) 9/27/14 25

Root and Contributing Causes ÷ Root Cause ◘ RC 1: Design Engineering organization does not consistently demonstrate ownership of design quality to provide early engagement and consistent, accountable resources for AE oversight. ÷ Contributing Causes ◘ CC 1: Engineering Management is not enforcing priorities to ensure the quality of design products prepared by the AEs. ◘ CC 2: AEs are not consistently holding themselves accountable for providing design products that meet EN standards. 26

Root and Contributing Causes ÷ In 2005, EN transitioned to the use of external AEs to supplement in-house engineering resources (not in same time zone, not on site) ÷ In October 2008, a plant evaluation determined that CGS had a weakness in the oversight of external engineering products (AR-CR 190612). ◘ This ACE resulted in multiple corrective actions that included revising the EN Project Manager qualification, enhancement of design procedures to clarify EN expectations for product in-progress reviews and condition reporting expectations for the AE, better defined roles and responsibilities in design engineering procedures and contracts, and implementation of a Comment Control Form formal communication between EN and AE firms ◘ Transition AE to on-site presence 27

Root and Contributing Causes ÷ In 2010, a Self-Assessment was completed to determine the effectiveness of the corrective actions from ACE 190612 (AR-SA 222381). The Self. Assessment determined that there were improvements made from implementing the administrative tools such as the Division of Responsibility form, Comment Control Form, and the Columbia Design Guidance Tool. However, the self-assessment also revealed there were still areas for improvement. The key findings determined that EN was still not holding the AEs accountable for product quality and EN had not effectively developed a resource loaded schedule to adequately develop design changes 28

Root and Contributing Causes ÷ In 2014, the first quarter Engineering Department Roll-Up Meeting (DRUM) report identified an adverse trend in the number of field changes. The report indicated the Design Engineering Excellence Plan would address the causes and corrective actions going forward. The Design Engineering Excellence Plan identified six focus areas for the department, one of which was Effectiveness of Design Engineering Changes, identifying that AE design products do not consistently meet EN engineering standards for effectiveness regarding quality, cost, and timeliness 29

Root and Contributing Causes ÷ In April 2014, a Level 1 event report was published, IER L 1 -14 -20, Integrated Risk – Healthy Technical Conscience, relating to integrated risk, technical conscience principles and engineering fundamentals. The report discusses an adverse trend in technical errors and risk management that have resulted in serious organizational and operational impacts. The report shares lessons learned regarding insufficient technical expertise, critical design requirements and loss of margin, and insights from engineering vendors. The recommendation for Engineering Vendor Oversight and Support focuses on adhering to sound project management fundamentals. The attributes of the recommendation include communicating achievable and understandable objectives, assessing technical and project management proficiencies, and ensuring prompt reporting of events during project execution. ◘ EN has approximately 50 assignments to address IER L 1 -14 -20; 24 are complete and the others are scheduled to be completed by 9/25/2015 30

Root and Contributing Causes ÷ EN uses AEs to develop engineering products such as design changes, calculations, and drawings for CGS. The AEs work under a Basic Ordering Agreement (BOA) contract with individual contract releases for each specific project scope. The AEs are expected to develop engineering products in accordance with EN procedures with oversight from EN project managers and design engineers. Design Engineering reviews and evaluates (also known as an owner acceptance review) in-progress and final AE engineering products using score sheets in accordance with DES-3 -3, Formal Design Reviews. Once the AEs have completed the engineering products, they are approved by EN and implemented in accordance with applicable procedures. ◘ Any issues or errors that are discovered after EN’s approval are generally resolved by Design Engineering as a design change revision or field change (not the AE). Each revision and field change is designated with one of the following cause codes for trending purposes; planned, scoping, material substitution, design error, construction convenience, installation error, administrative only, and user defined 31

Root and Contributing Causes ÷ Specific Mods ◘ Multiple design change revisions and field changes related to the Radwaste Building Exhaust Air (WEA) and Turbine Building Exhaust Air (TEA) modifications, § Installation difficulties and omissions, translation errors within design documents, Old mod sat on the shelf ◘ Fast track modifications for Motor Control Center (MCC) Buckets and 50 DHP-350 3000 A (N/S) Breakers resulting in critical path delays 32

Corrective Actions ÷ Immediate and Interim Corrective Actions ◘ Briefed design engineers and AEs on gap and importance to EN ◘ Interim corrective action – Perform self-assessment of R 22 designs for technical errors ÷ RC 1/CC 2 Corrective Actions ◘ Change Roles Responsibilities and Accountabilities § Assign a single Energy Northwest (EN) engineer for each AE design change to follow projects from cradle to grave § Modify design engineering and project management procedures to clarify roles and responsibilities 33

RCE Corrective Actions ÷ RC 1/CC 2 corrective actions ◘ Ensure future Basic Ordering Agreements (BOA)/ Contract Releases for the AEs reflect EN standards and quality, schedule, and cost ◘ Engineering VP to provide briefing to Engineering Department to reinforce priorities for Design Engineering and emergent plant response expectations (after procedures revised) ◘ Incorporate a SMART goal in Performance Appraisals for Design Engineering supervision ◘ Onsite Engineering Service Providers to develop and implement a product quality recovery plan that meet EN standards 34

Effectiveness Review ÷ Perform an interim corrective action effectiveness review following R 22 refueling outage. ◘ Review Engineering Management focused observations, corrective action program, performance indicators, product quality score sheets, and Comment Control Forms (CCFs) - 06/30/2016 ÷ Perform a final corrective action effectiveness review. ◘ Review performance indicators, quality score sheets for design changes, and field changes for implemented design changes - 12/31/2017 35

Procedure Upgrade Project ÷ Procedures Upgrade Project ◘ Three month effort to make large scale improvements to over 30 Design Engineering Procedures and 20 forms ÷ Significant Upgrades/ Improvements ◘ Implemented a 30% Conceptual Design Review Board ◘ Implemented required quorums for all stakeholders ◘ Implemented scorecards following both the Design Preparation (post 100%) and the modification implementation ◘ Added performance at risk options based on product quality, timeliness, and cost behaviors ◘ Addressed Enterprise Risk ◘ Developed an engineering walkdown procedure ÷ Training on these procedure changes is complete 36

Completed Excellence Plan Actions ÷ Obtained FY 15 funding to allow early start for high impact designs in first quarter 2015 ÷ Bi-weekly Design Engineering schedule review meeting ÷ Weekly meetings between DE manager and AE’s managers to discuss performance ÷ Developed PI metrics on ECs ◘ Engineering Field Changes (including design errors) ◘ Root Cause Evaluation reviewed and approved on 01/06/2015 37

Actions Remaining ÷ RCE (CR 315069) ◘ Implement corrective actions ◘ Perform effectiveness reviews ÷ Monitor progress through PI Metrics ◘ Engineering Change Field Changes (including design errors) ◘ Product Quality Grading scores ÷ Excellence Plan ◘ Improve accountability through contract management ◘ Perform snapshot assessment to ensure success 38

INPO 90 -009, Guidelines for the Conduct of Design Engineering, Rev 3 ÷ Field change requests are monitored and causes for field changes are identified analyzed to provide a basis for improving the design change process. Implementing appropriate corrective actions should decrease the number of field changes needed for future modifications. Experience has shown that large numbers of field changes are frequently caused by inadequate walkdowns, inaccurate or incomplete as-built drawings, or incomplete designs resulting from tight scheduling or insufficient planning. ÷ Field change requests for ease of construction (Construction Convenience) should be minimized, because the cost of analysis, documentation (e. g. rework to the EC package, WO task package, QC reviews, etc. ), and work delays related to the field change often will exceed the expected benefit 39

USA KPI Cause Categories USA Category 1: Engineering Errors - field changes required to correct design/engineering errors which could have (1) prevented installation, (2) resulted in improper installation, (3) resulted in undesirable operation of equipment, or (4) resulted in an unsafe condition to personnel of plant equipment. USA Category 2: Modification Acceptance Test Errors - field changes that involve testing methodology or the adequacy of testing. Included are inadequate or excessive testing requirements, inadequate interpretations of testing results, changes to testing requirements, and changes to hold points. CGS - Cause Code Definitions Construction Convenience Category Code 1 CC 01 40 Description Construction Convenience - Installation cost savings. The craft provided a recommended installation solution that deviates from the design but the recommendation will save labor, time and/or material. 1 CC 02 Construction Convenience - Dose savings. Reconfigure the original design to reduce radiological exposure. 1 CC 03 1 CIE 01 Construction Convenience - Ease of maintenance. Reconfigure the original design to allow future Maintenance activities to be performed more efficiently. Craft Installation Error - EC Rev is provided to revise the design rather than the cost for reworking the installation.

CGS - Cause Code Definitions Design Error 41 1 DE 01 Design Error - Functional design error such as the Top Tier drawing as designed inhibits properation or a Calculation change is needed based on Vendor info or unforeseen needs. Design Error - Physical design error such as the physical design documents not in agreement with the Functional Design Documents or inadequate tolerances, rerouting or mounting changes. 1 DE 02 1 DE 03 Design Error - Inadequate walkdown during the design phase caused re-design or missing enough omitted documents that could cause an error likely situation during the planning of Work Order Tasks prior to implementation. 1 DE 04 2 DE 05 1 DE 06 Design Error - Inadequate design reviews or verification during development of the design caused an EC revision due to lack of rigor in preparation of the design package by responsible engineer, review or input by participating SMEs, or overall design verification. Design Error - Inadequate Modification Test Plan. The Test Plan did not address all elements of the design change necessary to validate that SSCs as modified will perform required design functions as intended and can be returned to unrestricted service and/or the Modification Test Plan did not include necessary provisions for special testing of the design change or special test procedures could be developed for special test evolutions Design Error - Inadequate Modification Risk Assessment. The Modification Risk Assessment did not identify all the risks and mitigation strategies that caused unforeseen human performance, process and station implementation risks. 1 DE 07 Design Error - Inadequate identification and control of Design Inputs. Design inputs from sources such as design basis, regulatory requirements, codes and standards, etc, were lacking to address the technical considerations that bound the design. 2 DE 08 Design Error - Inadequate Design Evaluation and Safety Assessment – The present configuration before the modification was not fully addressed or understood which caused errors not considered in the Failure Modes and Effects Analysis, component or system functions. The safety assessment neglected to include certain aspects of the impact to licensing basis, FSAR/Tech Specs or overall impact to plant safety.

CGS - Cause Code Definitions Material Substitution/Scoping 42 1 1 1 MS 01 Material Substitution - Changes in material/parts due to availability or cost. MS 02 Material Substitution - Changes in vendor manufacturing and or configuration. SCOPING 01 SCOPING - Inadequate walkdown during the design phase resulted in a scope change during implementation. 1 SCOPING 02 SCOPING - Lack of design team member support from interfacing departments (e. g. Craft, Ops, RP, Chemistry, etc. ) contributed to changing the scope of the design. 1 SCOPING 03 SCOPING - Implementation is in-progress but will not be implemented by its expiration date and requires re-evaluation.

Functional/Physical Definition ÷ Functional Design Documents ‑ Documents which portray design criteria and schematic arrangements for the purpose of establishing and conveying the operational (functional) aspects of Structures, Systems, and Components. Examples: Elementary Diagrams, Flow Diagrams, Logic Diagrams, Safety Related Equipment Lists, Design and Safety Calculations, One‑line Diagrams, Engineering Standards that implement Criteria, and Procurement Specifications. ÷ Physical Design Documents ‑ Documents which convey physical layout (configuration, organization of elements) and arrangement requirements. Physical Design Documents interpret the functional requirements given in the Functional Design Documents into configuration requirements of Structures, Systems and Components. Examples: Connection Diagrams, Piping ISOs, Mechanical Layout Drawings, Cable Schedules, Raceway Plan Drawings. 43

EC Field Changes PI (Month to Month) 44

EC Field Changes PI (Month to Month) 45

EC Field Changes PI (6 Months Rolling) 46

Sustainability ÷ Set standards and expectations ◘ Performance Appraisals ◘ Technical Conscience ÷ Procedure Improvements ◘ Conceptual Design Review Board implementation ◘ Scorecard Improvements establishing performance at risk based on quality, cost and schedule performance ◘ Design Walkdown implementation ÷ Monitor Field Changes through Performance Metrics ÷ Perform periodic effectiveness reviews (self-assessments) 47

Excellence Plan ÷ Calculation Improvement ◘ No calculations >5 CMR’s 12/31/2015 ÷ Margin Management ◘ Technical Conscience Self Assessment ÷ Regulatory Margin ◘ CDBI Preparation/ Downgraded systems ÷ Effectiveness of Design Engineering Changes ◘ Procedures upgraded/ Self Assessments on schedule ÷ Design Engineering Backlog ◘ Drawing backlog Reduction 08/31/2017 48

49