Parallel Ops GoLive Checkpoint Weekly Update Wednesday Meeting

  • Slides: 62
Download presentation
Parallel Ops / Go-Live Checkpoint Weekly Update – Wednesday Meeting January 8, 2014

Parallel Ops / Go-Live Checkpoint Weekly Update – Wednesday Meeting January 8, 2014

Agenda • Markets System Updates 10: 00 – 10: 10 A • • •

Agenda • Markets System Updates 10: 00 – 10: 10 A • • • 1. 9 Installation Update, 10: 00 – 10: 10 A IDT Debrief 10: 10 – 10: 35 A • Updated IDT Schedule and Conditions, 10: 10 – 10: 15 A • IDT Report Review: IDT 12/17 and IDT 12/19, 10: 15 – 10: 30 A • IDT 1/07 Preliminary Debrief, 10: 30 – 10: 35 A Analysis and Announcements Updates 10: 30 – 11: 10 A • No EIS Suspension, 10: 35 – 10: 45 A • Ramped Setpoints, 10: 45 – 11: 05 A • Decommission of Daily Commitment Report, 11: 05 – 11: 10 A Look-Ahead 11: 10 – 11: 40 A • Upcoming IDTs and Scenarios, 11: 10 – 11: 30 A • Weekly Updates: Release Scorecard, Migration Report, Known Issues Log, 11: 30 – 11: 45 A Settlements Update 11: 45 – 12: 00 P Readiness Update 12: 00 – 12: 30 P 2

Markets Data Issues – 12/31/2013 • • Situation – Markets System includes database package

Markets Data Issues – 12/31/2013 • • Situation – Markets System includes database package used to generate test data in lower environments; package was mistakenly enabled during release 1. 9 installation – Package execution (early morning 12/31) caused overwrite of MP data with test data MP Impact – Data resubmission required § All Demand Bids for 1/1/2014 forward § All Resource Day-Ahead, RTBM, and Mitigation Offers § All Virtual Offers and Bids Annette Holbert 3

Markets Data Issues – 12/31/2013 (cont. ) • Other Impacts – SPP restored 12/30

Markets Data Issues – 12/31/2013 (cont. ) • Other Impacts – SPP restored 12/30 MP data to allow resubmission (afternoon 12/31) – Readiness Metrics for 12/31/2013, 1/1 and 1/2/2014 to be assessed and possibly removed for overall reporting consideration – Day-Ahead solution for 1/1 and 1/2/2014 not valid • Root Cause: Human Error • Lessons Learned – Additional installation validations put in place for future installs – Working with vendor to ensure release notes call out significant changes such as the package change that ultimately led to the issue Annette Holbert 4

IDT Schedule and Conditions IDT Date 1/9/14 1/10/14 IDT Times Conditions Notes Canceled 6:

IDT Schedule and Conditions IDT Date 1/9/14 1/10/14 IDT Times Conditions Notes Canceled 6: 30 A-4: 30 P 8 hr IDT begins at 7: 30 A Simple Failover will be a standard model upload failover and primary systems will not be down for but a few minutes. The secondary system will be up and providing setpoints along with the XML signals. ICCP Failover SPP has requested that an MP perform an ICCP failover in the middle of the January 10 th IDT. TARRF 1/21/14 4: 30 P-8: 30 P 2 hr IDT begins at 5: 30 P 1/27/14 4: 30 P-10: 30 P 2/25/14 8: 30 A-12: 30 P 2/28/14 7: 00 P Tieline Drop SPP loses select tie-line measurements. SPP will simulate the loss of real-time data with significant tie-line data providers. MPs will not need to take any action; this is an internal process for our operators. EDR Testing with specific MPs will occur during this IDT 4 hr IDT begins at 5: 30 P 2 hr IDT begins at 9: 30 A 4 hr IDT begins at 8: 00 P Cutover for March 1 Go-Live Ginny Watson 5

IDT 12/17 Report IDT reports posted to spp. org here 7 calendar days after

IDT 12/17 Report IDT reports posted to spp. org here 7 calendar days after each IDT spp. org > Integrated Marketplace > Market Trials > Integrated Deployment Test IDT 12/19 Report IDT reports posted to spp. org here 7 calendar days after each IDT spp. org > Integrated Marketplace > Market Trials > Integrated Deployment Test Brian Strickland 6

01 -07 -13 Deployment Test Debrief • Overall the Tenth Deployment test was a

01 -07 -13 Deployment Test Debrief • Overall the Tenth Deployment test was a success – All BAs and all but 1 MP were on the call 60 min before IDT. Thank you! • Ramp Sharing on 50% for the Jan 7 IDT • Another smooth transition into test • 3 CPS 2 Violations • – Load Change – Control Mode – New Pseudo Tie IDT Condition: Regulation Pre-selection and Co-optimize Regulation Brian Strickland 7

EIS to remain Enabled during IDTs During one of the IDT wrap up calls

EIS to remain Enabled during IDTs During one of the IDT wrap up calls the idea to disable EIS market was shared with the participants. • The thought was if the price of EIS was not visible then units would only follow IM • Also investigate possibility to use proxy pricing instead of EIS As investigation into this as a option furthered unintended consequences were discovered so we will - Continue as originally defined and leave EIS data feeds enabled. See pages 51 -60 for additional detail. Casey Cathey 8

Units not following during IDTs During the first Integrated Deployment Tests SPP staff noticed

Units not following during IDTs During the first Integrated Deployment Tests SPP staff noticed a number of abnormalities with units following issued Setpoints a. Units not appearing to switch to IM control and follow EIS targets through test b. Units that followed IM Setpoints except when prices diverged at which point they followed EIS c. Units that follow in a delayed mode Casey Cathey 9

Complicated Issue SPP researched internally and reached out to participants in an effort to

Complicated Issue SPP researched internally and reached out to participants in an effort to more fully understand the underlying issues. SPP found that the reasons for not following were more complicated than we had initially assumed a. Price divergence was magnified by RTBM solutions that struggled to constrain binding flowgates. b. Ancillary Services were not matched between EIS and IM causing units to have large movements in the beginning of tests and across hours. c. Ramped Setpoint calculation. Casey Cathey 10

Issue with calculation of Ramped Setpoint Due to a number of factors, the Ramped

Issue with calculation of Ramped Setpoint Due to a number of factors, the Ramped Setpoint that SPP sends the each resource is causing it to be difficult to maintain ACE. Currently SPP is trying to do two things with one ICCP Point • Provide a value that represents the cycle by cycle rate limiting of the Setpoint instruction. This value was originally provided for use in compliance monitoring and NOT expected to be used for resource deployments. • Provide a value that can be used to smoothly deploy units between desired output levels. Casey Cathey 11

What is the Ramped Setpoint doing today The Ramped Setpoint is equal to the

What is the Ramped Setpoint doing today The Ramped Setpoint is equal to the stepped Setpoint when a resource that is under control mode 1 or 2 is following exactly. However when a unit follows slower than their dispatched ramp rate or they move in the opposite direction the Ramped Setpoint can be quite different. Ramped Setpoint = Dispatch (5 min stepped) +/- OR deployment limited at unit ramp rate Casey Cathey 12

Dead Band Control Error Calculation Participant resource operators want to reduce wear and tear

Dead Band Control Error Calculation Participant resource operators want to reduce wear and tear on units and follow in the most efficient way possible. Typically there are dead bands in which a unit will not be moved until exceeded. Assume Unit: • Can ramp 1 MW/min • Controls have a 2 MW dead band in which they will not start pulsing the unit • Is dispatched up for two periods at full ramp amount 50 ->55 ->60 MW Casey Cathey 13

Lost market capacity with Ramped Setpoint Over the 15 minute period if the unit

Lost market capacity with Ramped Setpoint Over the 15 minute period if the unit is sending just Ramped Setpoint to the resource controls (ignore stepped and dispatch) then capacity is lost In this case you see the delay in starting to follow the Ramped Setpoint due to the 2 MW dead band causes them to not start moving the unit until minute 3 Casey Cathey 14

The Numbers Casey Cathey 15

The Numbers Casey Cathey 15

What does it look like Ramped Set point resets with RTBM case to unit

What does it look like Ramped Set point resets with RTBM case to unit Output Stepped Set point RTBM Dispatch Ramped Set point Unit Output Every 5 mins the unit briefly changes direction as it appears to have overshot Period up to 90 seconds after RTBM ramped set point reset where unit does not move Casey Cathey 16

Goals in modifying Ramped Setpoint 1. Reduce wear and tear on units following 2.

Goals in modifying Ramped Setpoint 1. Reduce wear and tear on units following 2. Stop the point from resetting itself into the unit dead band on 5 minute boundaries 3. Limit movement to ramp-able amount 4. Keep Ramp Setpoint and Stepped Setpoint moving in the same direction 5. Keep it simple to avoid unintended consequences 6. Ensure regulation is realized as quickly as possible Casey Cathey 17

Investigated a Number of Possible Solutions 1. Leave Ramp Setpoint as is and through

Investigated a Number of Possible Solutions 1. Leave Ramp Setpoint as is and through education and reach out work with participants to get individual workable solutions. 2. Change Ramp Setpoint to not reset a. Ramp between previous and (next dispatch + OR) b. Current output toward stepped Setpoint limited by ramp rate c. Current output toward stepped Setpoint limited by ramp rate + limit movement to be in same direction as stepped Setpoint from current output Casey Cathey 18

Recommendation 1. Each of the solutions help with some of the goal, but none

Recommendation 1. Each of the solutions help with some of the goal, but none of the solutions solve all the goals. 2. Time is of the essence if we want to change the approach for go live and reserve time to test. Casey Cathey 19

Announcements BSS Submittal Window: SPP plans on implementing an increase in the amount of

Announcements BSS Submittal Window: SPP plans on implementing an increase in the amount of time prior to an OD to 90 days • System is expected to be up for live production submittal starting Feb 22 • Any submittal before Feb 22 is for testing; will be wiped out before go live Ginny Watson 20

Announcements Decommission of COP True Up Report: • Originally created due to Gap between

Announcements Decommission of COP True Up Report: • Originally created due to Gap between Process and SPP Markets UI/Markets API • Gap was filled with the 1. 8 Markets Release • SPP will decommission this report as of January 13 th, 2014. • All information provided by the report is now available via the Markets UI • Path is: – Resource Offers>>DAY>>Resources>>Resource>>Commitment History – Resource Offers>>DAY>>Market Results>>Commitment History Gary Cate 21

 • Scenario Tracker as of Jan 8 Summary – Scenarios in Parallel Ops:

• Scenario Tracker as of Jan 8 Summary – Scenarios in Parallel Ops: – 4 Scenarios, 2 Partial Scenarios, Dispatch Simulator Test successful: § § § Scenario 7 Must Offer Scenario 28 a Scenario 15 Commitment Mismatch Part 2 Scenario 24 Scenario 13 DDR w/ Settlements Scenario 29 RSG § Dispatch Simulator Test – Note: Dispatch Simulator Test will occur again from Jan 15 -17 • Summary – Retests Pending: – 3 entire Scenario retests pending § § § – Scenario 11 scheduled for Jan 21 Scenario 31 BDR w/ Settlements to be retested Jan 14 Non-Functional Testing Scenario to be retested Jan 14 9 individual retest requests pending Ginny Watson 22

Look-Ahead: Jan 9 – Jan 19 Parallel Ops Calendar on spp. org > Integrated

Look-Ahead: Jan 9 – Jan 19 Parallel Ops Calendar on spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents • Settlements details on the Settlements Calendar marketplace. spp. org > Public > Settlements > Calendars Jan 8 – Jan 11 Schedule: Ramp Sharing Off Thurs Jan 9: Fri Jan 10: IDT Prep Call IDT Call 6: 30 A – 4: 30 P No Regular Parallel Ops Friday Meeting Jan 12 – Jan 18 Schedule: Ramp Sharing On Mon Jan 13: Tues Jan 14: IDT Debrief Scenario 31 BDR w/ Settlements Non-functional Test Scenario 10: 00 PM – 12: 00 AM Wed Jan 15: GFA Carve Out Scenario Jan 15 – 17: Dispatch Simulator Fri Jan 17: Regular Parallel Ops Friday Meeting Ginny Watson 23

Fri 1/10 IDT Reminders • 1/10 IDT Call starts at 6: 30 AM, 8

Fri 1/10 IDT Reminders • 1/10 IDT Call starts at 6: 30 AM, 8 hour IDT begins at 7: 30 AM – ICCP Failover § SPP has requested that an MP perform an ICCP failover in the middle of the January 10 th IDT. – TARRF – Simple Failover § Failover will be a standard model upload failover and primary systems will not be down for but a few minutes. The secondary system will be up and providing setpoints along with the XML signals. Brian Strickland 24

Scenario 31 BDR w/ Settlements *Scenario document on spp. org here spp. org >

Scenario 31 BDR w/ Settlements *Scenario document on spp. org here spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents > Scenarios in Parallel Ops Scenario Description • Verify functionality specific to BDRs MP Activities *Applicable to MPs with BDRs only • Verify that BDRs have valid offer data for all products • Ensure all required real-time measurements are defined and being submitted to SPP via ICCP • Submit hourly DRL forecasts when required for BDR resources • Verify that Dispatch Instructions are received via XML the same as other Resources • Submit meter data for time that BDR is dispatched Jarrett Friddle 25

Non-Functional Testing *Scenario document on spp. org here spp. org > Integrated Marketplace >

Non-Functional Testing *Scenario document on spp. org here spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents > Scenarios in Parallel Ops Scenario Description • SPP will simulate an outage at the Chenal facility for Reliability and Markets redundancy • Non-Functional Test includes critical Tier 0 and Tier 1 components to demonstrate that those systems can be entirely supported and function independently in the back-up Operations Data Center in Maumelle MP Activities • MPs should continue normal operations with no expected downtime. • If any unexpected results occur, please submit a detailed RMS Ticket here • Verify the following during Non-Funct Testing Jan 14 at 10 P – 12 A CPT • ICCP setpoint instructions are received • XML notifications are received • Marketplace Portal User Interfaces and API’s are available for data entry and retrieval Casey Cathey 26

36. 0 – GFA Carve Out Schedules Expected Result • MP should be able

36. 0 – GFA Carve Out Schedules Expected Result • MP should be able to verify their GFA Carve Out Schedules were successfully submitted and approved • MPs settlement statement will have appropriate charge/credit • MPs related to the GFA Carve Out will receive charges/credits to offset the DAMKT energy charges/credits • MPs NOT related to the GFA Carve Out will receive uplift charges on a load-ratio share to cover the funds used to credit the GFA Carve Out MP settlement statements won’t be available until OD+7 (or as scheduled on the calendar posted in the Marketplace Portal) • MP Activities *Applicable to only GFA Carve Out Entities MPs (NPPD, LES, BEPC) • • MP to submit DA Offer corresponding to the GFA Carve Out Resources MP to submit GFA Carve Out Schedules related to their relevant GFA Nick Parker 27

Dispatch Simulator • During Structured Market Trials SPP has utilized a COP True-up process

Dispatch Simulator • During Structured Market Trials SPP has utilized a COP True-up process to match commitments in EIS with IM in order to appropriately dispatch resources based on real time commitments. – Can cause confusion since there is no tie between DA Market/RUC and RTBM – MPs receive multiple commitment notifications between the DA Market, RUC and COP True-up • SPP’s vendor created a Dispatch Simulator to mimic the response of Resources to SPPs commitment and dispatch –Primarily used for testing purposes before EMS was integrated with RTBM • The Simulator provides a perfect response of Resources to the SPP XML energy deployment instructions • The Simulator was run previously from December 5 thru December 8, 2013. • The Simulator will be turned on from January 15 thru January 17, 2014. Ronnie Johnson 28

Dispatch Simulator 0: 01 01/15 to 23: 59 01/17 • MPs should offer their

Dispatch Simulator 0: 01 01/15 to 23: 59 01/17 • MPs should offer their Resources as they expect to do in Production • SPP will not be running any COP True-ups during this time. • Commitments will only come from DBDA, DA Market, DARUC, IDRUC, and MP Self Commitments – MPs should be looking at commitment notifications and understand that this is how SPP expects to run at Go Live – MPs should ignore the ICCP dispatch setpoints during the scheduled days that the simulator will be used. The dispatch deployment for energy coming from XML should be used for reviewing dispatch instructions. • SPP will overwrite meter settlement data for the three Operating days Jan 15, 16, and 17 with RTBM setpoint instructions Ronnie Johnson 29

Migration and Maintenance Report posted to spp. org > Integrated Marketplace > Market Trials

Migration and Maintenance Report posted to spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents Wendy Reynolds 30

Release Scorecard posted to spp. org: spp. org > Integrated Marketplace > Market Trials

Release Scorecard posted to spp. org: spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents Ginny Watson 31

Known Issues Log • Please review the Market Trials Known Issues Log on spp.

Known Issues Log • Please review the Market Trials Known Issues Log on spp. org > Integrated Marketplace > Market Trials > Structured/Unstructured Test > Integrated Marketplace Market Trials Project Documentation Folder Issue # Component 882 883 884 Severity Synopsis Markets UI Performance Issues have been discovered with the following web methods: Get. Market. LMPSummary. Set. By. Day Get. Energy. Mitigated. No. Load. Cost. Set. By. Day Get. Market. Node. Set. By. Day Get. Market. Message. Set. By. Day Get. Market. Binding. Limit. Set. By. Day Get. Market. LMPSummary. By. Interval. Set Get. Virtual. Bid. Set. By. Day Get. Energy. Mitigated. Offer. Set. By. Day Get. Energy. Mitigated. Resource. Offer. Set. By. Day Severity 3: Medium Markets UI The Markets API should allow participants to submit a null price and null Fixed MW for Operating Reserve Offers when the OR dispatch status is "Not. Qualified". The current validations for Not Qualified status allow a null Fixed MW, but not a null Price. This change will require updates to the validation rules as well as updates to the Reserve XSD used by members. As a temporary workaround, participants may submit a null price through the Markets UI or a price of 0 through the Markets API. Severity 3: Medium Markets UI The overrides display in MUI does not show when an override was entered by an operator. It only shows the intervals that were overriden. If an operator makes an override at 1550 for the current hour and the next 2 hours, the MUI only displays that an override was entered for 1500 -1800. This is misleading and the display should display a timestamp of when the override occurred. Severity 3: Medium Carrie Simpson 32

Market Trials Useful Resources Integrated Marketplace Parallel Operations Project Documents spp. org > Integrated

Market Trials Useful Resources Integrated Marketplace Parallel Operations Project Documents spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents • • • Parallel Ops Presentations, Statistics, MP Meetings Folder Parallel Ops Reference Documents Folder Scenarios in Parallel Ops Folder > Scenarios Schedule Parallel Ops Communication Plan Parallel Ops MP Guide Parallel Ops Calendar Known Issues Log Release Scorecard Migration Report Integrated Deployment Test (as part of Parallel Ops) spp. org > Integrated Marketplace > Market Trials > Integrated Deployment Test • • IDT MP Guide Parallel Ops Calendar Additional documents: • Settlements Calendar marketplace. spp. org > Public > Settlements > Calendars • Change Tracker spp. org > Integrated Marketplace > Marketplace Documents • Integrated Marketplace Structured/Unstructured Test Project Documents spp. org > Integrated Marketplace > Market Trials > Structured / Unstructured Test> Integrated Marketplace Structured / Unstructured Test Project Documents 33

Settlements • Update from Philip Bruich 34

Settlements • Update from Philip Bruich 34

Markets System Update – MOS 1. 9. 3 Delivery on Jan 17 th, PROD

Markets System Update – MOS 1. 9. 3 Delivery on Jan 17 th, PROD promotion Feb 3 rd § – Accelerated testing schedule will allow for functional regression, integration, performance and non functional testing against 1. 9. 3 Included in 1. 9. 3: § Market Control and MDB performance improvements § Mitigation and MCE Updates § FCA and EDR logic § MUI defects addressing MP requests – API not allowing Bids and Offers to span multiple ODs – CR deployment notification start time correction Erin Jester 35

Markets System Freeze Sam Ellis 36

Markets System Freeze Sam Ellis 36

Erin Jester READINESS METRICS UPDATE

Erin Jester READINESS METRICS UPDATE

Market Participant Readiness Activity Market Participant Readiness Status Variance Analysis Yellow 35 of 50

Market Participant Readiness Activity Market Participant Readiness Status Variance Analysis Yellow 35 of 50 Metrics are complete • This information and more details on overall program Readiness are available on the External Scorecard. • The December External Scorecard is posted on i. Dashboard. Select the “External Scorecard” dropdown, then “External Scorecard : : Period Ending 12/15/13”. *Metrics data is as of 12/28/2013 Erin Jester 38

Market Participant Readiness • At risk – ‘yellow’ status due to: – MP systems

Market Participant Readiness • At risk – ‘yellow’ status due to: – MP systems build § Concerns due to the compression of program timeline and vendor readiness, particularly in regards to Shadow Settlements – Readiness Metrics status § MP Readiness status is primarily measured by the established Readiness Metrics at this stage in the program Erin Jester 39

Readiness Metrics Overview Color Grey Blue Green Yellow Red • Total Number of Readiness

Readiness Metrics Overview Color Grey Blue Green Yellow Red • Total Number of Readiness Metrics: 50 • Number Complete (Blue): 35 • Number on Track (Green): 7 • Number at Risk (Yellow/Red): 7 • Number Not Started (Gray): 1 Description Monitoring of the metric has not begun or not applicable The metric success criteria has been met and no longer requires monitoring, and/or the metric has been completed by the target due date The metric is meeting identified success criteria, and/or The metric is on schedule to meet the target due date The metric has not met the highest level of the identified success criteria, and/or The metric is at risk not meeting the identified target due date The metric has not met the highest or mid-range success criteria as defined, and/or the metric has not been completed by the specified target due date Erin Jester 40

At Risk (Yellow) Readiness Metrics ID CBA-01 OPS-17 INT-02 RSG – 01 Metric Name

At Risk (Yellow) Readiness Metrics ID CBA-01 OPS-17 INT-02 RSG – 01 Metric Name CBA Certification Activities ICCP Model Validation Status Yellow Variance This metric has a defined milestone date of October 1, 2013 for EIS BAs to provide Operating Protocol data to SPP; that milestone is past due. The CBA team is working to get this complete and anticipates receiving all data by January 1, 2014. 1/41 of MPs ICCP models are not validated, but is expected to be in service shortly. There is 1 open Marketplace position. Final interview are ongoing, and a final hiring decision is scheduled for January. Organizational Staffing Readiness Yellow RSG Member Integration and Testing Complete Yellow Milestones tracking at risk Yellow 12/1 - 12/15: 2 Scenarios Passed (24, 13), 1 Scenario Failed (31), and 1 non-functional testing scenario failed Structured/Unstructured Test TRL-02 Complete Erin Jester 41

At Risk (Red) Readiness Metrics ID OPS-15 Metric Name CROW Outage Submission Status Red

At Risk (Red) Readiness Metrics ID OPS-15 Metric Name CROW Outage Submission Status Red Variance 24/25 of resource-owning MPs who have reported outages have not reported them through SCADA and CROW within 30 minutes. Communications are being sent to MPs directly by the outage team. OPS-18 ICCP Control Status Tracking Red 12/1 - 12/14: 85. 3% of CMODs submitted correctly. 11/41 MPs submitted 100% of CMODs correctly. No BA MP's are at 100% Erin Jester 42

Not Started Metrics • ID TCR-04 The following metric has not started tracking Metric

Not Started Metrics • ID TCR-04 The following metric has not started tracking Metric Name TCR Production - Two Monthly Auctions Status NS Variance Metric will start according to planned TCR Timeline Erin Jester 43

Erin Jester PARALLEL OPS EXIT CRITERIA

Erin Jester PARALLEL OPS EXIT CRITERIA

Parallel Ops Exit Criteria • 28 Parallel Operations Exit Criteria Total – 23 complete

Parallel Ops Exit Criteria • 28 Parallel Operations Exit Criteria Total – 23 complete – 1 tracking on time – 4 at risk (see next slide) • Posted on Readiness Center in the Go/No-Go Criteria Documents folder Erin Jester 45

‘At Risk’ Parallel Ops Exit Criteria The 4 ‘at risk’ P Ops Exit Criteria

‘At Risk’ Parallel Ops Exit Criteria The 4 ‘at risk’ P Ops Exit Criteria are 3 of the yellow/red Readiness Metrics • ID Metric Name Status Variance POX-20 OPS-15 CROW Outage Submission 24/25 of resource-owning MPs who have reported outages have not reported them through SCADA and CROW within Yellow 30 minutes. Communications are being sent to MPs directly by the outage team. POX-21 OPS-17 ICCP Model Validation Yellow 1/41 of MPs ICCP models are not validated, but is expected to be in service shortly. OPS-18 ICCP Control POX-22 Status Tracking 12/1 - 12/14: 85. 3% of CMODs submitted correctly. 11/41 Yellow MPs submitted 100% of CMODs correctly. No BA MP's are at 100% TRL-02 POX-01 Structured/Unstructured test complete Yellow 12/1 - 12/15: 2 Scenarios Passed (24, 13), 1 Scenario Failed (31), and 1 non-functional testing scenario failed 46

Erin Jester GO-LIVE CRITERIA & GO/NO-GO PROCESS

Erin Jester GO-LIVE CRITERIA & GO/NO-GO PROCESS

Go-Live Criteria • 21 Go-Live Criteria Total – 2 complete – 13 tracking on

Go-Live Criteria • 21 Go-Live Criteria Total – 2 complete – 13 tracking on time – 1 at risk (see next slide) – 5 not started tracking § • Includes activities that will be complete closer to go-live (e. g. publishing issues/risks, workarounds for high defects, etc. ) Latest status of criteria posted in the Go/No-Go Folder on SPP. org Erin Jester 48

‘At Risk’ Go Live Criteria ID GL-03 Metric Name Status SPP will provide an

‘At Risk’ Go Live Criteria ID GL-03 Metric Name Status SPP will provide an 30 Yellow day system freeze, and Go-Live will not begin until the conclusion of the freeze. Variance At risk due to Market system release issues. Erin Jester 49

Market Go-Live Decision Timeline • Go-Live Team checkpoint held on December 23 • Upcoming

Market Go-Live Decision Timeline • Go-Live Team checkpoint held on December 23 • Upcoming Go/No-Go milestones: Erin Jester 50

Reminder! Submit issues/risks • MPs can submit known issues and risks for Go-Live via

Reminder! Submit issues/risks • MPs can submit known issues and risks for Go-Live via RMS – Select the Readiness dropdown – Please submit by EOD Monday, January 13 • Issues/Risk list will be published with the Go-Live Go/No -Go Initial Assessment on January 17 Erin Jester 51

APPENDIX 52

APPENDIX 52

EIS Suspension: Units not following during IDTs During the first IDT tests SPP staff

EIS Suspension: Units not following during IDTs During the first IDT tests SPP staff noticed a number of abnormalities with Resources following issued Set Points a. Resources appearing to not switching to IM control and follow EIS targets through test b. Resources that followed IM Set Points except when prices diverged at which point they followed EIS c. Resources that follow in a delayed mode For Reference Only 53

EIS Suspension: Disable EIS SPP in its desire to simulate the IM as close

EIS Suspension: Disable EIS SPP in its desire to simulate the IM as close as possible wanted to remove the incentive to follow EIS. During one of the IDT wrap up calls the idea to disable EIS market was shared with the participants. • The thought was if the price of EIS was not visible then units would only follow IM • Also a possibility to use proxy pricing instead of EIS For Reference Only 54

EIS Suspension: More complicated As we researched internally as well as reached out to

EIS Suspension: More complicated As we researched internally as well as reached out to participants in an effort to more fully understand the underlying issues, SPP found that the reasons for not following were more complicated than we had initially assumed a. Price divergence was magnified by RTBM solutions that struggled to relieve binding Flowgates b. Ancillary services were not matched between EIS and IM causing Resources to have large movements in the beginning of tests and across hours c. Ramped Set Point calculation For Reference Only 55

EIS Suspension: Price Divergence Due to Flowgates 1. In later tests we have been

EIS Suspension: Price Divergence Due to Flowgates 1. In later tests we have been able to manage Flowgates with ramp sharing enabled and keep them in line with the flows observed in the EIS systems. 2. Better control room visualization tools to see what is impacting Flowgates and product scarcity For Reference Only 56

EIS Suspension: Ancillary Service Offers Not Matching 1. Most of this problem was solved

EIS Suspension: Ancillary Service Offers Not Matching 1. Most of this problem was solved through education of how offers should be entered into the IM in order to match participants intents 2. Increased tools to see Resources being dispatched differently in an effort to be proactive 3. Override of control modes where offers are incorrect For Reference Only 57

EIS Suspension: Ramped Set Point 1. Members following the Ramped Set Point are finding

EIS Suspension: Ramped Set Point 1. Members following the Ramped Set Point are finding issues with EMS and control settings related to dead bands as the point resets to actual output every 5 minutes 2. This topic will be covered in other presentations For Reference Only 58

EIS Suspension: Problems with Disabling EIS 1. It is not obvious how to safely

EIS Suspension: Problems with Disabling EIS 1. It is not obvious how to safely turn off EIS ICCP during the test and be able to enable after the test. 2. Prices would still be on the contour map – possibly allowing the same conditions 3. Members could estimate EIS prices by conditions and XML messages For Reference Only 59

EIS Suspension: Problems with Proxy pricing 1. Not clear if we are allowed to

EIS Suspension: Problems with Proxy pricing 1. Not clear if we are allowed to proxy price without a system outage 2. As Resources would get paid based on cost there still exists a disincentive to follow down. 3. **** Would require MOPC approval before switching settlement process which would leave very few tests after approval **** 4. Other possible unintended consequences due to switching methodology For Reference Only 60

EIS Suspension: IDT pricing Recommendation Continue as originally defined and leave EIS data feeds

EIS Suspension: IDT pricing Recommendation Continue as originally defined and leave EIS data feeds enabled. a. SPP and participants have progressed substantially through the first tests. Price divergence has been minimized b. Timing needs with MOPC approval make a change difficult For Reference Only 61