Parallel Ops GoLive Checkpoint Weekly Update Friday Meeting

  • Slides: 77
Download presentation
Parallel Ops / Go-Live Checkpoint Weekly Update – Friday Meeting December 13, 2013

Parallel Ops / Go-Live Checkpoint Weekly Update – Friday Meeting December 13, 2013

Agenda • IDT Debrief: 9: 00 A – 9: 30 A • IDT Reminders,

Agenda • IDT Debrief: 9: 00 A – 9: 30 A • IDT Reminders, 9: 00 -9: 05 A • IDT Report Review: IDT 12/3 and IDT 12/5, 9: 05 -9: 15 A • IDT 12/10 Preliminary Debrief, slides for reference • IDT 12/12 Preliminary Debrief, 9: 15 - 9: 30 A • Scenario Analysis: 9: 30 A – 9: 50 A • Look-Ahead: 9: 50 A – 10: 20 A • Announcements, 9: 50 -10: 00 A • Look-Ahead, 10: 00 -10: 20 A • Updates, 10: 20 -11: 30 A • web. Mirroring and Tagging Q & A: 10: 20 A – 11: 00 A • Readiness Update: 11: 30 A – 11: 30 A • 11: 30 A Go/No Go Roll Call for IDT Sunday Dec 15: 11: 30 A – 12: 00 P 2

IDT Conditions page 1 of 2 • IDT Conditions posted to spp. org here

IDT Conditions page 1 of 2 • IDT Conditions posted to spp. org here on the PO Scenario Schedule spp. org > Integrated Marketplace > Market Trials > Integrated Deployment Test • Changes since the last posting are shown in green IDT Date Condition Various Follow Resource IDTs Online including 12/12 and 12/19 Notes SPP will coordinate with a Resource-owning MP such that if they have a planned Resource start, SPP will put it in COP as normal; however, the MP will monitor the start signals from SPP during the start window. There will be no official commitment from SPP, meaning no MWPs. SPP will monitor this condition and report any anomalies from either end. SPP is soliciting MPs to participate in this IDT condition. If you have a Resource already planning on coming online during a regularly scheduled IDT, please let us know via RMS. Casey Cathey 3

IDT Conditions IDT Date page 2 of 2 Condition Notes Mirroring Off SPP will

IDT Conditions IDT Date page 2 of 2 Condition Notes Mirroring Off SPP will require additional monitoring and reporting during the 12/17 IDT as this is the first IDT after web. Mirroring is disabled on 12/16. 12/17/13 EDR 12/19/13 Pseudo Tied In Resources RSG Event 1/7/14 An MP requested permission to test a Pseudo Tied In Resource during the 12/19 deployment test. SPP had asked the MP not to include the Resource in deployment testing before 12/19 due to the pseudotie issue. First IDT with all RSG changes effective Regulation Pre-Selection / SPP will exercise Regulation pre-selection during the 01/07 IDT. This will run through normal ID-RUC Co-Optimize Regulation regulation pre-selection, allow BA Operators to change Resources from certain regulation, etc. 1/9/14 ICCP Failover SPP has requested that an MP perform an ICCP failover in the middle of the January 9 th IDT. This was a request from the CWG for a real world condition. SPP will coordinate with individual MP. Casey Cathey 4

IDT 12/03 Report IDT reports posted to spp. org here 7 calendar days after

IDT 12/03 Report IDT reports posted to spp. org here 7 calendar days after each IDT spp. org > Integrated Marketplace > Market Trials > Integrated Deployment Test IDT 12/05 Report IDT reports posted to spp. org here 7 calendar days after each IDT spp. org > Integrated Marketplace > Market Trials > Integrated Deployment Test Brian Strickland 5

IDT Reminders Follow IM Setpoints Ensure Control Modes submitted correctly Ensure Operating Reserves in

IDT Reminders Follow IM Setpoints Ensure Control Modes submitted correctly Ensure Operating Reserves in both markets submitted correctly For Reference Only 6

IDT Reminders: Follow IM Setpoints • Prices / dispatch deviating from EIS substantially during

IDT Reminders: Follow IM Setpoints • Prices / dispatch deviating from EIS substantially during Dec 10 IDT • Swinging prices / dispatch not something SPP wants to see in Production • Difference in resource MW actuals and desired MW causes collective delta – • 200 MW delta in the down direction across Dec 10 IDT SPP is not getting response needed to keep ACE within L 10 – SPP has to put reliability first so SPP offsets targets and SCED for the Marketplace which causes scarcity; ramp shortages; and prices that diverge from EIS – Once SPP finally gets a response across the footprint, SPP relaxes the offset; SCED solves; MPs are closer to where SPP expects them to be For Reference Only 7

IDT Reminders: Follow IM Setpoints • Not due to specific MPs’ behavior but collective

IDT Reminders: Follow IM Setpoints • Not due to specific MPs’ behavior but collective behavior of all MPs – Challenge to SPP when everyone is off in the same direction – SPP follows up with individual MPs on the Resource side – Important for MPs with DVERs to follow IM Setpoints § SPP sends out follow flag via ICCP, please follow § Use realistic Ramp Rates and SPP will be able to align IM and EIS – Unrealistic Ramp Rates in offer parameters cause DVERs not to get dispatched down as quickly as they would in EIS • Please validate on the ICCP Side • SPP intends to follow-up with MPs to identify the issue(s) – Are MPs following EIS setpoints? Ramp setpoints? 5 min dispatches? For Reference Only 8

IDT Proposal: Block EIS • SPP is trying to uncover the root cause of

IDT Proposal: Block EIS • SPP is trying to uncover the root cause of MPs not following IM setpoints • • SPP believes some of that may be due to the presence of the EIS price signal SPP is reviewing a number of options, including blocking or suspending EIS price signals • SPP is working internally to develop a detailed approach document for MPs to review the week of Dec 16 • Any EIS blocking would not take place before the January 7 IDT 9 For Reference Only

12 -10 -13 Deployment Test Debrief • Overall the Fifth Deployment test was another

12 -10 -13 Deployment Test Debrief • Overall the Fifth Deployment test was another success – All BAs and MPs were on the call close to 60 min before IDT began. Thank you! – Able to start and stop the test on time as scheduled with no delay, reduction or extension – However, this test was the most concerning to date with the issues we had For Reference Only 10

12 -10 -13 Deployment Test Debrief • Though a “Success” there are things we

12 -10 -13 Deployment Test Debrief • Though a “Success” there are things we can improve upon/ are concerning – EIS BA’s switched EIS setpoints to IM setpoints and back again § Collection of L 10 for Dec 10 test was 168; at the beginning of the test ACE went to 200 -300 range: – – – SPP deployed Reg. Down MPs struggled to follow IM setpoints Wind was ramping up at the time 2 CPS 2 violations 45 minutes into the test before SPP ACE crossed 0 1530 RTBM interval of IDT was not approved § SPP is developing internal metrics to monitor RTBM performance 11 For Reference Only

12 -10 -13 Deployment Test Debrief • Though a “Success” there are things we

12 -10 -13 Deployment Test Debrief • Though a “Success” there are things we can improve upon/ are concerning – MPs/BAs reported significant divergence between EIS and IM § SPP received reports that large differences occurred between EIS and IM LIPs/LMPs as well as dispatch – Factors contributing to this were found to be the follows: » Turn around ramp rates » Ramp sharing being disabled » Mismatch between EIS and IM offers – This divergence caused large ACE problems for the EIS BAs at the conclusion of the test while reverting back to the EIS market. 12 For Reference Only

12 -10 -13 Deployment Test Debrief • Though a “Success” there are things we

12 -10 -13 Deployment Test Debrief • Though a “Success” there are things we can improve upon/ are concerning – SPP is seeing 5, 000 – 6, 000 MW more generation than would be committed if SPP was the sole BA § More units are committed through EIS than would be committed through IM § More information to come on how SPP intends to address this issue and during the January 7 th test 13 For Reference Only

12 -10 -13 IDT Condition: Ramp Sharing • Ramp Sharing was “off” for the

12 -10 -13 IDT Condition: Ramp Sharing • Ramp Sharing was “off” for the entire Dec 10 IDT – Offline, SPP has noted a number of issues turning Ramp Sharing off – Ramp Sharing was on for the first 4 IDTs – SPP is gathering comparison data for the Market Working Group to justify the percentage of Ramp Sharing SPP will need on during Go Live § – – SPP has Ramp Sharing off in lower QA environment and on in PROD to gather additional data While Ramp Sharing was off, SPP had to select 300% of the requirement § SPP has Parallel Runs of up to 5 different Scenarios and usually selects what most accurately represents what is happening in Real Time § While Ramp Sharing was off, SPP selected the “ 300% Scenario” to prevent Scarcity In order to get accurate data for Ramp Sharing, beginning with interval ending 1505, SPP selected only what was needed and Scarcity Pricing occurred § For example, Reg-Up prices were 8 -10 times as much when Scarcity Pricing occurred 14 For Reference Only

-D 10 EC-D 13 10 EC- 13. -D 13 30 10 EC- 13. -D

-D 10 EC-D 13 10 EC- 13. -D 13 30 10 EC- 13. -D 13 35 10 EC- 13. -D 13 40 10 EC- 13. -D 13 45 10 EC- 13. -D 13 50 10 EC- 13. -D 13 55 10 EC- 14. -D 13 00 10 EC- 14. -D 13 05 10 EC- 14. -D 13 10 10 EC- 14. -D 13 15 10 EC- 14. -D 13 20 10 EC- 14. -D 13 25 10 EC- 14. -D 13 30 10 EC- 14. -D 13 35 10 EC- 14. -D 13 40 10 EC- 14. -D 13 45 10 EC- 14. -D 13 50 10 EC- 14. -D 13 55 10 EC- 15. -D 13 00 10 EC- 15. -D 13 05 10 EC- 15. -D 13 10 EC 15 -1. 1 3 5 15. 2 0 10 -D 10 EC-1 -D 3 1 10 EC-1 3. 3 -D 3 0 1 10 EC-1 3. 3 -D 3 5 1 10 EC-1 3. 4 -D 3 0 1 10 EC-1 3. 4 -D 3 5 1 10 EC-1 3. 5 -D 3 0 1 10 EC-1 3. 5 -D 3 5 1 10 EC-1 4. 0 -D 3 0 1 10 EC-1 4. 0 -D 3 5 1 10 EC-1 4. 1 -D 3 0 1 10 EC-1 4. 1 -D 3 5 1 10 EC-1 4. 2 -D 3 0 1 10 EC-1 4. 2 -D 3 5 1 10 EC-1 4. 3 -D 3 0 1 10 EC-1 4. 3 -D 3 5 1 10 EC-1 4. 4 -D 3 0 1 10 EC-1 4. 4 -D 3 5 1 10 EC-1 4. 5 -D 3 0 1 10 EC-1 4. 5 -D 3 5 1 10 EC-1 5. 0 -D 3 0 1 10 EC-1 5. 0 -D 3 5 1 10 EC-1 5. 1 -D 3 0 EC 15 -1. 15 3 15. 2 0 10 Ramp Sharing – Scarcity Pricing 12/10/13 IDT REGDN MCP 80 70 60 50 40 30 20 10 0 12/10/13 IDT REGUP MCP 70 12/10/13 IDT REGUP MCP 60 50 40 30 20 10 0 15

12 -10 -13 IDT Condition: CRD Event • Intentional 50 MW resource drop at

12 -10 -13 IDT Condition: CRD Event • Intentional 50 MW resource drop at 2: 25 P to induce a CRD Event – Net load relatively flat – SPP clearing and deploying down to the. 1 MW § As designed / Protocol compliant – Good test - no system defects found – 70 Resources sent a CR Deployments at a very small MW level – § Only 3 did not respond; however, Resources were not off much so they did not get capped for the day according to the Protocols § All 70 Resources were protocol compliant RSS event ended at 3: 00 P 16 For Reference Only

12 -10 -13 IDT Analysis: Pumping Loads • • • Systems performed well and

12 -10 -13 IDT Analysis: Pumping Loads • • • Systems performed well and did what they were intended to do • 2 MP concerns: – MP submitted data for IE 15: 00 which is data from 14: 50 to 15: 00 they Generation was capped at 0 MW Load showed real time pumping interpreted as data for 15: 00 to 15: 10 – • MP submitted mid-term forecast that did not correlate to the submitted short-term forecast. RTBM will be use mid-term if no short-term is available. In this case, RTBM assumed there was 24 MW of load 1 SPP concern: – The pumping resource was still connected to main island ( breaker remained close). Even though everything worked, SE was pushing 3 MW. SPP would like to remove the resource from main island when in pumping mode 17 Brian Strickland

12 -12 -13 Deployment Test Debrief • Overall the Sixth Deployment test was another

12 -12 -13 Deployment Test Debrief • Overall the Sixth Deployment test was another success – Majority of BAs and MPs were on the call close to 60 min before IDT began § 1 BA late to join and 1 MP missing – Able to start and stop the test on time as scheduled – Resources were brought online during the test; systems performed correctly § Coordinated Resource start worked well § Reminder: All Resources that come online for the remaining IDTs are requested to coordinate with BA and Gen Dispatcher at SPP. Please submit via RMS, Parallel Ops Quick Pick. – Small, natural CRD Event occurred; handled as expected – No big issues; overall, test managed and controlled well Brian Strickland 18

12 -12 -13 Deployment Test Debrief • Though a “Success” - areas of concern/opportunities

12 -12 -13 Deployment Test Debrief • Though a “Success” - areas of concern/opportunities to improve – SPP working on process to improve first 30 -45 min of IDTs – Issues with ACE going into the test (ran at -300 to -400 for the first 30 minutes of the test – at one point went to -554 MWs) § Combination of factors: – IDT started with collective EIS BAs ACE negative – Wind Resources across the footprint was ramping down by ~300 MWs – A lot of Resources off dispatch » Number of Resources on Energy and being deployed UP accounted for ~300 of ACE » SPP reached out to a number of Resources – § – Ramp Sharing contributed, but was not main culprit 2 CPS 2 violations § 30 minutes into the test before SPP ACE crossed 0 ACE was under control by 0700 for the reminder of the test § 0900 frequency dip on interconnect (ACE went to -200 to -250 range), overall was managed well 19 Brian Strickland

12 -12 -13 Deployment Test Debrief • Though a “Success” - areas of concern/opportunities

12 -12 -13 Deployment Test Debrief • Though a “Success” - areas of concern/opportunities to improve – Ramp Sharing Off § 0655 Interval: SPP decided to put full Ramp Sharing on to clear more Reg – Turning Ramp Sharing “on” helps clearing, but MPs need to get on Deployment – SPP cleared ~300 MW and deployed 170 MW when Ramp Sharing turned on § Backed down to 50% Ramp Sharing after the 0700 interval § Ramp Sharing “off” at 0800 hour § After 0835 interval, SPP backed off Reg Commitment ~200% – To that point, SPP was getting 300% of what was needed – Scarcity occurred 20 Brian Strickland

12 -12 -13 Deployment Test Debrief • Though a “Success” - areas of concern/opportunities

12 -12 -13 Deployment Test Debrief • Though a “Success” - areas of concern/opportunities to improve – SPP working to improve internal process for Reg Selection – SPP hopes to help MPs with any systematic issues and ensure MPs are equipped to follow signals after Go-Live 21 Brian Strickland

Scenario 7 Must Offer Update: • The Settlements Parallel Ops Calendar reflects an R

Scenario 7 Must Offer Update: • The Settlements Parallel Ops Calendar reflects an R 2 for 11/13 Operating Day posted Wednesday, December 11, 2013 Settlements Calendar marketplace. spp. org > Public > Settlements > Calendars • The resettlement addresses the Must Offer Scenario that Markets conducted on November 13 th Ginny Watson 22

Scenario 31 BDR w/ Settlements *Scenario document on spp. org here spp. org >

Scenario 31 BDR w/ Settlements *Scenario document on spp. org here spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents > Scenarios in Parallel Ops Results • Failed – Scenario will require retest • BDRs are not modeled as Resources, they are modeled as loads • Control Modes are hard-coded to be non-dispatchable but should be hard-coded to dispatchable (Control Modes for BDRs are not submitted by MPs for BDRs • This was not caught while testing in the lower environments since we simulate the Control Mode inputs from MPs • An issue has been created and added to the KIL list with a KIL # of 856 • This is corrected and will be delivered in 1. 9+ Alan Adams 23

Scenario 13 DDR w/ Settlements *Scenario document on spp. org here spp. org >

Scenario 13 DDR w/ Settlements *Scenario document on spp. org here spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents > Scenarios in Parallel Ops Results • Pass • Control Mode was not valid coming from MP. SPP overrode the value and DDR was dispatched appropriately • DDR Cleared 2 MW as expected and MP received correct dispatch instructions vial XML and ICCP • MP overrode the actual load measurement to simulate a drop in load and SPP correctly calculated the new generation value and fed it back to STLF Alan Adams 24

Dispatch Simulator – Preliminary results • *Scenario document on spp. org here –spp. org

Dispatch Simulator – Preliminary results • *Scenario document on spp. org here –spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents > Scenarios in Parallel Ops 0: 00 12/06 to 23: 59 12/09 • SPP did not run any COP True-ups • Commitments came from DBDA, DA Market, DARUC, IDRUC, and MP Self Commitments – MPs should have received “Production-like” commitment notifications at expected timeframes • SPP will overwrite meter settlement data for the four Operating days Dec 6, 7, 8, and 9 with RTBM setpoint instructions Alan Adams 25

Dispatch Simulator – Preliminary results • There was an issue with not all schedules

Dispatch Simulator – Preliminary results • There was an issue with not all schedules being propagated to the RUC process. – Around 1000 Mw of imports was not accounted for which caused an extra 1000 MW of capacity to be committed out of the RUC process. – This did not affect the test other than SPP carried an extra margin of around 1000 MW during the test. Alan Adams 26

Dispatch Simulator Test EIS vs IM Capacity Dec 6, 1300 - 2200 Alan Adams

Dispatch Simulator Test EIS vs IM Capacity Dec 6, 1300 - 2200 Alan Adams 27

Dispatch Simulator Test EIS vs IM Capacity Dec 7, 1300 - 2200 Alan Adams

Dispatch Simulator Test EIS vs IM Capacity Dec 7, 1300 - 2200 Alan Adams 28

Dispatch Simulator Test EIS vs IM Capacity Dec 8, 1300 - 2200 Alan Adams

Dispatch Simulator Test EIS vs IM Capacity Dec 8, 1300 - 2200 Alan Adams 29

Dispatch Simulator Test EIS vs IM Capacity Dec 9, 0100 - 2300 Alan Adams

Dispatch Simulator Test EIS vs IM Capacity Dec 9, 0100 - 2300 Alan Adams 30

 • Scenario Tracker as of Dec 13 Summary – Scenarios in Parallel Ops:

• Scenario Tracker as of Dec 13 Summary – Scenarios in Parallel Ops: – • 4 Scenarios and 2 Partial Scenarios Executed successfully to date: § Scenario 7 Must Offer pending Settlements results § Scenario 28 a Successful § Scenario 15 Commitment Mismatch Part 2 Successful § Scenario 24 Successful § Simulator Test Successful § Scenario 13 DDR w/ Settlements Successful Summary – Retests Pending: – – 2 entire Scenario retests pending § Scenario 11 scheduled for Jan 17 § Scenario 31 BDR w/ Settlements to be retested Jan 14 after R 1. 9++ goes to PROD Dec 30 13 individual retest requests pending Ginny Watson 31

Announcements R 1. 9 goes to PROD on December 30 web. Mirroring off on

Announcements R 1. 9 goes to PROD on December 30 web. Mirroring off on December 16 Ramp Sharing is turned on and turned off every other week Process to Register Multiple Listeners • Currently, MPs cannot register secondary listeners in the UI themselves. • Please submit an RMS ticket and SPP will register it for you – Use the “Application/Technical Support” Quick Pick • It will likely be post go live before that functionality is added in the UI and API. Ginny Watson 32

Look-Ahead: Dec 16 - 22 Parallel Ops Calendar on spp. org > Integrated Marketplace

Look-Ahead: Dec 16 - 22 Parallel Ops Calendar on spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents • Settlements details on the Settlements Calendar marketplace. spp. org > Public > Settlements > Calendars Sun Dec 15: IDT Call 11: 30 A – 5: 30 P 4 hr IDT begins at 12: 30 P Mon Dec 16: Scenario 29 RSG IDT 12/15 Debrief and 12/17 Prep Call 3: 00 P Tues Dec 17: IDT Call 2: 30 A – 8: 30 A 4 hr IDT begins at 3: 30 A Scenario 11 EDR Wed Dec 18: IDT 12/17 Debrief and 12/19 Prep Call 3: 00 P Thurs Dec 19: IDT Call 5: 30 A – 9: 30 A 2 hr IDT begins at 6: 30 P Dec 20: Parallel Ops Friday Meeting and IDT 12/19 Debrief 9: 00 A Ginny Watson 33

Schedule Dec 23 - 29 Dec 24: SPP Holiday Dec 25: SPP Holiday Ginny

Schedule Dec 23 - 29 Dec 24: SPP Holiday Dec 25: SPP Holiday Ginny Watson 34

Scenario 11 EDR *Scenario document on spp. org here spp. org > Integrated Marketplace

Scenario 11 EDR *Scenario document on spp. org here spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents > Scenarios in Parallel Ops Scenario Description • Demonstrate the clearing and dispatch of an EDR MP Activities • Review commit status and offer parameters in the Scenario Document • Submit an Eco Min and Eco Max for the EDRs. Limits will be used for Normal Min and Normal Max, Emergency Min and Emergency Max, and Reg Min and Reg Max. • EDRs not on outage must also have an accompanying Dynamic Schedule crated from the Source BA to the SPP BA submitted via RTOSS with a registered source that is associated with the EDR Resource Settlement Location. If the EDR offers with a min less than 0, a Dynamic Schedule must also be created sourcing from the SPP BA to the Sink BA • Send a Control Status and the MW response of the EDR via ICCP the same as any other Resource. • Reserve Offers should be submitted sufficiently low enough or with a Fixed Dispatch Status in order to ensure clearing in the DA Market and RTBM • For the OD reserved for EDR testing, MPs should submit offer parameters and schedules to test different options Alan Adams 35

Scenario 29 Reserve Sharing Group *Scenario document on spp. org here spp. org >

Scenario 29 Reserve Sharing Group *Scenario document on spp. org here spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents > Scenarios in Parallel Ops Scenario Description • SPP Reserve Sharing System (RSS) events are required whenever there is a loss of generation above 50 MW (pending 200 MW limit change) within SPP Reserve Sharing Group (RSG) footprint. MP Activities *SPP MPs with Resources • SPP Members will see CR Deployments based on these submitted events • Please keep in mind that no action should be taken on these CRDs • The focus of the scenario is to test the RSS System and to ensure that Reserve Share Schedules flow correctly to each RSG Member • SPP and each RSG Member will validate this so no action needs to be taken by MPS Alan Adams 36

Non-Functional Testing *Scenario document on spp. org here spp. org > Integrated Marketplace >

Non-Functional Testing *Scenario document on spp. org here spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents > Scenarios in Parallel Ops Scenario Description • SPP will simulate an outage at the Chenal facility for Reliability and Markets redundancy • Non-Functional Test includes critical Tier 0 and Tier 1 components to demonstrate that those systems can be entirely supported and function independently in the back-up Operations Data Center in Maumelle MP Activities • MPs should continue normal operations with no expected downtime. • If any unexpected results occur, please submit a detailed RMS Ticket here • Verify the following during Non-Funct Testing Dec 13 at 10 P – Dec 14 at 12 A CPT • ICCP setpoint instructions are received • XML notifications are received • Marketplace Portal User Interfaces and API’s are available for data entry and retrieval Casey Cathey 37

IDT Sunday 12 -15 -13 IDT Call 11: 30 A – 5: 30 P

IDT Sunday 12 -15 -13 IDT Call 11: 30 A – 5: 30 P 4 hr IDT begins at 12: 30 P • No Prep Call on Sat 12/14 • IDT Debrief call Mon 12/16 at 3 P Roll Call for Sun 12/15 IDT at 11: 30 A CJ Brown / Ginny Watson 38

IDT 12 -17 -13 IDT Call 2: 30 A – 8: 30 A 4

IDT 12 -17 -13 IDT Call 2: 30 A – 8: 30 A 4 hr IDT begins at 3: 30 A • IDT Prep Call Mon 12/16 at 3 P • IDT Debrief Call Wed 12/17 at 3 P • IDT Conditions: • Web. Mirroring Off • EDR CJ Brown 39

IDT 12 -19 -13 IDT Call 5: 30 A – 9: 30 A 2

IDT 12 -19 -13 IDT Call 5: 30 A – 9: 30 A 2 hr IDT begins at 6: 30 A • IDT Prep Call Wed 12/17 at 3 P • IDT Debrief Call Fri 12/20 at 9 A • IDT Conditions: • RSG Changes • Pseudo Tied In Resources CJ Brown 40

Migration and Maintenance Report posted to spp. org > Integrated Marketplace > Market Trials

Migration and Maintenance Report posted to spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents Wendy Reynolds 41

Release Scorecard posted to spp. org: spp. org > Integrated Marketplace > Market Trials

Release Scorecard posted to spp. org: spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents Ginny Watson 42

Known Issues Log • Please review the Market Trials Known Issues Log on spp.

Known Issues Log • Please review the Market Trials Known Issues Log on spp. org > Integrated Marketplace > Market Trials > Structured/Unstructured Test > Integrated Marketplace Market Trials Project Documentation Folder Issue # Component Severity Synopsis 866 Markets Systems BDRs are not pushing the correct control mode to the market. Severity 2: High 857 Markets Systems DVER fixed max limit while in follow forces dispatch down when in the money Severity 2: High Unable to view or submit data via the Markets UI or API for the 7 th future Operating Day. The UI/API allows users to view and edit six days into the future, but not on the seventh day. The code should be modified to allow the seventh operating day in the future to be accessed (viewed/edited) via the UI/API. Severity 3: Medium 855 (RMS Markets UI 6971) Carrie Simpson 43

Market Trials Useful Resources Integrated Marketplace Parallel Operations Project Documents spp. org > Integrated

Market Trials Useful Resources Integrated Marketplace Parallel Operations Project Documents spp. org > Integrated Marketplace > Market Trials > Parallel Operations Test> Integrated Marketplace Parallel Operations Project Documents • • • Parallel Ops Presentations, Statistics, MP Meetings Folder Parallel Ops Reference Documents Folder Scenarios in Parallel Ops Folder > Scenarios Schedule Parallel Ops Communication Plan Parallel Ops MP Guide Parallel Ops Calendar Known Issues Log Release Scorecard Migration Report Integrated Deployment Test (as part of Parallel Ops) spp. org > Integrated Marketplace > Market Trials > Integrated Deployment Test • • IDT MP Guide Parallel Ops Calendar Additional documents: • Settlements Calendar marketplace. spp. org > Public > Settlements > Calendars • Change Tracker spp. org > Integrated Marketplace > Marketplace Documents • Integrated Marketplace Structured/Unstructured Test Project Documents spp. org > Integrated Marketplace > Market Trials > Structured / Unstructured Test> Integrated Marketplace Structured / Unstructured Test Project Documents 44

Market Trials Useful Resources Scheduling Information spp. org > Integrated Marketplace > Market Trials

Market Trials Useful Resources Scheduling Information spp. org > Integrated Marketplace > Market Trials > Integrated Deployment Test • Interchange Scheduling Reference Manual http: //www. spp. org/publications/Interchange%20 Scheduling%20 Reference%20 Manual%20 Published%202 -1 -2013. doc • MT Tagged Transactions and Tag Validation Criteria http: //www. spp. org/section. asp? group=2802&page. ID=27 • Ramp Reservation User Guide http: //www. spp. org/section. asp? group=2479&page. ID=27 45

What is the Official Date for End of Tag Mirror? • SPP will turn

What is the Official Date for End of Tag Mirror? • SPP will turn off the Tag mirror/copy process on December 15 th after the deployment test is complete. • SPP staff will zero out any tags that were previously copied from Real Time for the remaining operating days of December. • A few MP have already begun entering test tags. These will not be zeroed. These will be kept in the system. • MP will begin entering tags into the IM ITE E-Tag (Testing tagging) system on December 16 th. Shari Brown 46

What tags should MP begin entering into the E-TAG Test System ? • •

What tags should MP begin entering into the E-TAG Test System ? • • Import Tags into the SPP CBA (Required) – Consist of Load Serving Contractual obligations for SPP BA – Necessary for realistic NSI Export Tags out of the SPP CBA (Required) – Consist of Load Serving Contractual obligations for neighboring BA – Necessary for realistic NSI • Internal SPP CBA tags (not allowed) • Wheel/Through (not required/optional) Shari Brown 47

Why will MP be entering into the E-TAG Test System ? • To provide

Why will MP be entering into the E-TAG Test System ? • To provide Market Participants the opportunity to enter completely accurate Interchange transactions for participation in the IM testing processes. • Market Participants need to have their PSE physically enter the required tags in order to ensure proper understanding of the new Business Rules for IM and CBA • Many tags are monthly so even starting this early, the PSE will only have a couple of tag creation periods before they are expected to enter tags for Go-Live in late February. Shari Brown 48

How do I know if I need to Enter Test Tags? • Most Market

How do I know if I need to Enter Test Tags? • Most Market Participants have participated in Market Training and testing Processes and are aware which tags will be used for IM and CBA • SPP staff will be monitoring the test tag submissions and will be contacting MP from December 18 th through December 31 st to notify them of required tags that have not been entered. • A list of normal Firm load serving transactions has been developed and the responsible MP identified. The list is included in the Meeting Information for Today. Shari Brown 49

How will I know if I am entering the Test Tag correctly? • SPP

How will I know if I am entering the Test Tag correctly? • SPP Operators will evaluate for new Business Practices: – Accurate Transmission Service – Accurate GCA and LCA for “SWPP” BA – Accurate Marketplace Protocols evaluation – Ramp Availability – Accurate Physical Path – Must include SWPP Transmission Service **** • SPP operators will deny test tags that do not comply and will add a comment for the reason of denial • SPP will have additional staff on the SPP Tariff/Interchange (TI) desk to assist MPs and explain needed corrections. Shari Brown 50

How will I know if I am entering the Test Tag correctly? *Continued* •

How will I know if I am entering the Test Tag correctly? *Continued* • MP/PSE should call the Tariff/Interchange (TI) desk for any concerns with the denial for the test tag. (TI Desk # 501 -614 -3588) • If you have questions about entering test tags, do not enter RMS tickets, it will delay the process of getting your test tag completed and approved • Do not Panic. Please be patient. This is not Real Time operations. If it takes time to get your test tag correct Parallel Operations will not be hindered by our work on your test tag. Shari Brown 51

How will I know if I am entering the Test Tag correctly? *Continued* •

How will I know if I am entering the Test Tag correctly? *Continued* • Some test tag issues and errors will take time to remedy. • The biggest current issue are Import or Exports that do not have SWPP Transmission Service. (GFA trans) – SPP has already been working with MP to get SWPP TSR on all tags – SPP has already created over 70 new TSR in preparation – SPP cannot always see these transactions and been notified of additional tags that need SWPP TSR – Operators cannot create these TSR so the MP will need to work with SPP Analysts as needed Shari Brown 52

Impacts on Market Participants • From December 16 through December 23 MP will work

Impacts on Market Participants • From December 16 through December 23 MP will work on getting the required tags in the Test Tag environment for December Operating Days • MP will be contacted if the required tags have not been entered and will work with SPP. • From December 26 th through December 31 st MP will work on getting the required tags entered for the January Operating Days. • MP will be contacted if the required tags have not been entered and notified to get them entered before December 31 st. Shari Brown 53

Impacts on Market Participants (CONT. ) • For the month of January MP will

Impacts on Market Participants (CONT. ) • For the month of January MP will essentially be “dual” enter tags Daily tags. One tag in EIS Real time and an additional tag in IM Test Tag. • If the tag is entered for the entire month of January, the work is done. • Test Tags do not have to be updated for real time. • An average contractual mw profile will be adequate. • Additional Tags for Sales of Opportunity are encouraged for Financial Market • MP encouraged to compare EIS and IM settlement statements for strategy planning Shari Brown 54

Additional Communications • SPP will be contacting individual MP on the status of their

Additional Communications • SPP will be contacting individual MP on the status of their Test tags. • SPP will make additional announcements over the next few weeks about the status of the individual MP activities. • Remember that not all Import and Export tags are created by SPP Market Participants, so those tags will need to be entered by responsible MP for their test settlement statements to be correct. • OATI E-Tag Users: Test Tag URL: - https: //demo 1. spp. tag. oati. com/ Shari Brown 55

Market Participants Responsible for Test Tags *****These numbers are approximated for this table only

Market Participants Responsible for Test Tags *****These numbers are approximated for this table only to give a general idea of the tag (transaction) entry impact to each MP**** MP/PSE CODE Firm Transactions FIRM Hydro non-Firm TOTAL 1 GRDX 10 6 16 2 KCPS 14 14 3 WRSG 12 1 2 15 4 MEAN 7 4 11 5 AEPM 8 5 13 6 MEUC 7 3 10 7 NPPM 6 6 8 OMPA 3 1 4 9 WFES 2 2 4 10 UGPM 4 4 11 OPPM 3 3 12 REMC 3 3 13 TNSK 2 7 9 14 AECC 1 1 2 15 EXGN 1 1 2 16 KPP 2 2 17 LESM 1 1 2 18 OGEPS 2 2 20 SPRM 2 2 21 UPLS 2 2 22 BEPM 2 2 23 FR 2 W 1 1 24 KBPU 1 1 25 KMEA 1 1 26 KEPC 1 1 27 SPSM 1 1 29 WRNM 1 1 30 PLWC 1 1 32 ENDU 4 4 33 CRGL 1 1 34 FPLP 3 3 35 TEAO 5 1 6 TOTAL 101 26 22 149 Shari Brown 56

Erin Jester READINESS METRICS UPDATE

Erin Jester READINESS METRICS UPDATE

Readiness Metrics Overview • Total Number of Readiness Metrics: 50 • Number Complete (Blue):

Readiness Metrics Overview • Total Number of Readiness Metrics: 50 • Number Complete (Blue): 32 • Number on Track (Green): 10 • Number at Risk (Yellow/Red): 6 • Number Not Started (Gray): 2 Color Description Grey Monitoring of the metric has not begun or not applicable Blue The metric success criteria has been met and no longer requires monitoring, and/or the metric has been completed by the target due date Green The metric is meeting identified success criteria, and/or The metric is on schedule to meet the target due date Yellow The metric has not met the highest level of the identified success criteria, and/or The metric is at risk not meeting the identified target due date Red The metric has not met the highest or mid-range success criteria as defined, and/or the metric has not been completed by the specified target due date 58

Complete Readiness Metrics • There are 32 “Complete” Readiness Metrics out of 50 –

Complete Readiness Metrics • There are 32 “Complete” Readiness Metrics out of 50 – 23 new metrics have been completed since last CWG Checkpoint § • 17 Operations Metrics, 1 Internal SPP Metric, 5 Settlements Metrics The Readiness workstream will continue to track applicable Metrics throughout Parallel Ops – For example, completion criteria for many of the Operations metrics has been met, but the metrics will continue to be tracked on a daily basis 59

Complete Operations Readiness Metrics ID Metric Name Status OPS-02 State Estimator Availability B OPS-03

Complete Operations Readiness Metrics ID Metric Name Status OPS-02 State Estimator Availability B OPS-03 State Estimator Accuracy B OPS-04 STLF Accuracy B OPS-05 a MTLF - 4 AM ID RUC B OPS-05 b MTLF - 12 PM ID RUC B OPS-05 c MTLF - 5 PM DA RUC B Variance State Estimator has ninety-seven percent (97%) availability of five (5) minute solutions for 7 consecutive days during Parallel Ops State Estimator solves 90% of the time with Mismatch level lower than 50 MW for 7 consecutive days during Parallel Ops Short term load forecast for the SPP market footprint used for RTBM for conforming loads solves 97% of time with +/- 1% accuracy for seven consecutive days during Parallel Ops 4 am Mid-term load forecast for conforming loads for the SPP market footprint solves 97% of time with +/- 5% accuracy for seven consecutive days during Parallel Ops 12 pm Mid-term load forecast for conforming loads for the SPP market footprint solves 97% of time with +/- 3% accuracy for seven consecutive days during Parallel Ops 5 pm Mid-term load forecast for conforming loads for the SPP market footprint solves 97% of time with +/- 3% accuracy for seven consecutive days during Parallel Ops 60

Complete Operations Readiness Metrics ID Status Variance B Wind forecast for SPP Market footprint

Complete Operations Readiness Metrics ID Status Variance B Wind forecast for SPP Market footprint solves with +/- 15% accuracy for six out of seven consecutive days during Parallel Ops OPS-06 b Wind Forecast Accuracy - RTBM B Wind forecast for SPP Market footprint solves with +/- 10% accuracy for six out of seven consecutive days during Parallel Ops OPS-06 c Wind Forecast Accuracy - DA RUC B Wind forecast for SPP Market footprint solves with +/- 20% accuracy for six out of seven consecutive days during Parallel Ops Market Flow Calculations successful ninety-seven percent (97%) of 5 minute solutions for seven consecutive days during Parallel Ops OPS-06 a Metric Name Wind Forecast Accuracy - Pre RTBM OPS-07 Market Flow Calculations Availability - Current Hour B OPS-08 Market Flow Calculations Availability - Next Hour B OPS-09 DA Market Results Posted B Market Flow Calculations successful ninety-seven percent (97%) of 5 minute solutions for seven consecutive days during Parallel Ops Day-Ahead Market results are posted every day by 4 pm for 5 consecutive days during Parallel Ops 61

Complete Operations Readiness Metrics ID Metric Name Status Variance OPS-10 Generate DA LMPs and

Complete Operations Readiness Metrics ID Metric Name Status Variance OPS-10 Generate DA LMPs and Settlement Point Prices B LMPs are made available for each settlement location every day by 4 pm for 5 consecutive days during Parallel Ops OPS-11 RTBM Calculation Availability B Real-Time Calculations successful ninety-nine percent (99%) of five (5) minute solutions for 7 consecutive days during Parallel Ops OPS-12 Operating Reserve Requirements B OPS-13 Regulation Pre-Certification B OPS-14 ACE Performance Accuracy B Operating Reserves are calculated and posted on time for 7 consecutive days during Parallel Ops All MP identified regulation resources have completed the Regulation Pre-Certification The 5 min rolling average delta between the SPP ACE and the SUM of the individual EIS BA ACE values is within a band defined as the greater of +/- 20 MW and +/- 10% of the corresponding Average Sum of the individual SPP BA ACE for 7 days 62

Complete Internal SPP Readiness Metric ID INT-04 Metric Name SPP Operations Training Status Variance

Complete Internal SPP Readiness Metric ID INT-04 Metric Name SPP Operations Training Status Variance B 96% of SPP Operations staff have no incomplete training requirements within 30 days of due date 63

Complete Settlements Readiness Metrics ID Metric Name Status STL-01 Access & Download Statements B

Complete Settlements Readiness Metrics ID Metric Name Status STL-01 Access & Download Statements B STL-02 Access & Download Invoices B STL-03 Submit Meter Data B STL-05 Submit Dispute B STL-06 Settlement Statement Timeliness B STL-07 SPP Validation of Settlement Charge Types B Variance 100% of Target Mass were able to successfully download a Statement 100% of Target Mass were able to successfully download an Invoice 100% of meter agents successfully submit meter data for each of their assets 100% of Target Mass successfully submitted a dispute Settlement Statements are posted by end of day for 7 consecutive calendar days during Market Trials, including Initial and Final 100% of Charge Types with calculated results were utilized by Target Mass. Charge Types with calculated results not utilized by Target Mass are explainable. 64

At Risk (Yellow) Readiness Metrics ID CBA-01 OPS-17 INT-02 Metric Name CBA Certification Activities

At Risk (Yellow) Readiness Metrics ID CBA-01 OPS-17 INT-02 Metric Name CBA Certification Activities ICCP Model Validation Organizational Staffing Readiness Status Y Y Y Variance This metric has a defined milestone date of October 1, 2013 for EIS BAs to provide Operating Protocol data to SPP; that milestone is past due. The CBA team is working to get this complete and anticipates receiving all data by January 1, 2014. 2/41 of MPs ICCP models are not validated due pending data checkout and waiting for MP response. SPP is actively working with those MPs. There 2 open Marketplace positions ; one expected fill date of 12/16 and the other fill date is TBD. SPP HR and the applicable department are actively working on filling the “TBD” position. 65

At Risk (Red) Readiness Metrics ID OPS-15 OPS-18 Metric Name CROW Outage Submission ICCP

At Risk (Red) Readiness Metrics ID OPS-15 OPS-18 Metric Name CROW Outage Submission ICCP Control Status Tracking Status R R Variance 24/25 of resource-owning MPs who have reported outages have not reported them through SCADA and CROW within 30 minutes. Communications are being sent to MPs directly by the outage team. There will be an update from Jason Smith at the December CWG. 30% of the MPs are submitting correctly 100% of the time. Overall, 93% of all MP's control modes submitted were correct between 11/18 and 11/29. There will be an update at the December CWG from Tim Miller. TRL-01 Market Trials Participation R < 80% of Target Mass is participating in Market Trials for the month of November. One BA (KBPU) did not fully participate. Readiness has reached out to the MPs that are not participating. 66

Not Started Metrics • ID The following two metrics have not started tracking Metric

Not Started Metrics • ID The following two metrics have not started tracking Metric Name Status Variance Finalizing data collection - metric will be published in mid-December. OPS-16 MP Response to Deployment Testing NS TCR-04 TCR Production - Two Monthly Auctions NS Readiness will be working with Operations to begin a supplement process to analyze if the MPs are completely and consistently responding to dispatch. Metric will start according to planned TCR Timeline 67

MP Certification Form Process REMINDER - Completion of the MP Readiness Certification is an

MP Certification Form Process REMINDER - Completion of the MP Readiness Certification is an official Readiness Metric filed with FERC (RGL-03) • Please return the form signed by an authorized signatory of your company as a hard copy by December 20, 2013 – • A scanned copy can be emailed in order to make the deadline; a hard copy must be sent after As a part of the FERC Readiness Filing scheduled for 12/27/13, Bruce Rew, Executive Sponsor, will also be signing a similar letter expressing SPP’s intent to be ready for Go. Live on March 1, 2014 68

Erin Jester PARALLEL OPS EXIT & MARKETPLACE GO LIVE CRITERIA

Erin Jester PARALLEL OPS EXIT & MARKETPLACE GO LIVE CRITERIA

Parallel Operations Exit and Go-Live Criteria • Feedback via RMS was requested by December

Parallel Operations Exit and Go-Live Criteria • Feedback via RMS was requested by December 6, 2013 – No MP feedback was received • The posted criteria are no longer considered DRAFT • Posted on Readiness Center in the Go/No-Go Criteria Documents folder 70

Parallel Ops Exit Criteria • 28 Parallel Operations Exit Criteria Total – 22 complete

Parallel Ops Exit Criteria • 28 Parallel Operations Exit Criteria Total – 22 complete § 21 are Readiness Metrics § 1 is a roll-over from the Parallel Ops Go/No-Go Criteria (PO-12: Charge Type Verification) – 2 tracking on time § 1 is a roll-over from the Parallel Ops Go/No-Go Criteria (PO-03: 100% Scenarios executed successfully) – 3 at risk (see next slide) – 1 tracking not started § 7 consecutive days of Bid to Bill 71

‘At Risk’ Parallel Ops Exit Criteria • ID OPS-15 OPS-17 The 3 ‘at risk’

‘At Risk’ Parallel Ops Exit Criteria • ID OPS-15 OPS-17 The 3 ‘at risk’ P Ops Exit Criteria are 3 of the yellow/red Readiness Metric Name CROW Outage Submission ICCP Model Validation Status R Y Variance 24/25 of resource-owning MPs who have reported outages have not reported them through SCADA and CROW within 30 minutes. Communications are being sent to MPs directly by the outage team. There will be an update from Jason Smith at the December CWG. 2/41 of MPs ICCP models are not validated due pending data checkout and waiting for MP response. SPP is actively working with those MPs. OPS-18 ICCP Control Status Tracking R 30% of the MPs are submitting correctly 100% of the time. Overall, 93% of all MP's control modes submitted were correct between 11/18 and 11/29. There will be an update at the December CWG from Tim 72 Miller.

Go-Live Criteria • 21 Go-Live Criteria Total – 16 tracking on time – 5

Go-Live Criteria • 21 Go-Live Criteria Total – 16 tracking on time – 5 not started tracking § Includes activities that will be complete closer to go-live (e. g. publishing issues/risks, work-arounds for high defects, etc. ) • Latest status of criteria posted in the Go/No-Go Folder on SPP. org 73

Go-Live Criteria • 21 Go-Live Criteria Total – 16 tracking on time – 5

Go-Live Criteria • 21 Go-Live Criteria Total – 16 tracking on time – 5 not started tracking § Includes activities that will be complete closer to go-live (e. g. publishing issues/risks, work-arounds for high defects, etc. ) • Latest status of criteria posted in the Go/No-Go Folder on SPP. org 74

Notable Go-Live Criteria • Testing Criteria is tracking on time – • GL-04 to

Notable Go-Live Criteria • Testing Criteria is tracking on time – • GL-04 to GL-08 are testing criteria that were rolled over from the Parallel Ops Go/No-Go Criteria 30 -day System Freeze (GL-03) is tracking on time – The freeze is scheduled to begin as planned (12/31/13 for Markets and 1/10/14 for Settlements) – SPP received 1. 9. 1 on 12/11 and is in the process of reviewing. We are aware of some Missing SPRs that are needed for Go-Live and will be evaluating what the next steps are. An update will be provided at the CWG next week in the Program update (or the System Release update). 75

December CWG Meeting • Reminder: December CWG meeting is two days – Wednesday, 12/18

December CWG Meeting • Reminder: December CWG meeting is two days – Wednesday, 12/18 and Thursday, 12/19 § Wednesday scheduled to 6: 00 p. m. § Thursday scheduled to 5: 00 p. m. 76