STEREOSECCHI Early Operations Lessons Learned and Lots of

STEREO/SECCHI Early Operations - Lessons Learned and Lots of Images 5 th SECCHI Consortium Meeting March 5, 2007 Orsay, France Nathan Rich - NRL/Interferometrics Inc. (Nathan. rich@nrl. navy. mil) Lynn Mc. Nutt - NRL/Interferemotrics Inc. SECCHI Consortium 3/5/07 - GDS 1. 1

How do I …? • One Source for all your SECCHI information needs: http: //secchi. nrl. navy. mil/wiki/ SECCHI Consortium 3/5/07 - GDS 1. 2

Early Operations At A Glance • Launch (061025) • Functional Tests A (061106, pzt 061128); B (061109 -10, pzt 061128) • Calibration Events and Outcomes – Closed-door Calibration – Dark, LED – All decontamination heaters off by 061128 – GT - use by S/C enabled (061112, 061113) - calibration A (061205 -6, 14, 26) B (061209, 10, 16, 20, 21) – First Light - SCIP-A 061204 - SCIP-B 061213 - HI-A 061213 - HI-B 070111 SECCHI Consortium 3/5/07 - GDS 1. 3

Calibration Events and Outcomes (2) • Offpoints – GT-controlled Offpoints - Description: GT bias induced via table in EEPROM; up to 16 separate tables loaded via scheduled command - Dates: A (061206, 061214, 061221, 061226, 070112, 070203, 070213, 070221); B (061216, 061221, 061226, 070203, 070204) – S/C-controlled Offpoints - Solar Power testing: A (061211 13: 55); B (070102 18: 20) - IMPACT STE -2 to +2 deg: B (070105 16: 55 - EUVI, COR 1, COR 2 incidental) - SECCHI Flat-field calibration: A (070105 15: 55: COR doors did not close because scheduled door ops not enabled, S/C did greater than expected offpoint; 070215 08: 00) B (070205 05: 15) - HI straylight: A (070110 22: 00, 070130 16: 00) B (070113 08: 10, 070125 21: 15) SECCHI Consortium 3/5/07 - GDS 1. 4

Calibration Events and Outcomes (3) • Rolls – MAG Smooth Calibration Rolls (15 deg/min x 4): - Dates: A (061120 14: 20, 061222 09: 30); B (061120 18: 25, 061129 21: 00 [SECCHI off], 061212 11: 00) – SWAVES Smooth Calibration Rolls (6 deg/min): - Dates: A (061218 06: 50 -16: 50, 061220 07: 20, 061223 04: 30); B (061219 02: 35, 070129 05: 10, 070131 04: 50) – Stepped Roll (Dwell at 90, 120, 180, 240, 270, 300 & 360 deg): - Dates: A (070220 14: 25); B (070206 06: 50) – HI calibration A and B 90/270 and 180 deg: - Dates (070313, 070315 TBC) • Lunar Flybys (images for A only) – 061215 21: 59 - Series of images of dark side of the moon in HI-2. • Lunar Transit (B only) – 070225 07: 00 -18: 00 (rehearsal 070214) SECCHI Consortium 3/5/07 - GDS 1. 5

Calibration Events and Outcomes (4) • Image Processing Tests: short term onboard software tests – A (070119, 070125, 070201) • New tables implemented – Exposure: A (061115, 061121, 061127, 061129, 061204, 061205, 061221, 061222, 070119, 070221, 070227); B (061115, 061122, 061123? , 061129, 061205, 061221, 070219, 070228) – Image Processing: A (061118? , 061221, 070104, 070125? , 070202? , 070301); B (061118? , 061221, 070103, 070228) - Error in imagetbl. img 061221 caused images to be incorrectly sent down realtime channel, causing lots of buffer overrun errors and skipped images - FSW Bug 103 (open) caused images on B (070302 -070303) to have some problems – Event-detect Threshold: A (070119, 070201, 070301) B (070216, 070228) – Icer Wavelet: A (070119, 070201); B (061221? ) • SSR 2 testing: A (070113 ff) SECCHI Consortium 3/5/07 - GDS 1. 6

Calibration Events and Outcomes (5) • Mechanism Tuning – Description: multiple operations at different Delay settings to determine optimum setting; use encoder position or exposure duration – Dates: A (061130, 061219 -21 ); B (061130, 061219 -20) • FSW update to A (5. 01 061218, 5. 01. 05(IP) 070301); B (5. 01 061226, 5. 01. 05(IP) 070228) • SECCHI SSR 1 filled week 47 of 2006; some data lost as a result – Contributing factor: incorrect compression predictions – Resolution: more conservative scheduling • Watchdog Timer Reset on SECCHI Ahead 070125 04: 37 Z – Response: auto reboot to MAINT mode - transition to OPS at 19: 30 Z – Cause is not determined – Impact to data and operations - S/C G&C configured to not use rate component of GT data on 070127 - current status unknown (by me) SECCHI Consortium 3/5/07 - GDS 1. 7

Day-to-Day Operations Nominally from NRL: SECCHI Consortium 3/5/07 - GDS 1. 8

ITOS/POC Status • • Telemetry – POC workstations are configured to connect automatically at each pass, currently 4 hrs/day – We had problems getting ITOS working on web-servers so telemetry web pages could be made available outside of NRL: 1. Unreliable connecting to realtime telemetry socket from NRL (connection was not persistent) - Workaround: proc can reconnect at each pass (TBD) 2. We are generating telemetry products (S/W Events, page snaps) that are in an http-accessible area Commanding – Procedures are in place to upload commands and/or files to MOC before pass (dailyload. proc) and monitor execution of commands during pass (verifydailycmds. proc) - Soon, commanding during pass only upon advance request – Some tweaking of procs required to account for increased lag time between sending of command confirmation of receipt SECCHI Consortium 3/5/07 - GDS 1. 9

Data Processing Status • It is taking a while to get all the steps in sync for LZ image processing (file retrieval, playback, database update, secchi_reduce) – Database update still needs to be better implemented to minimize delays – Two new sun workstations are in place as of 2 Mar 07 to do telemetry playback and pipeline processing, which should significantly reduce processing times. • Linux compatability: – Best option for speed – Has been implemented for all steps except HI 32 -bit images • Bugzilla being used to track problems, tasks and enhancement requests for the ground system and flight software (report following) • All data to date will be re-processed after majority of bugs are fixed and currently planned enhancements completed – Target is April 2007 – Temporary workarounds for most problems should be part of secchi_prep SECCHI Consortium 3/5/07 - GDS 1. 10

Ground System Issue Status • http: //secchi. nrl. navy. mil/bugzilla Bug/Task Summary FSW ITOS/POC Pipeline Processing PT WWW Analysis • Detailed Bug List (pdf) SECCHI Consortium 3/5/07 - GDS 1. 11 OPEN 3 1 11 0 0 4 Res/Ver /Closed 20 2 36 2 1 6

SECCHI Image Statistics • As of 23 Feb 2007 (non-spaceweather) Cor 1 A Cor 1 B Cor 2 A Cor 2 B Euvi. A Euvi. B Hi 1 A Hi 1 B Hi 2 A Hi 2 B N Images 13439 16035 8051 8354 23924 27696 1207 846 1149 707 Size (Raw GB) 9. 51 9. 78 4. 97 4. 96 5. 98 6. 79 4. 07 2. 76 3. 85 1. 86 Size (FITS GB) 26. 11 35. 60 63. 38 60. 92 192. 40 211. 37 5. 88 4. 27 5. 55 3. 39 Avg Comprssn 2. 7 3. 6 12. 8 12. 3 32. 2 31. 1 1. 44 1. 55 1. 44 1. 82 SECCHI Consortium 3/5/07 - GDS 1. 12

SECCHI-A Compression Averages SECCHI Consortium 3/5/07 - GDS 1. 13

SECCHI-B Compression Averages SECCHI Consortium 3/5/07 - GDS 1. 14

SECCHI-A Number of Images SECCHI Consortium 3/5/07 - GDS 1. 15

SECCHI-B Number of Images SECCHI Consortium 3/5/07 - GDS 1. 16

SECCHI-A Cumulative Volume SECCHI Consortium 3/5/07 - GDS 1. 17

SECCHI-B Cumulative Volume SECCHI Consortium 3/5/07 - GDS 1. 18

SECCHI-A Daily Image Telemetry Volume SECCHI Consortium 3/5/07 - GDS 1. 19

SECCHI-B Daily Image Telemetry Volume SECCHI Consortium 3/5/07 - GDS 1. 20

Lifecycle Of A SECCHI Image 5 th SECCHI Consortium Meeting March 5, 2007 Orsay, France Nathan Rich - NRL/Interferometrics Inc. Lynn Mc. Nutt - NRL/Interferemotrics Inc. Ed Esfandiari - NRL/Interferometrics Inc. Simon Plunkett - NRL SECCHI Consortium 3/5/07 - GDS 1. 21

Overview of Pre-Image Information Flow SECCHI Consortium 3/5/07 - GDS 1. 22

Pre-Observation Ground Events Our Example: 20070129_224540_s 4 eu. A. fts • (T = DATE-CMD) • Pre-Observation Ground Events – T-3+ days Define OS_1343 - $PT/PRO/schedule. pro – T-3+ days Define schedule file: - OS_1343 summary: - 1614 2007/01/20 22: 45: 40 EUVI 2000 2176 2112 HG 1 x 1 SCIP Seq 1/2 1343 (IP 89: SSR 1, ICER 6=40. 00) FPS S 1 195 A - OS_1343 in uploaded RTS file: - ---00224440 S 442 F 113433189100000120110002 ---– T-2+ days send schedule file to instrument - rtsfile= ”PT/flight/commissioning/A/synoptic/720/070120/SEC 20070120000. IPT. RTS” - start uploadfile(rtsfile, ”temp/07012000. sch”) - /FMCOPY from= ”temp/07012000. sch”, to=”temp/07012900. sch” SECCHI Consortium 3/5/07 - GDS 1. 23

Planning Tool Graphic SECCHI Consortium 3/5/07 - GDS 1. 24

Onboard Events • T-22: 45: 40 Image is scheduled at 0000 UT of day of observation • T-0 Shutter opens • T+EXPTIME+READTIME+IP_TIME – Image file (7129018 T. 443: data+header) goes to SEB memory, then waits in line to be be packetized and transferred to SSR 1 at 153 kbps (~ 1 MB/minute) • T+1 day – Image is part of SSR 1 playback → DSMS → APL MOC archive (for SSR 2 could be T+5 days) SECCHI Consortium 3/5/07 - GDS 1. 25

Pipeline Processing Timeline • Retrieve Telemetry – T+2 (0311 ET) APL MOC generates Level-0 archive file – T+2 (0730) Level-0 archive file retrieved to NRL - get_sds (nathan@calliope crontab) – T+2 (1030) Level-0 file played back (ITOS pblevel 0 cont. proc) to regenerate 7129018 T. 443 on the ITOS workstation. – T+2 (1130) HK database scripts for 1/29/07 copied to lambda and run so temperatures can be retrieved. • Level-0. 5 Pipeline Processing – T+2 (1200) secchi_reduce started – T+2 (2030) 20070129_224540_s 4 eu. A. fts is written to - $secchi/lz/L 0/a/img/euvi/20070129/ by secchi_reduce. pro (~11 hr/day in Jan. 07) – T+3 (1200) Entries for 20070129_224540_s 4 eu. A. fts are transferred to image database. SECCHI Consortium 3/5/07 - GDS 1. 26

Data Processing Diagram SECCHI Consortium 3/5/07 - GDS 1. 27

Pipeline Processing Steps - Level 0. 5 (1) • Telemetry playback (ITOS) – 2 Sun Ultra 25 workstations (Solaris) • Playback to HK - ITOS sequential print • HK to database - cphklz (/bin/sh) • Database - scc_hk_update (esfand@lambda crontab) – My. SQL on Sunfire 280 R (Solaris) • Science file to Level-0. 5 - secchi_reduce. pro (IDL) – Sun Ultra 25 Solaris or PC linux (TBI) – Populate headers - make_scc_hdr. pro - Orbit: SPICE: get_stereo_lonlat. pro, get_stereo_coord. pro - Attitude/pointing info: getsccpointing. pro: get_stereo_hpc_point. pro, get_stereo_roll. pro (SPICE), euvi_point. pro (from GT data, TBI) - Database to Headers (temperatures) - get_hbtemps. pro - Current table info - getscctableinfo. pro (queries My. SQL database) - See SECCHI FITS header definition for details: ftp: //louis 14. nrl. navy. mil/pub/secchi/ssw/doc/FITS_keywords. pdf SECCHI Consortium 3/5/07 - GDS 1. 28

Pipeline Processing Steps - Level 0. 5 (2) • Science file to Level-0. 5 - secchi_reduce. pro (IDL) - (cont. ) – Populate Extended Header for multiple exposure level 0. 5 images (mostly HI) – Decompression binaries: ricerecon 64 (Rice) or idecomp (Icer) or hd 64 (Hcompress) - COMPRSSN keyword in FITS header--get key with read_ip_dat. pro in $SCC_DATA/PT/PRO (currently NRL only) - NMISSING and MISSLIST (TBI) derived from decompression output – Rectify non-cal images: secchi_rectify. pro SECCHI Consortium 3/5/07 - GDS 1. 29

Post-Processing - Local Archive • FITS Archive - $secchi (/net/corona/secchi/flight/lz) see ftp: //louis 14. nrl. navy. mil/pub/secchi/ssw/doc/scctree. txt – 2 Silicon Mechanics storage servers: each has 12 drives, - venus: currently 1. 8 TB available (free) for secchi - earth: currently ? – Electronic Distribution - Maryland: SSC/GSFC (W. Thompson, rsync) - France: Orsay (F. Auchere, rsync) - California: JPL (J. Hall, rsync), LMSAL - Germany: MPAe; Belgium: ROB; …? – DVD archive - TBD (not for distribution) • IDL Image Query (local archive): – Scc_read_summary. pro: queries summary files in $secchi/*/*/summary – Scclister. pro: use GUI to find available files and query summary files for telescope, time, size, onboard source, polarizer/quadrant/filter, SEB_PROG, EXPTIME (future: COMPRSSN) – Sqdb. pro: query Image database - most flexible SECCHI Consortium 3/5/07 - GDS 1. 30

Post-Processing (cont. ) • Level-1 Processing - secchi_prep. pro (R. Colaninno) • Remove Background - getbkgimg. pro (K. Battams) – We have begun generating COR 2 “monthly” backgrounds for 0/120/240/TB at NRL – Insertion in pipeline pending • Analysis, CME Search, Daily MVI - in development • Website Access – CME Info - TBD – Beacon Mpegs - TBD – Daily Mpegs - TBD – Download Images and HK Plots - http: //stereo. nrl. navy. mil/secchi_database. html SECCHI Consortium 3/5/07 - GDS 1. 31