Kosmos Software NOAO Software Implementation Phil Daly Kosmos

Kosmos Software NOAO Software Implementation Phil Daly Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

Overview • What Is The NOCS? – How it works, history etc – KOSMOS implementation – Data management • Schedule & Deliverables • Q&A Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

What Is The NOCS? The NOAO Observation Control System (NOCS) was developed for NEWFIRM, a 16 megapixel IR mosaic camera. The architecture diagram for NEWFIRM is shown to the right and the NOCS. Inherent in the NOCS paradigm is: • scripting • control of various sub-systems: – – – data handling system MONSOON detector controller Telescope system motion(s) instrument hardware meta-data repositories • NGUI, a separate utility, provides a scripting engine for creating welldefined science recipes. Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

Re-Using The NOCS Clearly it makes sense to re-use code where applicable and not re-invent the wheel. The rationale for using the NOCS is: It has 3+ years of use – – – First First Light: Light: NEWFIRM, KPNO Mayall 4 m, February 2007 NEWFIRM , CTIO Blanco 4 m, May 2010 Mosaic 1. 1, KPNO Mayall 4 m, scheduled for October 2010 Mosaic 1. 1, KPNO WIYN 0. 9 m , anticipated in semester 2011 A KOSMOS, KPNO Mayall 4 m, planning for semester 2011 B COSMOS. , CTIO Blanco 4 m, planning for semester 2012 A? Failure modes are well documented It is ideal for rapid deployment projects (as recommended by the Re. STAR report) since 80 -90% of functionality exists It is written with the GWC infrastructure software, common to all 3 telescopes, built-in It is integrated into the NOAO E 2 E data system It has been commissioned on 2 of the 3 telescopes Support staff at distributed sites are now familiar with it It will offer a “common look and feel” to NOAO instruments Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

The Future The table to the right shows current and planned instruments for the NOAO operated science system. As can be seen, these facilities represent different science capabilities on different telescopes in different wavelength regimes. Rather than implement a complete new control system for each instrument, we undertook a modest design study to see if the NOCS could be re-used to tackle these other instruments. So far, we have the NOCS working on the 4 m telescopes (for NEWFIRM) and are well on the way to having Mosaic 1. 1 using the NOCS too. KPNO Mayall 4 m NEWFIRM 4 k x 4 k IR Imager Mosaic 1. 1 8 k x 8 k OUV CCD Imager Spectrograph 2010 -2016+ 2011+ Now-2020+ CTIO Blanco 4 m NEWFIRM 4 k x 4 k IR Imager KOSMOS 2 k x 4 k OUV CCD COSMOS 2 k x 4 k OUV CCD Spectrograph Now-2020+ KPNO WIYN 0. 9 m 2012+ Mosaic 1. 1 8 k x 8 k OUV CCD Imager 2010 -2016+ Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

The NOCS Pre-Requisites • Original design in 2002 (? ) • NOAO Observation Control System had to conform to – – GWC in 4 m environment GPX over sockets Tcl/tk widely used at KPNO Software trade study implied DRAMA as best message passing mechanism – Common API for data transport – Not burden KPNO support staff • All done in Tcl/tk with loadable shared libraries —buzzword free zone (no Java, XML, Web 2. 0, python etc)! Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

The NOCS Interfaces Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

The K. I. S. S. Principle • wc -l *. tcl – Includes comments and blank lines – Includes infrastructure code • NOAO Observation Control System – NMSL, talks to Monsoon/Torrent • 1792 lines of Tcl/tk (Mosaic 1. 1) – NTCS, talks to the telescope • 1090 lines of Tcl/tk (Mosaic 1. 1) – NICS, talks to instrument hardware • 1343 lines of Tcl/tk (Mosaic 1. 1) – NOHS, talks to DHS and listens for environment meta-data • 790 libnes of Tcl/tk (Mosaic 1. 1) – NGUI, prepares scripts for observation • 3045 lines of Tcl/tk (Mosaic 1. 1) – nocs. Common. tcl, 1061 lines of Tcl/tk (Mosaic 1. 1) Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

NOCS Features • NOCS is elegantly simple • NOCS has modular and orthogonal tasks – Not a monolithic software system – Change any one does not affect the others – Task may be run independently for test • NOCS has complete logging for PM debugging and “on sky” wellness checks • Create scripts and run them separately! • Scripts tell the GUIs what to do • Supports test<task>, for example: – testnics filter U – testntcs offset “ 30 45” – Uses same comms path as scripts Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

Message Sequence Chart Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

A Very Simple Script #!/bin/sh ditscmd nmsl_gpx. Set. AVP Argument 1="int. Time=0. 7" ditscmd nmsl_gpx. Set. AVP Argument 1="row. Bin=1" ditscmd nmsl_gpx. Set. AVP Argument 1="col. Bin=1" ditscmd nics_filter Argument 1="1" EXPID=`$NEWFIRM_BIN/msd` ditscmd nohs_newobs Argument 1=“NOCID=${EXPID}…” ditscmd nmsl_gpx. Set. AVP Argument 1="exp. ID=${EXPID}" ditscmd nmsl_gpx. Get. State Argument 1="IGNORE" ditscmd nmsl_gpx. Start. Exp ditscmd nohs_endobs Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

Scripts: Can Do / Can’t Do • Can – Can abort single observation from NMSL (with cleanup) – Abort script using Ctrl/C • Then follow instructions for cleanup – – Re-use them (or use iterators from NGUI) Edit them (but not advised) Create hierarchies (but not advised with exceptions) Be as long/complex as you like • Have seen an 8000+ line script • Mark Dickinson linearity test: ~ 250 observations, 3. 5 hours • Can’t – Automatically clean up after a Ctrl/C – Re-start from specific place (but you can use vi) Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

KOSMOS Implementation X X X Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

NMSL • NMSL - Talks to MONSOON/Torrent if voltages and temperatures are nominal • ICD 4. 1 - nothing else to do? Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

NTCS • NTCS - Talks to Telescope etc • ICD 5. 1 - Add rotator, secondary focus Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

NOHS • NOHS - Listens to 4 m env, talks to DHS • ICDs 6. 1, 3. 1 - Create nohs. gwc file (already done in appendix A, ICD 6. 1) Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

NICS • NICS - Talks to hardware via NICC • ICD 3. 1, 3. 2 - Add KOSMOS elements, create filters file Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

nics. cfg #+ # NEWFIRM Filter Wheel List 06 -Jan-2010 for semester 2010 A # #NICC NGUI Combination FW 1 PosFW 2 PosSerial Number(s) #J J JX + Open 1 8 2104+open H H HX + Open 2 8 3104+open Ks KXs + Open 3 8 4103+open 2124 2124/68 block + 2124 nm 4 4 4303+4301 2168 Br gamma 2124/68 block + 2168 nm 4 5 4303+4302 J 1 J 1 blocker + J 1 5 1 2202+2201 J 2 J 2+3 blocker + J 2 6 2 2205+2203 J 3 J 2+3 blocker + J 3 6 3 2205+2204 H 1 H 1+2 blocker + H 1 7 6 3203+3201 H 2 H 1+2 blocker + H 2 7 7 3203+3202 Dark Dark. Slide + 2124 nm 8 4 dark+4301 Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

NGUI • NGUI - Creates scientifically useful scripts from OCDD input • Not an observing tool! Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

NGUI Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

NGUI / xterm Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

Data Management Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

Data Handling System • Collects data and meta-data from PAN and NOCS, processes data (as required) and assembles final FITS file on disk • ICD 6. 1 provides structured comms and procedures for data capture regardless of instrument • Data processing independent of readout cadence so processing continues as long as there is data waiting Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

Data Handling System • Supervisor layer controls actions in the DHS providing single point of contact for instrument (excluding pixel data capture) • Distributed application using component architecture • Components may be added, as needed, for multiple PANs • Produces pipeline ready product • Post-processing for ingestion into NOAO science archive via Save the Bits Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

DHS Components • Supervisor – Primary GUI interface – Provides user feedback and control • Collector(s) – Responsible for reading data from PAN and NOCS into shared memory cache • SMCMgr – Rectifies pixel orientation (lower left corner) – Provides RTD of sub-rasters Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

DHS Components • PXF – Puts segments from SMC onto message bus • DCA (data capture agent) – Spawns the Keyword Translation Module (KTM) to transform PAN/NOCS metadata into standard FITS header, provides rough WCS, science keywords, etc – Assembles final FITS image • SMC (shared memory cache) – – Interface to manage the many meta/data segments Allows for data to be managed in memory Maintains relationship of associated segments Provides buffer for data to backup during fast sequences Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

DHS Supervisor GUI Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

NOCS Schedule Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

NOAO Staff • Use people who know the system and developed original system where possible … Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

Hardware Rack • 4 x Lancelot 1184 -T, 1 Tb disk, $6980 (1 spare) • 2 x Systrans PCIe, $7280 (1 spare) • 1 x 12 U rack, $500 • 1 x HP Pro. Curve 1800 -24 G Gigabit switch, $360 • 1 x Trend. Net TK-802 R KVM switch + cables, $172 • 1 x 16 -port network power controller, $295 • Total: ~$15, 500 + tax + S/H Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

Deliverables • From OSU: ieb. Utils library by 31 -Oct 2010 • To OSU: completely populated rack (hardware and software) for OSU to perform lab integration and test using a full NOCS system by 31 -Jan-2011 • Fallback: mec etc Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH

The End … ? KPNO Mayall 4 m NEWFIRM 4 k x 4 k IR Imager Mosaic 1. 1 8 k x 8 k OUV CCD Imager Spectrograph 2010 -2016+ 2011+ Now-2020+ CTIO Blanco 4 m NEWFIRM 4 k x 4 k IR Imager KOSMOS 2 k x 4 k OUV CCD COSMOS 2 k x 4 k OUV CCD Spectrograph Now-2020+ KPNO WIYN 0. 9 m 2012+ Mosaic 1. 1 8 k x 8 k OUV CCD Imager 2010 -2016+ Kosmos Review, 2 -3 August 2010 OSU, Columbus, OH