Online Data Processing Motivation Bigger Instruments more complex

Online Data Processing

Motivation Ø Bigger Instruments, more complex acquisition processes. Ø Need for Quicklook procedures specially during instrument acquisition. Ø Data Reduction Software (DRS) normally includes these routines for quality control. Instrument Control Systems Seminar 2019 2

Overview Definition and Goals Ø ODP is a simple component of the framework trying to solve this problem. Ø It resembles the VLT CLIP package. Ø It reuses the VLT INS image processing library (clipm) Ø Goals Ø To provide a data processing toolkit, flexible and well integrated with the INS SW. Ø To share the same image processing algorithms whenever possible between INS SW and pipeline DRS Ø To implement some specialized image processing routines which are generally required during the instrument acquisition process. Instrument Control Systems Seminar 2019 3

Overview Definition and Goals Image Processing Library Ø odp/clipm extending CPL (reused from VLT CLIP) C++ CPL Ø odp/cppcpl Python Bindings Ø odp/pycpl Instrument Control Systems Seminar 2019 4

Image Processing Library Goals 1. Cover common image processing needs, for example Ø star centroiding and object characterisation, Ø image alignment, 2. Serve as temporary development platform (until routines are accepted for CPL), Ø thus providing flexibility by separating DFS and INS SW development cycles Instrument Control Systems Seminar 2019 5

Image Processing Library Functions Image Analysis • Full bad pixel support • Background Estimation • Point-Source Centroiding • Fitting • Barycentre • Alignment • Cross-correlation • Transformation • Aperture Characterisation • Variously shaped objects • Automatic region of interest • Optimised results • Statistics in- & outside Instrument Control Systems Seminar 2019 Mathematics • Linear Regression • Random Number Generation • Gaussian • Poisson 2 d-pattern • Pattern Alignment • Different degrees of freedom • Transformation Interface • ANSI-C • Error Message History • CPL Objects in Memory 6

C++ CPL Library Goals Ø Main Features: Ø Direct access of images from shared memory Ø C++ wrappers for some CPL objects Ø Automatic initialization and cleaning up of CPL Ø Verbose and logging facilities Ø Byte swapping and data conversion. Instrument Control Systems Seminar 2019 7

C++ CPL Library Example odp: : Recipe recipe; odp: : Image img; odp: : Center. Gauss cen. Gauss; EXPECT_NO_THROW(img. Load(TEST_IMG 1)) << "loading FITS file " << TEST_IMG 1; EXPECT_NO_THROW( recipe. Get. Center. Gauss(&img, llx, lly, urx, ury, &cen. Gauss, 1)) << "executing center gauss" ; Instrument Control Systems Seminar 2019 8

Python Binding Library pybind 11 — Seamless operability between C++11 and Python Ø Support Ø Included in the Dev. Env Ø Selected by CII as the binding tool Ø Main Features: Ø It is simple to use Ø It allows to map a reach set of C++ to Python Instrument Control Systems Seminar 2019 9

Python Bindings Integration with Python Ecosystem odp: : Image odp: : Array odp: : Matrix source: arokem. github. io Instrument Control Systems Seminar 2019 10

Python Bindings Binding Example #include <pybind 11/pybind 11. h> namespace py = pybind 11; PYBIND 11_MODULE(pycpl, mod) { // Wrapping of class odp: : Image py: : class_<odp: : Image> pyimage(mod, "image", py: : buffer_protocol()); // Wrappers for constructors pyimage. def(py: : init<>()); pyimage. def(py: : init<const odp: : Image&>()); pyimage. def_property_readonly("width", &odp: : Image: : Get. Width); … } Instrument Control Systems Seminar 2019 11

Python Bindings Python Example import pycpl import numpy as np import matplotlib. pyplot as plt image = pycpl. image() image. Load(“ngc 1275. fits”, 0, 0, pycpl. cpl_type. CPL_TYPE_FLOAT) image_data = np. array(image, copy=False) plt. imshow(image_data, cmap=‘gray’) plt. show() Instrument Control Systems Seminar 2019 12

Extendability Image Processing - Development Strategy Consortium ESO Repositories Observatory SVN Pipeline World DRL Developer Pipeline DRL Pipeline Responsible Pipeline Integration <xx>clipm DRL (Master) Copy DFS Responsible GIT INS World Custom clipm INS Developer Instrument Control Systems Seminar 2019 C++ & Python (Wrappers) INS / DDT Integration 13

Status & Outlook Ø Status Ø It uses CPL 7. 0 Ø It reuses VLT Image Processing routines (clipm) Ø Wrapping in C++ and Python implemented Ø Outlook Ø Ø ODP Version 1 (alfa) will be shipped with next version of IFW. Integration with DDT (shared memory) Upgrade to CPL 7. 1. 2 Integration to CII services Instrument Control Systems Seminar 2019 14

Short Live Demo Instrument Control Systems Seminar 2019 15