UPDATES AND CURRENT STATUS OF THE HITRAN APPLICATION

UPDATES AND CURRENT STATUS OF THE HITRAN APPLICATION PROGRAMMING INTERFACE (HAPI) ROMAN V KOCHANOV, IOULI E GORDON, LAURENCE S. ROTHMAN, YAN TAN, Atomic and Molecular Physics, Harvard-Smithsonian Center for Astrophysics, Cambridge, MA, USA; JOSHUA KARNS, WYATT MATT, Computer Science, State University of New York at Oswego, NY, USA; CHRISTIAN HILL, Atomic and Molecular Data Unit, International Atomic Energy Agency, Vienna, Austria;

Latest HITRAN 2016 edition Features: q 47+2 molecules with about >120 isotopologues in line-by-line section q >320 molecules represented as experimental cross-sections at various pressures and temperatures (+ auxiliary info) q Spectral range: from microwave to ultraviolet q Applications: remote sensing, atmospheric absorption/radiance modeling, planetary science, astrophysics … q Access using HITRANonline / HAPI : http: //hitran. org q Software for calculating absorption (HAPI + Fortran) 2

Non-Voigt profile parameters in HITRAN New HITRAN data on foreign broadeners • Wilzewski et al, “H 2, He, and CO 2 line-broadening coefficients, pressure shifts and temperature-dependence exponents for the HITRAN database. Part 1: SO 2, NH 3, HF, HCl, OCS and C 2 H 2, ” JQSRT 168 (2016) 193– 206. • Li et al, “Rovibrational Line Lists for Nine Isotopologues of the CO Molecule in the X 1Σ+ Ground Electronic State, “ Ap. J. Suppl. Ser. 216 (2015) 15. • See posters Samuels et al. P-18 for additional molecules that are being added, and 3 Tan et al P-17 for broadening by water

Absorption cross-sections in HITRAN 2016 update features: • Categorized compounds for remote sensing, environment monitoring, climate applications, industrial pollution tracking, instrument calibration… • Atmospherically relevant temperature and pressure range (temperature 180 -362 K, pressure up to 760 Torr) • Update contains spectra with features in near, middle and far infrared. • FTS and CRDS measurements: PNNL, Harrison et al. , Bravo et al. , Godin et al. , Wagner and Birk, Highwood and Shine, Reed et al. , Wallington et al. , etc. 4

HITRAN Application Programming Interface - Python library for spectroscopic codes What HAPI does? q Retrieves and filters spectroscopic data, e. g. from HITRANonline (line-by-line data, cross sections, information on isotopic species etc. . . ) q Models absorption cross-sections in a flexible way accounting many parameters (air-, self-, and foreign broadenings and shifts, environment and instrument properties, custom line-shapes and partition sums …) q Provides basic routines for creating spectroscopic codes involving absorption crosssections (atmosphere absorption modeling, direct and inverse spectroscopic problems etc…) q Processes large volumes of data (support for HDF 5) q Provides codes for profiles and partition sums rewritten in Python and tested q Provides an interactive help Download (Open Source License): http: //hitran. org/hapi (Code/Manual) https: //github. com/hitranonline/hapi -> Github project Prerequisites: Python 2/3: https: //www. python. org/ Numpy: http: //www. numpy. org/ R. V Kochanov, I. E. Gordon, L. S. Rothman, P. Wcislo, C. Hill, J. S. Wilzewski. HITRAN Application Programming Interface (HAPI): a comprehensive approach to working with spectroscopic data. Submitted to Journal of Quantitative Spectroscopy and Radiative Transfer 5 High. Rus Special Issue (2015)

Molecules available via HAPI 2. 0 6

HITRANonline/HAPI infrastructure HAPI Python SDK HAPI functions Processed and filtered data API Key HTTP REQUEST JSON RESPONSE 7

Line summary: Linelist summary: 8

Calculation of absorption spectra for gas media MIXTURE HITRANonline User data API QUANTUM TRANSITIONS ABSORPTION CROSS-SECTIONS CIA HAPI Software Development Kit 9

Line profiles ● 7 line profiles (including p. Cq. SDHC: Partially Correlated Quadratic Speed dependent Hard Collision Profile, Ngo et al. JQSRT 129 (2013) 89– 100) 10

OPTIMIZATION AND PERFORMANCE BENCHMARK The most computationally intensive part – absorption spectra calculation! - HUMLICEK • One of the most widespread implementation of CPF • Goes with the Hartmann-Tran profile implementation by Ngo et al. [JQSRT 129 (2013)] - SCHREIER • Schreier F. Optimized implementation of rational approximations for the Voigt and complex error function (2011) [JQSRT 112 (2011)]. • Combination of Weidemann and Humlicek approximations based on subdivision of the upper half-plane in two regions For the absorption coefficient SCPF gives better results than HCPF in all three cases. Extensive calculations show that SCPF is 2 -4 times faster than HCPF for a wide range of pressures and temperatures. Switching CPF in HAPI: VARIABLES[‘CPF’] = hcpf # Humlicek VARIABLES[‘CPF’] = hum 1_wei # Schreier 11

Collisional line-mixing in CO 2 Collisional line-mixing effects are neglected by the widely used Voigt profile… CO 2 -N 2, 73 atm T=296 K Experimental No LM (Voigt) CO 2, 2. 1μm (TCCON) No LM With LM 1% > 0. 3% Lamouroux et al. , JQSRT 113 (2010), 2321 -2331 With LM

New python function for CO 2 line-mixing q The absorption_coefficient_linemixing function (derived from J. Lamouroux et al. , JQSRT 151, 88 -96 (2015)) q But: Only for CO 2 Full relaxation matrix calculation with speed-dependent Voigt profil not available

14

Example: modeling and comparing the methane spectra from HITRAN 2016 and 2012

Partition sums ● Total internal partition sum (TIPS-2017): R. Gamache et al. JQSRT (2017) 70 -87 – – Included information about 166 isotopologues, 51 molecule Temperature range up to 9000 K (isotopologue-dependent) 3500 CH 3 Br, SF 6, COF 2, CH 3 OH, OCS, O 3, CF 4, NO 2, Cl. O, NO+, HC 3 N, HCOOH, CH 4, HO 2, HOCl, C 4 H 2, N 2 O, C 2 N 2, HOBr, HNO 3, SO 3, SO 2, CS, C 2 H 6, C 2 H 4, HCN, C 2 H 2, CH 3 Cl, CO 2, OH, H 2 S, CH 3 CN, H 2 CO 4000 H 2 S 5000 7500 O 2 ● 9000 N 2, CO, OH TIPS-2011 also available COCl 2, H 2 O, CO 2, N 2 O, C 2 H 2 4500 PH 3, CH 4 6000 H 2 O 2, HI, NH 3, HF, HCl, HBr 17

HAPIEST • “HAPI and Efficient Spectroscopic Tools” • Graphical user interface (GUI) software • Interactive tools giving access to the basic features of HAPI • Access to most of the controls which are involved in the spectral filtering and simulation • Available in source code and binary distribution – Only Windows distribution available at the moment – Linux and Mac. OS distro’s are in preparation 18

HAPIEST Main “fetch” window 19

HAPIEST Graphing windows 20

HAPIEST Github page 21

WHERE TO GET HAPI/HAPIEST Official page: http: //hitran. org/hapi

Github project fof HAPI: https: //github. com/hitranonline/hapi

WHERE TO GET HAPI/HAPIEST Official page: http: //hitran. org/hapi Github projects: https: //github. com/hitranonline/hapiest SUMMARY PREREQUISITES FOR HAPI • Python 2. 6 / Python 3 • Numpy • Python library for fast operations on arrays • Numba • For faster spectra calculations Community and archive http: //zenodo. org/collection/user-hapi 25

Acknowledgements Current and former members of the HITRAN team, Harvard-Smithsonian Center for Astrophyscs: Dr. Iouli Gordon, Dr. Laurence Rothman, Dr. Yan Tan, Dr. Robert Hargreaves, Joshua Karns, Wyatt Matt, Eamon Conway, Dr. Christian Hill, Dr. Piotr Wcisło, Robab Hashemi, Shanelle Samuels, Dr. Tijs Karman, Dr. Aleksandra Kyuberis, Jonas Wilzewski, Lorenzo Barrett HAPI users outside Cf. A (who has given a feedback): Dr. Erin Adkins (NIST, USA), Dr Anand K. Ramanathan (NASA, USA), Dr. Teofil Minea (FOM DIFFER, Netherlands), Solveig Froen (DLR, Germany), Dr. Mike Georgiadis (Thermo Fisher Scientific, USA), Xiongguo Liu, (RWTH Aachen University, Germany), Asst. Prof. Joseph Lefkowitz (Israel Institute of Technology), Dr. Nils-Holger Löber (HFG-DGG, Germany), Dr. Eugenio Giacomazzi (ENEA, Italy), Igor Saito (MIPT, Russia), Dr. Atef Rashed (AMETEK, UK), Dr. Leonard Nitzsche (Fraunhofer Society, Germany), Dr. Guillaume Drolet (Ministère des Forêts, Canada), Dr. William A. Spencer (SRNL, USA), Dr. Thomas A. Depew (Coanda Research Corp. , Canada) NASA GRANTS: PATM, PDART, AURA TSU: International Competitiveness Improvement 26 Program

Thanks for your attention! 27

28

HITRAN line-by-line data format: transition parameters 160 -character: “. par” • • • + Foreign broadenings, non-Voigt profile parameters, … Database fully in SQL format available at HITRANonline (http: //hitran. org) ASCII text format can be derived from database Flexible output format (CSV, column-fixed, HDF 5, API) Beer–Lambert law: where I is the intensity of monochromatic radiation transferred through a layer of the width l, k is the absorption coefficient that is independent of l, I 0 is an initial radiation intensity at l=0, and ν is a vacuum wavenumber of the radiation. Absorption coefficient for the Voigt profile: where n(i) – isotopologue concentration, Sij – line intensity, Φ – Voigt line-shape, γD – Doppler line half-width, γL – collisional (Lorentz) line-width, δL – collisional line shift. Additional parameters: temperature dependence exponent (nair), Einstein A-coefficient, lower state energy, upper and lower state quanta, statistical weights => full list in [JQSRT 203 (2017) 3 -69] 29

Molecules in available in HAPI 2. 0 30

TIPS-2017 upper temperature limits: 3500 CH 3 Br, SF 6, COF 2, CH 3 OH, OCS, O 3, CF 4, NO 2, Cl. O, NO+, HC 3 N, HCOOH, CH 4, HO 2, HOCl, C 4 H 2, N 2 O, C 2 N 2, HOBr, HNO 3, SO 3, SO 2, CS, C 2 H 6, C 2 H 4, HCN, C 2 H 2, CH 3 Cl, CO 2, OH, H 2 S, CH 3 CN, H 2 CO 4000 H 2 S 5000 7500 O 2 9000 N 2, CO, OH COCl 2, H 2 O, CO 2, N 2 O, C 2 H 2 Lagrange interpolation option available: 4500 PH 3, CH 4 6000 H 2 O 2, HI, NH 3, HF, HCl, HBr

HITRANonline/HAPI infrastructure User’s local machine API Key HAPI Python SDK HAPI functions HTTP REQUEST JSON RESPONSE Processed and filtered data APACHE WEB SERVER DJANGO WEB FRAMEWORK MYSQL DATABASE REST API Protocol Database Management System: My. SQL v. 5. 5 OS: Linux Ubuntu 14. 04. 5 LTS Other: Django v 1. 9, Apache Server 32

33

API response for molecule (JSON/YAML) 34

API response for molecule (XML) 35

HITRAN Application Programming Interface Main page of the project: http: //hitran. org/hapi ● What is API? ● API, an abbreviation of Application Program Interface, is a set of routines, protocols, and tools for building software applications. The API specifies how software components should interact and be used in a program code. – A good API makes it easier to develop a program by providing all the building blocks. A programmer then puts the blocks together. What is HITRAN API toolkit (HAPI)? – – Software development kit (SDK) in Python working with spectroscopic data REST-API interface to interact with HITRANonline information system Main purpose: extending user's code by the data from the spectroscopic databases 36

HITRAN Application Programming Interface (http: //hitran. org/hapi) - Python library for spectroscopic codes Embedded help system: Embedded tutorials: 37

Summary on all molecules: Summary on particular molecule: 38

● What is API? ● API, an abbreviation of Application Programming Interface, is a set of routines, protocols, and tools for building software applications. The API specifies how software components should interact and be used in a program code. – A good API makes it easier to develop a program by providing all the building blocks. A programmer then puts the blocks together. What is HITRAN API (HAPI)? – – Python module (library of functions) to work with HITRAN data Main purpose: extending user's code by the data of HITRAN and functionality of HITRANonline Offline part of HITRANonline functionality 39

Why is API a powerful tool for creating radiative-transfer codes? ● Seamless data format conversion Parsing data (cross-sections, line-by-line, parameters) “in a background” – ● Flexibility – – – ● ● Large number of input parameters to control calculation Possibility to add custom user code Possibility to use different line parameters and compare results Portability Access to external libraries – Python, Fortran, C++. . . 40

What is HAPI for? ● ● Retrieve and use of HITRANonline data (line lists, cross sections, molecule and isotopologue info. . . ) from a program written in Python (HITRANonline data is parsed automatically) Reduce the load of HITRANonline web service (calculation of cross sections can be resource-demanding!) Give more flexibility in data filtering (as in RDB) Provide a possibility to work with HITRAN data “offline” (locally) 41

API Key generation rkochanov@cfa. harvard. edu 42

Environment description Python code: Resulting JSON: 43

OPTIMIZATION AND PERFORMANCE BENCHMARK Methane_I Dataset properties Number of lines 3804 1000 -1100 Wavenumber range, cm-1 Methane_II 59805 1000 -1500 Methane_III 468013 0. 001 -11505 Data filtering time, s Filtering using full table 0. 275 scan 4. 176 34. 535 Absorption coefficient calculation time, s SCPF, 50 HWHMs HCPF, 500 HWHMs 1. 756 2. 897 4. 948 6. 895 9. 855 15. 447 Instrumental function convolution time, s Step = 0. 01 cm-1 0. 007 0. 953 102. 58 SCPF, 200 HWHMs SCPF, 500 HWHMs HCPF, 50 HWHMs HCPF, 200 HWHMs Step = 0. 001 cm-1 Step = 0. 0001 cm-1 27. 331 44. 101 79. 493 109. 530 154. 834 249. 162 0. 032 3. 820 661. 858 317. 371 437. 317 710. 458 869. 950 1202. 387 1930. 663 0. 740 94. 058 12456. 012 44

Supplemental folder Originally suppled programs: TIPS: http: //hitran. org/suppl/TIPS/ Fortran + Python standalone (Python 3 is now supported) LM: http: //hitran. org/suppl/LM/ Fortran version 45

Calculation of absorption spectra from the line-by-line data Transitions PROFILE TYPE LOCAL DATASET: HITRAN / CUSTOM Code Inputs TEMPERATURE AND PRESSURE DEPENDENCES ENVIRONMENT PARAMETERS: Pressure (p), temperature (T), path length (l) MIXTURE DEFINITION INSTRUMENTAL FUNCTION 1 st CONTROL PARAMETERS: Wavenumber range, step size, line wing, intensity cutoff, output format ν 0 , S, ΓD , Γ 0 , Γ 2 , Δ 0 , Δ 2 , νVC, η, … LM flags: order (Rosenkranz), Full W line mixing PART. SUMS: TIPS-2017 TIPS-2011 p, T LINE MIXING PARAMETERS LINE PROFILE COMPLEX PROBABILITY FUNCTIONS Absorption coefficient Transmittance spectrum Data Absorption spectrum Radiance spectrum Planck-mean abs. coef. instrumental functions SIMULATED SPECTRUM 46

Collisional line-mixing For a collisionally isolated transition, the main effects of intermolecular collisions are the Lorentz broadening and shifting of the line Collisions induce transfers of populations between the levels of the two lines that lead to transfers of intensity between the lines Line mixing in HITRAN 2016: • 1 st order (Rosenkranz) LM • Full relaxational matrix (CO 2)

An example of run absorption_coefficient_linemixing('data_new_python/', mix_sdv=True, omega_range=[6400. , 6500. ], omega_step=0. 004, environment={'p': 1. 0365, 'T': 295. 7}, diluent={'self': 5. 06 e-2, 'water': 1. 05 e-2}, file_name='test. out', file_format='%10. 3 F %. 20 E') « HAPI CPF » « Fortran CPF »

Example: modeling of C 2 H 2 v 5 -band absorption in a He+H 2 atmosphere 49

HAPIEST Line-by-line data display 50

HAPIEST Absorption spectra calculation 51