Deutscher Wetterdienst GRIB 2 and its implementation in

Deutscher Wetterdienst GRIB 2 and its implementation in INT 2 LM and the COSMO-Model Ulrich Schättler Source Code Administrator COSMO-Model and (still) INT 2 LM

Deutscher Wetterdienst Contents è Structure of GRIB 2 è Usage of grib_api è Local Use Areas è New: General Vertical Coordinate 23. 02. 2021 WG 6 Meeting / Sibiu 2

Deutscher Wetterdienst GRIB 2 ? - WMO - Code forecast products è GRIdded Binary : Edition 0 (1985/88), Edition 1 (1990 ff) è General Regularly-distibuted Information in Binary form: Edition 2 (2001) è self explaining, compressed / packed è GRIB message contains metadata (explaining the product, grid, …) plus data Edition 2 is more flexible through a template/table structure, ammendments by adding templates/tables. v v v Version 2: 2003 Version 3: 2005 Version 4: 2007 Version 5: 2008 Version 6: 2009 etc. Changes include • coding of new elements/parameter • new product definition templates • new packing/compressing formats • new horizontal grid(s), e. g. ICON • new vertical coordinate type 23. 02. 2021 WG 6 Meeting / Sibiu Dörte Liermann 3

Deutscher Wetterdienst Comparing GRIB 1 – GRIB 2 GRIB 1 vs. 2: Different Structure GRIB 1 SECTION 0 Indicator (GRIB) SECTION 1 Product definition SECTION 2 Grid Description SECTION 3 Bitmap SECTION 4 Binary Data SECTION 5 End (7777) GRIB 2 SECTION 0 Indicator(GRIB, …) SECTION 1 Identification SECTION 2 Local Use SECTION 3 Grid Definition SECTION 4 Product Definition SECTION 5 Data Representation SECTION 6 Bitmap SECTION 7 Binary Data SECTION 8 End (7777) In GRIB 2 several variables are defined with more precision (e. g. angles are in micro-degrees, more vertical coordinates possible) see: www. cosmo-model. org/content/model/documentation/grib 2 keys. htm In GRIB 2 the description of the data (parameter, time, statistics, …) is more complex and is template/table based Dörte Liermann 23. 02. 2021 WG 6 Meeting / Sibiu 4

Deutscher Wetterdienst Element coding of model fields in GRIB 2 è GRIB 1: Element coding by ee (element) and tab (table) Example: temperature is ee=11 in tab=2 (WMO) è GRIB 2: Element coding by a triplet Discipline Category Parameter Example: temperature is (0, 0, 0) è Through setting of short. Name, these three parameters are set (and for some products also some other keys; see short. Name. def in definitions. edzw) Dörte Liermann 23. 02. 2021 WG 6 Meeting / Sibiu 5

Deutscher Wetterdienst GRIB_API (Application Programmers Interface) è ECMWF source code for de-/encoding of GRIB 1 AND GRIB 2 è no internal knowledge of GRIB structure needed è each element of a grib message has an alphanumeric name (key) that can be used to access the information linked to it (value) è key – value approach: short. Name=T type. Of. Level = hybrid. Layer top. Level = 37 bottom. Level = 38 è How to find keys: grib_keys –F file. grib lists all keys of a file è flexible – local definitions for each centre possible è i. e. local definition tables short. Name. def for edzw (GRIB 1 and GRIB 2), è local definition of GME icosahedral grid for GRIB 1 è But: needs more coordination within COSMO Dörte Liermann 23. 02. 2021 WG 6 Meeting / Sibiu 6 6

Deutscher Wetterdienst Product Identifying Keys è significance. Of. Reference. Time (GRIB 2 -WMO Tab. 1. 2) è production. Status. Of. Processed. Data (GRIB 2 -WMO Tab. 1. 3) è type. Of. Processed. Data (GRIB 2 -WMO Tab. 1. 4) è generating. Process. Identifier (local definition in GRIB 1/2) è background. Generating. Process. Identifier (local definition in GRIB 2) è type. Of. Generating. Process (GRIB 2 -WMO Tab. 4. 3) è local. Definition. Number (local section = 254, 253, 252; 250 for COSMO) è local. Number. Of. Experiment (GRIB 2) è local. Information. Number (GRIB 2) / local. Element. Number (GRIB 1) è local. Decode. Date: s 23. 02. 2021 WG 6 Meeting / Sibiu 7

Deutscher Wetterdienst Product Identifying Keys (II) Key Values significance. Of. Reference. Time 0 analysis 1 start of forecast 2 verifying time of forecast 3 observation time production. Status. Of Processed. Data 0 operational – Routine 1 operational test – Parallelsuite 2 research – Experiments 3 re-analysis products (possible local use) type. Of. Processed. Data 23. 02. 2021 0 analysis 1 forecast 2 analysis and forecast 3 control forecast 4 perturbed forecast 5 control and perturbed forecast WG 6 Meeting / Sibiu 8

Deutscher Wetterdienst Product Identifying Keys (III) Key Values type. Of. Generating. Process 0 analysis 1 initialization 2 forecast 195 interpolated analysis / forecast (local use) background. Generating. Process. Identifier (local use) generating. Process. Identifier 0 main run 1 pre-assimilation 2 assimilation 3 test Represents data base identifier (local use) 23. 02. 2021 WG 6 Meeting / Sibiu 9

Deutscher Wetterdienst Local Use Section (local. Definition. Number=254) è local. Definition. Number Identifier for content (historical: 254) è local. Host. Identifier è local. Creation. Date. Year/Month/Day/Hour/Minute/Second è local. Validity. Date. Year/Month/Day/Hour/Minute/Second è local. Number. Of. Experiment Number of Experiment è local. Information. Number è Identifier for host system/ computer Also need a local use section for COSMO (local. Definition. Number = 250) 23. 02. 2021 WG 6 Meeting / Sibiu 10

Deutscher Wetterdienst Implementation of Local Use SELECT CASE (ncenter) CASE (78) ! DWD … CASE DEFAULT END SELECT è We soon need to know about your local use of special keys! è We want to document local use on the Web Page! 23. 02. 2021 WG 6 Meeting / Sibiu 11

Deutscher Wetterdienst General Vertical Coordinate type. Of. Level = 150

Deutscher Wetterdienst Why a new vertical coordinate? è As a non-hydrostatic model, COSMO needs a special vertical grid: fixed in space and time è Also post-processing programs have to be aware of this grid (or the HHL) è But the algorithm to compute it, is rather complex (not just ak + bk * ps) è Therefore a proposition was made to WMO, to introduce a new type. Of. Level=150 è To process atmospheric data using that type. Of. Level, another 3 D field is necessary: the HHL fields è If a product has type. Of. Level=150, then there are 6 additional meta data in the Product Definition Section, which replace the vertical coordinate parameters è number. Of. VGrid. Used to identify a special vertical coordinate è nlev number of levels of the HHL file è uuid. Of. VGrid: unique universal identifier to ensure correct identification of HHL 23. 02. 2021 WG 6 Meeting / Sibiu 13

Deutscher Wetterdienst Current Situation è INT 2 LM and the COSMO-Model both compute the HHL fields and the reference atmosphere p 0. è The necessary vertical coordinate parameters (for HHL) and the reference atmosphere parameters are given to è INT 2 LM by Namelist variables è COSMO-Model by GRIB 1 meta data: but this always was a non-standard GRIB usage! è The new generalized vertical coordinate does not know meta data for vertical coordinate parameters and for the reference atmosphere. How to compute HHL and p 0 when using GRIB 2? Or what can we do? 2/23/2021 WG 6 Meeting / Sibiu 14

Deutscher Wetterdienst What can be done? è HHL can be transferred from INT 2 LM to COSMO or within the assimilation cycle from COSMO to COSMO by an extra file. è Do we then still need to transfer the vertical coordinate parameters vcoordk and vcflat? è YES! For computations in the nudging and some diagnostices (meanvalues, convective indices). Try to discuss this with the code experts. è Can we also transfer p 0? Or the full pressure? è For full pressure the problem is the GRIB packing. But we could use more bits for the packing (pressure deviation is a more homogeneous field with less precision loss in the packing) è Other question: Has anybody ever changed the Namelist variables in INT 2 LM (p 0 sl, t 0 sl, dt 0 lp, delta_t, h_scal) to compute another p 0? 2/23/2021 WG 6 Meeting / Sibiu 15

Deutscher Wetterdienst Current implemented Method for HHL è INT 2 LM can compute HHL and write an extra file COSMO_HHL_xxx. g 2 (if lnew. VGrid =. TRUE. ) or read it from such a file (is done in IEEE 64 -bit) è (Note: for COSMO such a HHL file has also to be read for the coarse grid) è INT 2 LM computes the atmospheric variables based on this HHL è The COSMO-model has to read the same COSMO_HHL_xxx. g 2 file to get the same HHL field as the INT 2 LM (checking the UUID should ensure using the correct HHL field) è INT 2 LM and the COSMO-Model define several sets of vertical coordinate parameter defaults (for COSMO-EU, COSMO-DE, COSMO-I 7, COSMO-7, etc. ). Every set has a special ID è HHL stores the ID of the default set used in the local. Information. Number in HHL 2/23/2021 WG 6 Meeting / Sibiu 16

Deutscher Wetterdienst Current implemented Method for p 0 è INT 2 LM and the COSMO-Model both compute p 0 (as before). è To get the correct reference atmosphere parameters, both programs store several sets of default parameters. Every set has a special ID è pp stores the ID of the used set in the local information number è INT 2 LM and the COSMO-Model now use the new module vgrid_refatm_defaults. f 90 2/23/2021 WG 6 Meeting / Sibiu 17

Deutscher Wetterdienst Problems and Drawbacks of this Approach è Have to deal with another file in our suites è Have to hard-code all possible COSMO configurations in the source code 2/23/2021 WG 6 Meeting / Sibiu 18

Deutscher Wetterdienst What can be done about HHL è Can we transfer HHL through the laf-file? Perhaps in higher precision? But this imposes problems to archiving! è Do we need full precision? è Check , whether usage of vcoordk, vcflat is really necessary or can be avoided 2/23/2021 WG 6 Meeting / Sibiu 19

Deutscher Wetterdienst What can be done about p 0 è Can we transfer full pressure (or p 0) through the laf-file? Perhaps in higher precision? Again this imposes problems to archiving! è Which precision is really needed? è Check , whether usage of reference atmosphere parameters can be „fixed“ 2/23/2021 WG 6 Meeting / Sibiu 20

Deutscher Wetterdienst Migration to GRIB 2: Timetable è The plans already pronounced by DWD have been revised last week è A prerequisite for GRIB 2 at DWD is the stable operation of the new database software SKY (old software is not able to process GRIB 2 meta data). And this is not the case at the moment. è New date for operational use of SKY („only the SKY is the limit“): November è If this date can be kept, then we could switch to GRIB 2 in January 2014. If not, GRIB 2 will not come before May / June 2014. Because in February / March we are migrating to the new CRAY computer system. è Message for COSMO and other partners: è Be prepared to read GME data in GRIB 2 with INT 2 LM 2/23/2021 WG 6 Meeting / Sibiu 21

Deutscher Wetterdienst Thank you very much for your attention 10. 09. 201223. 02. 2021 WG 6 General Meeting / Sibiu COSMO Meeting 2012 22 22