The Joint Effort for Data assimilation Integration JEDI

The Joint Effort for Data assimilation Integration (JEDI) Bias Correction (Under development) Joint Center for Satellite Data Assimilation (JCSDA) JEDI Academy - 10 -13 June 2019

Introduction • Many observations have systematic errors (biases), such as satellite radiance data, radiosonde and aircraft data etc. • Observational data are biased due to the error associated with the instrument, calibration, radiative transfer models, and inconsistency between the data usage and the model or algorithms (Zhu et al. 2015). • Variational Bias Correction (Var. BC) is extensively used in almost all operational DA systems and plays important role in satellite radiance data assimilation, in particularly.

Introduction • Different observation type employs different bias model/algorithm. • Different NWP center may choose different bias model/algorithm for the same observation type. • Even for a specific type of observation, there can be multiple bias models to try. for example, Zhu et al. (2015) presents the performance of the flight phase model (FP), the quadratic aircraft ascent/descent rate model (QAD) and the aircraft ascent/descent rate model (AD), respectively, for aircraft observation.

Design principle • Share as much as possible without imposing one approach Ø One Var. BC term, multiple methodologies/configurations • Ease maintenance Ø Plug & Play, yaml configuration controlled • Speed-up future developments Ø Clear pathway for developers to write a new bias model for a observation type • OO is not magic and will not solve scientific questions by itself. • Scientific questions (bias correction model) remain but scientific work can start. (From Objectives to Code Design, Yannick)

Cost function with Var. BC

OOPS orchestrating level Obs. Aux. Control. h Obs. Aux. Increment. h Obs. Aux. Covariance. h

OOPS orchestrating level Obs. Aux. Control. h Obs. Aux. Increment. h Obs. Aux. Covariance. h

OOPS orchestrating level Obs. Aux. Control. h Obs. Aux. Increment. h Obs. Aux. Covariance. h

Observers (OOPS) Observer is a Post. Processor that simulates observations: - At each step, Geo. Va. Ls are populated from the State - In the finalization, simulated observations are generated from Geo. Va. Ls - The observation filters are applied template <typename MODEL> void Observer<MODEL>: : do. Finalize() { Log: : trace() << "Observer: : do. Finalize start" << std: : endl; filters_->prior. Filter(*gvals_); hop_. simulate. Obs(*gvals_, yobs_, Obs. Aux. Control_ & ybias_); filters_->post. Filter(yobs_); Log: : trace() << "Observer: : do. Finalize done" << std: : endl; } JEDI Y. Trémolet, JCSDA

Control. Variables (OOPS) Control. Variable class in OOPS defines the control variables in minimization:

From interface to implementation Obs. Bias. h Obs. Bias. cc (UFO) Obs. Aux. Control. h Obs. Aux. Increment. h Obs. Aux. Covariance. h (OOPS) Obs. Bias. Increment. h Obs. Bias. Increment. cc (UFO) Obs. Bias. Covariance. h Obs. Bias. Covariance. cc (UFO)

From interface to implementation Ufo: : Obs. Operator. cc Added for bias correction term

From interface to implementation biasmodel_ is a pointer, points to specific bias model, which is created at run-time based on the configuration

From interface to implementation Derived Bias Models Bias. Model Base class Bias Model Factory Obs. Aux. Control. h (OOPS) Obs. Bias. h Obs. Bias. cc (UFO) Obs. Bias. Model Radiance. GSI. h (UFO) Obs. Bias. Model. Base. h Obs. Bias. Model. Base. cc (UFO) • The application-specific bias model is created at runtime based on the Obs. Bias configuration • The Factory pattern promotes loose coupling by eliminating the need to bind application-specific classes into the code. • The Factory pattern helps make a system independent of how objects are created. Obs. Bias. Model Radiosonde. h (UFO) Obs. Bias. Model Aircraft. FP. h (UFO) Obs. Bias. Model Aircraft. QAD. h (UFO) Future developments Obs. Bias. Model XXXX. h (UFO)

Yaml controlled amsua_crtm_bc. yaml ufo/src/ufo/biasmodel/Obs. Bias. Model. Radiance. GSI. cc

FV 3 -JEDI Traits OOPS level Obs. Aux. Control. h interface (Model Interface, Dan) FV 3 -JEDI Templates passed in through Traits. Basically just a list of implemented classes.

future prospects fully connected feedforward neural network with no hidden layer and using a linear activation function (namely the identity activation function): Variational cost function with variational bias correction VS. Solution: Neural Network With Regularization https: //www. cs. cmu. edu/afs/cs. cmu. edu/academic/class/15381 -s 06/www/nn. pdf

future prospects • Is it possible to leverage Machine Learning (ML) neural network library (say Tensorflow C++ API: https: //www. tensorflow. org/guide/extend/cc ) to construct bias model ? • Is it worth to try deep neural network (multiple-layer) to compute bias ?