Challenges in Verification and Validation of CFD for

Definitions CONFERENCE | 2018 Fluid Development and Use of Engineering. LAW Standards for Computational

LAW CONFERENCE | 2018 Why is this important? Code Verification • Provides confidence equations

LAW CONFERENCE | 2018 Verification: Laplace’s Equation § Temperature distribution in a uniform square

Verification: Inviscid Channel with Bump CONFERENCE | 2018 Wang, et al. “High-order CFDLAW methods:

LAW CONFERENCE | 2018 Turbmodels. larc. nasa. gov Verification: Far Field Boundary Condition §

LAW CONFERENCE | 2018 Verification: Discontinuous Flows 20° § Cartesian box grid (2. 5

LAW CONFERENCE | 2018 Turbmodels. larc. nasa. gov Validation: Backward facing step Experiment: 6.

LAW CONFERENCE | 2018 Validation: Backward facing step Experiment 6. 26” ”± 0. 01”

Validation Example: High-Speed CRM Schaefer, J. A. , Cary, A. W. , and Mani,

Validation Example: High Speed CRM LAW CONFERENCE Rivers and Dittberner. “Experimental Investigations| 2018 of

LAW CONFERENCE | 2018 Verification & Validation: What is Required? § Attention to detail

LAW CONFERENCE | 2018 Summary § Verification is much harder than one would expect

Copyright © 2020 Boeing. All rights reserved.

LAW CONFERENCE | 2018 Model Maturity/ Experiment Completeness Matrix W. L. Oberkampf and B.

LAW CONFERENCE | 2018 Theory: Lax Equivalence Theorem Consistency Truncation error Stability Convergence Discretization

LAW CONFERENCE | 2018 Theory: Discrete Error Transport Equation § Copyright © 2020 Boeing.

LAW CONFERENCE | 2018 Backward Facing Step: Cf distribution Additional refinement uncovers new features

Slides: 18

Download presentation

Definitions CONFERENCE | 2018 Fluid Development and Use of Engineering. LAW Standards for Computational Dynamics for Complex Aerospace Systems, AIAA Committee on CFD Standards, 2016 -3811 § Verification: the process of determining that a model implementation accurately represents the developer’s conceptual description of the model and the solution to the model – Code verification: adequacy of numerical algorithms to provide accurate numerical solutions to the PDEs assumed in the mathematical model and fidelity of programming to implement the numerical algorithms to solve the discrete equations – Solution verification: Quantitative estimation of numerical accuracy of solution to the PDEs computed by a code (numerical error estimation). Sometimes also refers to review of all code inputs for correctness (0 order error). – Code regression: software practice for rerunning functional tests to ensure that previously developed and tested software still performs after a change § Validation: the process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model – “Benchmarking” represents comparison/co-plotting experiment with simulation Definitions accepted in numerical science; not consistent across aerospace Copyright © 2020 Boeing. All rights reserved. Boeing Research & Technology 2

LAW CONFERENCE | 2018 Why is this important? Code Verification • Provides confidence equations are being solved correctly Copyright © 2020 Boeing. All rights reserved. Solution Verification • Develops estimate of numerical sensitivities Validation • Develops estimate of model form uncertainty Predictive Capability • Confidence in answer for new problems Boeing Research & Technology 3

LAW CONFERENCE | 2018 Verification: Laplace’s Equation § Temperature distribution in a uniform square plate § Problem given to students § Richardson extrapolation assumes: – – Smooth solutions Uniform/consistent grid refinement Asymptotic solution convergence Consistent scheme Straightforward, Easy, No Problem… Copyright © 2020 Boeing. All rights reserved. Boeing Research & Technology 4

Verification: Inviscid Channel with Bump CONFERENCE | 2018 Wang, et al. “High-order CFDLAW methods: current status and perspective”. Int. J. Numer. Meth. Fluids. 2013. 72: 811 -845 § Many codes perform sub 2 nd order (potentially due to first order wall boundary condition) § Observed order of accuracy of “second order” codes not second order: – L 2(Entropy Error): O(h**2. 56) – Drag Error: O(h**2. 90) – Point values also vary in observed convergence rate § Effect of non-linearity of equations? § Are codes verified? Copyright © 2020 Boeing. All rights reserved. M =0. 5

LAW CONFERENCE | 2018 Turbmodels. larc. nasa. gov Verification: Far Field Boundary Condition § Turbulence model resource (TMR) near-wake case, original far field boundary § Specific details of boundary condition not always known § Far field boundary has subsequently been extended and agreement has been obtained Copyright © 2020 Boeing. All rights reserved. Boeing Research & Technology 6

LAW CONFERENCE | 2018 Verification: Discontinuous Flows 20° § Cartesian box grid (2. 5 K-10 M cells) at different rotations (including shock-aligned) § Multiple flux functions and gradient limiting § Discontinuity suggests scheme is inconsistent -1 0 Grid aligned Roe grid aligned Copyright © 2020 Boeing. All rights reserved. 1 HLLC grid aligned – Consequence of simplicity of case – Results from stabilizing dissipation term that does not disappear for non-smooth solutions Boeing Research & Technology 7

LAW CONFERENCE | 2018 Turbmodels. larc. nasa. gov Validation: Backward facing step Experiment: 6. 26”± 0. 01” 8. 2 M 1. 3 M § Assessing grid convergence of backward facing step (TMR) to perform UQ analysis § Level of grid required for convergence depends on quantity of interest § What does it take to indicate “verification”? Copyright © 2020 Boeing. All rights reserved.

Validation Example: High-Speed CRM Schaefer, J. A. , Cary, A. W. , and Mani, M. CONFERENCE “Uncertainty Quantification LAW | 2018 and Sensitivity Analysis of SA Turbulence Model Coefficients in Two and Three Dimensions, ” AIAA Paper 2017 -1710, January 2017 § Copyright © 2020 Boeing. All rights reserved. Boeing Research & Technology 10

Validation Example: High Speed CRM LAW CONFERENCE Rivers and Dittberner. “Experimental Investigations| 2018 of the NASA Common Research Model”. AIAA 2010 -4218. § Ultimately compare to data § In this case from entries into two different tunnels (NTF and Ames 11 ft) § Area metric reduces to single interval [1. 16, 13. 39] counts § Did not account for aeroelastic deformation in analysis… Copyright © 2020 Boeing. All rights reserved. Boeing Research & Technology 11

LAW CONFERENCE | 2018 Verification & Validation: What is Required? § Attention to detail § Inclusion of uncertainties/sensitivities in analysis – Discretization error – Model uncertainties – Boundary condition/geometry uncertainties § Experimental data with additional measurements to anchor simulations – Redundant measurements – Tight controls – Repeat experiments (treat as statistical experiment) Validation does not indicate fit for use Copyright © 2020 Boeing. All rights reserved. Boeing Research & Technology 12

LAW CONFERENCE | 2018 Summary § Verification is much harder than one would expect – Always find nuances – Identify where value is added – Solution verification should not be overlooked (and is user responsibility) § Validation is necessary to provide comparison to data for specific purposes – Codes are not “validated” – Need quality validation experiments (including uncertainty and additional measurements) – Need to include UQ of CFD results As use and scope of simulations increase, need to have better understanding of how to assess the quality and expected results Copyright © 2020 Boeing. All rights reserved. Boeing Research & Technology 13

LAW CONFERENCE | 2018 Model Maturity/ Experiment Completeness Matrix W. L. Oberkampf and B. L. Smith, “Assessment Criteria for Computational Fluid dynamics Validation Benchmark Experiments” AIAA 2014. W. L. Oberkampf, M. Pilch, and T. G. Trucano, “Predictive Capability Mturity Model for Computational Modeling and Simulation. ” Sandia National Labs SAND 2007 -5948. Copyright © 2020 Boeing. All rights reserved.

LAW CONFERENCE | 2018 Theory: Lax Equivalence Theorem Consistency Truncation error Stability Convergence Discretization error For a well-posed linear initial value problem, a consistent finite difference method is convergent iff it is stable Copyright © 2020 Boeing. All rights reserved.