Physical Experiments Computer Experiments Preliminaries Chapters 12 Design

Physical Experiments & Computer Experiments, Preliminaries Chapters 1&2 “Design and Analysis of Experiments“ by Thomas J. Santner, Brian J. Williams and William I. Notz EARG presentation Oct 3, 2005 by Frank Hutter 1

Preface What they mean by Computer Experiments n n Code that serves as a proxy for physical process Can modify inputs and observe how process output is affedcted Math n Should be understandable with a Masters level training of Statistics 2

Overview of the book Chapter 1: intro, application domains Chapter 2: n n Research goals for various types of inputs (random, controlled, model parameters) Lots of definitions, Gaussian processes Chapters 3 -4: Predicting Output from Computer Experiments Chapter 5: Basic Experimental Design (similar to last term) Chapter 6: Active Learning (sequential experimental design) Chapter 7: Sensitivity analysis Appendix C: code 3

Physical experiments: a few concepts we saw last term (2 f) Randomization n In order to prevent unrecognized nuisance variables from systematically affecting response Blocking n n Deals with recognized nuisance variables Group experimental units into homogeneous groups Replication n Reduce unavoidable measurement variation Computer experiments n n Deterministic outputs “None of the traditional principles [. . . ] are of use“ 4

Types of input variables (15 f) Control variables xc n n Can be set by experimenter / engineer Engineering variables, manufactoring variables Environmental variables Xe n n Depend on the environment/user Random variables with known or unknown distribution When known for a particular problem: xe Noise variables Model variables xm n n Parameters of the computer model that need to be set to get the best approximation of the physical process Model parameters, tuning parameters 5

Examples of Computer Models (6 ff) ASET (Available Safe Egress Time) n 5 inputs, 2 outputs Design of Prosthesis Devices n n 3 environment variables, 2 control variables 2 competing outputs Formation of Pockets in Sheet Metal n 6 control variables, 1 output Other examples n n Optimally shaping helicopter blade – 31 control variables Public policy making: greenhouse gases – 30 input variables, some of them modifiable (control variables) 6

ASET (Available Safe Egress Time) (4 f) Evolution of fires in enclosed areas Inputs n n n Room ceiling height and room area Height of burning object Heat loss fraction for the room (depends on insulation) Material-specific heat release rate Maximum time for simulation (!) Outputs n n Temperature of the hot smoke layer Distance of hot smoke layer from fire source 7

Design of Prosthesis Devices (6 f) 2 control variables n n b, the length of the bullet tip d, the midstem parameter 3 environment variables n n n , the joint angle E, the elastic modulus of the surrounding cancellous bone Implant-bone interface friction 2 conflicting outputs n n n Femoral stress shielding Implant toggling (flexible prostheses minimize stress, but toggle more loosen) 8

Formation of Pockets in Steel (8 ff) 6 control variables n n n Length l Width w Fillet radius f Clearance c Punch plan view radius p Lock bead distance d Output n Failure depth (depth at which the metal tears) 9

Research goals for homogeneous-input codes (17 f) Homogeneous-input: only one of the three possible variable types present All control variables: x=xc n n Predict y(x) well for all x in some domain X Global perspective Integrated squared error s. X [y‘(x) - y(x)]2 w(x) dx Can‘t be computed since y(x) unknown, but in Chapter 6 we‘ll replace [y‘(x)-y(x)]2 by a computable posterior mean squared value n Local perspective Level set: Find x such that y(x) = t 0 = maximum value 10

All environmental variables: x=Xe (18) How does the variability in Xe transmit through the computer code ? Find the distribution of y(Xe) When the problem is to find the mean: n Latin hypercube designs for choosing the training sites 11

All model variables x=xm (18 f) Mathematical modelling contains unknown parameters (unknown rates or physical constants) Calibration (parameter fitting) n Choose the model variables xm so that the computer output best matches the output from the physical experiment 12

Research goals for mixed inputs (19 ff) Focus on case with control and environmental variables x=(xc, Xe) where Xe has a known distribution Example: hip prosthesis y(xc, Xe) is a random variable whose distribution is induced by Xe n n Mean (xc) = E{y(xc, Xe)} Upper alpha quantile = (xc): P{y(xc, Xe) >= } = 13

Research goals for mixed inputs: simple adaption of previous goals (20 ff) Predict y well over its domain: n n Minimize s. X [ ‘(x) - (x)]2 w(x) dx (again, there is a Bayesian analog with computable mean) Maximize the mean output: maxxc (xc) 14

When the distribution of Xe is unknown Various flavours of robustness G-robust: minimax n n Want to minimize your maximal loss Pessimistic (. )-robust n Minimize weighted loss (weighted by prior density on distribution over Xe) M-robust n n n Suppose for a given xc, y(xc, xe) is fairly flat Then value of Xe doesn‘t matter so much for that xc Maximize (xc) subject to constraints on variance w. r. t. Xe 15