Comparison between DerivativeFree Optimization Methods for DEB parameter
Comparison between Derivative-Free Optimization Methods for DEB parameter estimation of different species Fifth international symposium on Dynamic Energy Budget theory: metabolic organization plays a role in planetary stewardship J. V. Morais, A. L. Custódio and G. M. Marques jessica. morais@tecnico. ulisboa. pt goncalo. marques@tecnico. ulisboa. pt 1 June 2017
Summary 1 Problem definition 2 Nelder-Mead Simplex method 3 Directional Direct-Search methods 4 SID-PSM algorithm 5 DS-Random algorithm 6 Numerical results 7 Conclusions and future research 2
Problem definition Biological interpretation (data) Estimation (parameters) 3
Problem definition Biological interpretation Estimation (parameters) (data, results & parameters) . . . Final parameters 4
Problem definition DEB parameters estimation - Observed values - Predicted values - Parameters 5
Derivative-Free Optimization Methods The problem is to minimize a nonlinear function of several variables 6
Derivative-Free Optimization Methods • The derivatives of this function are not available The methods are known as Derivative-Free Optimization methods (DFO) 7
Nelder-Mead Simplex method 8
Nelder-Mead Simplex method - Suited for unconstrained Derivative-free Optimization problems - Simplex can become flat or needle shaped, causing convergence to non stationary points - Final result depends on starting point Mc. Kinnon function 10
Directional Direct-Search methods 11
Directional Direct-Search methods – Poll step 12
Directional Direct-Search methods 13
Directional Direct-Search methods - Scalling Sensitive to the dimension of the variables 14
SID-PSM method (A. L Custódio and L. N Vicente - 2007) Search step Poll step - Optional - Oportunistic polling - Unnecessary for establishing convergence properties - Ordering of the poll directions - Used to improve the numerical efficiency of the method - Based on quadratic polynomial interpolation models 15
SID-PSM method Convergence to some form of stationarity from arbitrary starting points Why? 1. Convergence of step size parameters to zero 2. Control of the geometry of the sample sets with the use of positive bases in the poll step 16
DS-Random method (C. W Royer and L. N Vicente - 2015) Search step No search step Poll step Random sets Such that: 17
Problem definition Constraints (Ω): - Lower and upper bounds - Linear inequalities - Black-box constraint 18
Nelder-Mead Simplex method - Constraints Extreme barrier approach Consequences: - Rapid degeneration of simplex vertices - Unexplored area near to boundary 19
Directional Direct-Search methods - Constraints Bounds Linear inequalities Black-box constraint – Extreme barrier approach 20
Results Academic Problems (Dimension = 8) Number of function evaluations 1 E+02 f(xfinal) - f(xoptimum) 1 E+00 1 E-02 1 E-04 1 E-06 1 E-08 1 E-10 1 E-12 1 E-14 1 E+03 1 E+01 P 2 P 3 P 4 P 5 P 6 P 7 Problem P 8 P 9 P 10 P 11 P 12 P 1 P 2 NM Simplex 5, 90 SID-PSM 0, 08 DS-RANDOM 2, 36 0% -49, 57 % -18, 62% P 3 P 4 P 5 P 6 P 7 Problem P 8 P 9 P 10 P 11 P 12 21
Results Number of function evaluations Academic Problems (Dimension = 20) f(xfinal) - f(xoptimum) 1 E+03 1 E+01 1 E-03 1 E-05 1 E-07 1 E-09 P 1 P 2 P 3 P 4 Problem P 5 P 6 P 7 P 8 1 E+04 1 E+02 1 E+00 P 1 P 2 P 3 P 4 P 5 P 6 P 7 P 8 Problem NM Simplex SID-PSM DS-RANDOM 709, 79 0, 28 90, 25 0% -81, 46 % -77, 67% 22
Numerical Results First scaling Best lossfunction value Channa punctata Pleuroxus aduncus Pleuroxus striatus Credipula fornicata Pleuronectes platessa NM Simplex 1, 9665 E+00 2, 0020 E+03 1, 0213 E+00 3, 0430 E+03 3, 6570 E+00 3, 1970 E+03 3, 8032 E+00 2, 4000 E+03 1, 4233 E+01 3, 0000 E+03 Also best SID-PSM 5, 2173 E-01 1, 6002 E+04 1, 0218 E+00 1, 6000 E+04 9, 5702 E-01 1, 6008 E+04 3, 4061 E+00 1, 8088 E+04 5, 8704 E+00 3, 0008 E+04 Best number of functions evaluations DS-RANDOM 3, 1470 E-01 1, 6000 E+04 2, 9992 E+00 1, 3745 E+04 1, 1759 E+00 1, 1789 E+04 3, 5250 E+00 2, 4000 E+04 3, 0093 E+00 3, 0000 E+04 Dimension 8 8 8 12 15 23
Numerical Results Second scaling Best lossfunction value Channa punctata Pleuroxus aduncus Pleuroxus striatus Credipula fornicata Pleuronectes platessa NM Simplex 1, 9665 E+00 2, 0020 E+03 1, 0213 E+00 3, 0430 E+03 3, 6570 E+00 3, 1970 E+03 3, 8032 E+00 2, 4000 E+03 1, 4233 E+01 3, 0000 E+03 Also best SID-PSM 1, 9621 E-03 1, 7520 E+03 1, 9999 E+00 1, 2710 E+03 2, 9997 E+00 1, 7240 E+03 5, 0293 E+00 2, 4004 E+04 2, 2216 E+00 2, 4330 E+03 Best number of functions evaluations DS-RANDOM 7, 9500 E-01 1, 4784 E+04 4, 2446 E-01 6, 9800 E+02 6, 4884 E-01 3, 8070 E+03 2, 8043 E+00 2, 4000 E+04 2, 0264 E+00 6, 1560 E+03 Dimension 8 8 8 12 15 24
Simplex 2. 0 Minimum
Numerical Results First scaling Best lossfunction value Channa punctata Pleuroxus aduncus Pleuroxus striatus Credipula fornicata Pleuronectes platessa Also best NM Simplex SID-PSM 1, 9665 E+00 2, 0020 E+03 1, 0213 E+00 3, 0430 E+03 3, 6570 E+00 3, 1970 E+03 3, 8032 E+00 2, 4000 E+03 1, 4233 E+01 3, 0000 E+03 1, 9621 E-03 1, 7520 E+03 1, 9999 E+00 1, 2710 E+03 2, 9997 E+00 1, 7240 E+03 5, 0293 E+00 2, 4004 E+04 2, 2216 E+00 2, 4330 E+03 Best number of functions evaluations DS-RANDOM “Simplex 2. 1” 7, 9500 E-01 1, 4784 E+04 4, 2446 E-01 6, 9800 E+02 6, 4884 E-01 3, 8070 E+03 2, 8043 E+00 2, 4000 E+04 2, 0264 E+00 6, 1560 E+03 6, 3811 E-31 1, 2401 E+04 9, 8659 E-05 1, 5507 E+04 5, 3102 E-05 1, 5543 E+04 2, 4933 E+00 2, 1161 E+04 1, 2404 E+00 1, 3398 E+04 26
Numerical Results Second scaling Best lossfunction value Channa punctata Pleuroxus aduncus Pleuroxus striatus Credipula fornicata Pleuronectes platessa fvalue fevals fvalue fevals Also best Best number of functions evaluations NM Simplex SID-PSM DS-RANDOM 1, 9665 E+00 2, 0020 E+03 1, 0213 E+00 3, 0430 E+03 3, 6570 E+00 3, 1970 E+03 3, 8032 E+00 2, 4000 E+03 1, 4233 E+01 3, 0000 E+03 1, 9621 E-03 1, 7520 E+03 1, 9999 E+00 1, 2710 E+03 2, 9997 E+00 1, 7240 E+03 5, 0293 E+00 2, 4004 E+04 2, 2216 E+00 2, 4330 E+03 7, 9500 E-01 1, 4784 E+04 4, 2446 E-01 6, 9800 E+02 6, 4884 E-01 3, 8070 E+03 2, 8043 E+00 2, 4000 E+04 2, 0264 E+00 6, 1560 E+03 “Simplex Turbo” 6, 3811 E-31 1, 2401 E+04 9, 8659 E-05 1, 5507 E+04 5, 3102 E-05 1, 5543 E+04 2, 4933 E+00 2, 1161 E+04 1, 2404 E+00 1, 3398 E+04 27
Results Pars_init_mypet Pars_init_Channa_punctata Optimum SIMPLEX SID-PSM DS-RANDOM SIMPLEX 2. 0 z 28
Conclusions • Possible existence of different local minimums • SID-PSM and DS-RANDOM present better performance than Nelder-Mead Simplex: • in academic problems, both for final objective function value and total number of function evaluations • in species problems, in terms of lossfunction value • “Simplex 2. 0” presents the best lossfunction values 29
Future research • Explore global derivative-free optimization • Adapt “Simplex 2. 0” approach to SID-PSM method • Develop the Nelder-Mead Simplex for constrained optimization 30
- Slides: 29