Unconstrained Optimization One Dim Minimization Steepest Descent Method

Unconstrained Optimization • • 최적설계 일차원 최소화 (One Dim. Minimization) 최속강하법 (Steepest Descent Method) 공액경사도법 (Conjugate Gradient M. ) 뉴톤의 방법 (Newton’s Method) 유사뉴톤방법 (Quasi-Newton Method) 공학적 응용 (Engineering Applications) 최적설계 변환법 (Transformation Methods) H. J. Yim@KMU

5. 2 수치알고리즘의 일반개념 • Analytical approach: - Optimality conditions - Candidate local minimum design • Numerical methods: - Initial design - Iterations until optimality conditions satisfied 최적설계 H. J. Yim@KMU

5. 2. 1 A General Algorithms Vector form 현재 설계에서의 미소 변화 Component form 최적설계 H. J. Yim@KMU

Change in design 현재 설계에서의 미소 변화 Step size Search direction 최적설계 H. J. Yim@KMU

Change in design 최적설계 H. J. Yim@KMU

5. 2. 2 Descent Step Idea (강하, 감소) Desirable direction of design change (바람직한 설계 변화의 방향) 최적설계 H. J. Yim@KMU

Searching Direction (탐색방향) 최적설계 H. J. Yim@KMU

Descent direction (강하방향) Descent condition (강하조건) 최적설계 H. J. Yim@KMU

• 5. 2. 3 Convergence of Algorithms The property of convergence to a local optimum point irrespective of the starting point • 5. 2. 4 Rate of Convergence Faster algorithms use 2 nd order information of the function Newton’s Method, Quasi-Newton method 최적설계 H. J. Yim@KMU

5. 3 One-dimensional Minimization Step size Search direction 최적설계 H. J. Yim@KMU

5. 3 One-dimensional minimization • 5. 3. 1 Problem Definition • 5. 3. 2 Equal Interval Search (등간격 탐색) • 5. 3. 3 Golden Section Search (황금분할 탐색) • 5. 3. 4 Polynomial Interpolation (다항식 보간법) 최적설계 H. J. Yim@KMU

Problem Definition • 설계변화 탐색 방향을 찾았다면 known unknown 최적설계 H. J. Yim@KMU

Change in design 최적설계 H. J. Yim@KMU

Analytical Method for step size 를 최소화하기 위한 필요조건 최적설계 충분조건 H. J. Yim@KMU

Numerical method for step size Unimodal function if = 최적설계 H. J. Yim@KMU

Equal Interval Search (등간격 탐색) 초기추정 최적설계 H. J. Yim@KMU

5. 3 Golden Section Search (황금분할탐색) • Initial bracketing (최소치의 초기 추정) Fibonacci sequence – Golden ratio • Reduction of interval of uncertainty (불확정 구간의 축소) 최적설계 H. J. Yim@KMU

Fibonacci Sequence 피보나치 수열 황금비 최적설계 H. J. Yim@KMU

5. 4 Steepest Descent Method (최속강하법) • 탐색방향을 구하는 방법 Desirable direction of design change (바람직한 설계 변화의 방향) • First order method: Steepest Descent Method • Second order method: Newton’s Method 최적설계 H. J. Yim@KMU

Steepest Descent Algorithm 최적설계 H. J. Yim@KMU

5. 5 Conjugate Gradient Method (공액경사도방법) By Fletcher & Reeves (1964) Very simple and effective modification of the Steepest Descent Method 최적설계 H. J. Yim@KMU

5. 5 Conjugate Gradient Method (공액경사도방법) 최적설계 H. J. Yim@KMU

5. 6 Newton’s Method • 2 nd order information 사용 최적설계 H. J. Yim@KMU

5. 6 Newton’s Method • Modified Newton’s Method 최적설계 H. J. Yim@KMU

Example 5. 16 최적설계 H. J. Yim@KMU

Marquardt Modification DFP (Davidon, Fletcher, Powell) Approximate Inverse of Hessian BFGS (Broyden, Fletcher, Goldfarb, Shannon) Update Hessian at every iteration 최적설계 H. J. Yim@KMU