IEOR 4004 Midterm Review part I March 10

IEOR 4004 Midterm Review (part I) March 10, 2014

Summary • Modeling optimization problems • Linear program • Simplex method – Algorithm – Initialization – Variants • Sensitivity analysis (in part II) • Duality (in part II) – Upper bounds and Shadow prices – Complementary slackness

Mathematical modeling • Simplified (idealized) formulation • Limitations – Only as good as our assumptions/input data – Cannot make predictions beyond the assumptions We need more maps

Mathematical modeling Problem simplification Model formulation Model Algorithm selection Numeric calculation Problem Interpretation Sensitivity analysis Solution

Mathematical modeling • Deterministic = values known with certainty • Stochastic = involves chance, uncertainty • Linear, non-linear, convex, semi-definite

Formulating an optimization Linear program problem Linear • Decision variables objective • Objective • Constraints • Domains Linear constraints x 1, x 2, x 3, x 4 x 1 Minimize 2 x 2 + 3 x 3 + (1 − 2 x 2)2 (x 1)2 + (x 2)2 + (x 3)2 + (x 4)2 ≤ 2 x 1 − 2 x 2 + x 3 − 3 x 4 ≤ 1 − x 1 + 3 x 2 + 2 x 3 + x 4 ≥ − 2 x 1 x 3 = 1 x 1 ≥ 0 x 2 ≠ 0. 5 x 3 in {0, 1} x 4 in [0, 1] Sign restriction

Linear program • solution = values of variables that together satisfy all constraints {x 1, x 2} x 1 − 2 x 2 + x 3 − 3 x 4 = 1 − x 1 + 3 x 2 + 2 x 3 + x 4 = − 2 x 1 ≥ 0 • • • x 1 = − 1 x 2 = − 1 x 3 = 0 x 4 = 0 x 1 = 4/3 x 2 = 0 x 3 = − 1/3 x 4 = 0 Feasible solution = satisfies also sign constraints Infeasible solution = not feasible Basis = set of m variables (where m = # of equations) Variable in basis = basic variable, all other non-basic Basic solution = set all non-basic variables to zero

Linear program • Feasible region = set of all feasible solutions objective function growth is limited • LP is Infeasible if feasible region empty • LP is Unbounded if the objective function is unbounded over the feasible region objective function grows beyond bounds

Simplex algorithm • Converting to standard form + − 3 x− x 5 = 3 − x 1 + − x+ ’− 34++x 3 x − 3 x + 23 x − x+2’ 432 x − 3 x 4 ≤ x 12 x − x 3 4= 3 54 x 6 = 2 − x 1−+− + ’− 34+2 x 3 x+4+− xx 64− 4≥ x 2+’ 433 x − 22 xx+ 2 x x 13 x + =− 2 z= − 2 x 2 + 3 x’ 3 x 1, x 2 ≥ 0 xx 3’ 3≤ Add slack variables Substitute negative variables Substitute unrestricted variables Dictionary x 5, x 6 ≥ 0 x+4, x− 4 ≥ 0

Simplex method (maximization) START Initial feasible solution Optimality test Are all coefficients in z non-positive ? Alternative solutions? x 5 = 3 − x 1 + 2 x 2 − x’ 3+ 3 x+4 − 3 x− 4 x 6 = 2 − x 1 + 3 x 2 + 2 x’ 3+ x+4 − x− 4 z= − 2 x 2 + 3 x’ 3 xj Pivot xj to the basis Ratio test NO YES optimal solution found Variable selection END xi Find min b/a a = coeff of xi in xj b = value of xj a, b opposite signs xj does not exist LP is unbounded

Simplex algorithm • Anatomy of a dictionary Ratio test: (different sign) basis objective(s) x 5 = 3 − x 1 + 2 x 2 − x 6 = 2 − x 1 + 3 x 2 + z= x 5: 3/1 = 3 x 6: no constraint x’ 3+ 3 x+4 − 3 x− 4 2 x’ 3+ x+4 − x− 4 (same sign) − 2 x 2 + 3 x’ 3 zero negative positive coeff x’ 3 enters x 5 leaves (min ratio)

Simplex/Graphical method • Illustration 150 0 profit: 180 80 max 3 x 13 x +2 x 1 +2 x 2 2 x 1 + x 2 +x 2 x 3≤=80 80 x 1 + 2 x 2 x x 2 + 100 60 x 2 x≤ 100 1 +1 + 4= x 1, xx 2, , xx 3, x≥ 00 4≥ 1 start: x 1=0, x 2=0, x 3=80, x 4=100 x 1 increases until x 4=0 x 2 increases until x 3=0 2 40 z x 1 isoprofit line 3 x 1 +2 x 2 = profit x 3, x 4 are slack variables 20 0 optimal x 3 x 4 x 2 20 40 60 80

Initialization - Phase I • Goal: make all original variables non-basic (=0) • If all-0 doesn’t satisfy a constraint, add artificial variable ai: – add +ai if the right-hand side is > 0 – add –ai if the right-hand side < 0 from equations without artificial variables • Add slack variables (where needed) • New objective: minimize the sum of artificial variables • Starting basis: all artificial + slack variables If optimal value 1 2−−x 3 x 2 x 2 − −x 3 a+ 3 x z = w =s 1 = 2 x Maximize 1+ 3 2 − a 43 x 1 = 5/2 − (7/2)x + (7/4)e + (1/4)a − 3 x (7/4)a − x(5/4)s + (7/4)ethen negative, a 2 x 1=− 3 − 2 (5/4)s − x + 3 x + 2 x + 12− 3 1 2 x + x 1 2 3 4 1 + s 1 =3 1 2 3 4 ≤ x 2 = 0 − −a 3 x 2 x (1/2)s + + (1/2)a −x(1/2)e (1/2)a − 2 x (1/2)s + (1/2)e LP Infeasible = 1 − − x − + e 3− 12+ 3 1 3 + 3 x + 2 x x 2 3 4 3 −− a 22 =− 2 1 2 3 4 = x 4 = 1/2 + (1/2)x 3 + (1/4)s +− 2 x (1/4)e − x(1/4) a 12+ (1/4)a 33 + (1/4)s (1/4)e Else artificial 1 +drop 3 + x ≥ 1 − e = 1 2 3 4 + a 3 − a 2+−xa 1 3− 4 x 2 − 3 x 3 − 3 x 43 − e 3 variables w = − 3 and − 3 x 3 7 x 3 − sa 12+ ea 33 wz = 2 x 2 − − − introduce z

Degeneracy • Basic solution is degenerate if some basic variable is zero – multiple dictionaries max 3 x 1 +2 x 2 x 1 + x 2 + x 3 = 80 2 x 1 + x 2 + + x 4 = 100 6 x 1 + x 2 + +x 5 = 180 x 1, x 2, x 3, x 4, x 5 ≥ 0 x 1 = 20 + x 3 − x 4 x 2 = 60 − 2 x 3 + x 4 x 5 = − 4 x 3 + 5 x 4 z = 180 − x 3 − x 4 optimal x 1 = 20 + 0. 2 x 3 − 0. 2 x 5 x 2 = 60 − 1. 2 x 3 + 0. 2 x 5 x 4 = 0. 8 x 3 + 0. 2 x 5 z = 180 − 1. 8 x 3 − 0. 2 x 5 x 1=20 x 2=60 x 3=x 4=x 5=0 80 60 40 20 20 40 60 80 x 1 = 20 + 0. 25 x 4 − 0. 25 x 5 x 2 = 60 − 1. 5 x 4 + 0. 5 x 5 x 3 = 1. 25 x 4 − 0. 25 x 5 z = 180 − 2. 25 x 4 + 0. 25 x 5 optimal ? ? ?

Upper bounded Simplex • Anatomy of a dictionary Ratio test: (different sign) basis objective(s) bounds x 5 = 3 − x 1 + 2 x 2 − x 3 + 3 x 4 x 6 = 1 − x 1 + 3 x 2 + 2 x 3 + x 4 z= − 2 x 2 + 3 x 3 0 ≤ x 1, x 2 , x 4, x 5 0 ≤ x 3 ≤ 2 0 ≤ x 6 ≤ 4 0 ≤ x’ 6≤ 4 1. (different sign) x 5: 3/1 = 3 2. (entering variable) x 3: 2 3. (same sign) x 6: (4− 1)/2 = 1. 5 (same sign) x 3 enters substitute x 6 = 4 − x’ 6 x 6 leaves (min ratio)

Dual Simplex • Anatomy of a dictionary infeasible! basis objective(s) Ratio test: x 1: no constraint (positive) negative x 2: 5/2 = 2. 5 value x 3: no constraint x 5 = − 3 − x 1 + 2 x 2 − x 3 + 4 x 4 x 4: 7/4 = 1. 75 x 6 = 1 − x 1 + 3 x 2 + 2 x 3 + x 4 z= − 5 x 2 − 3 x 3 − 7 x 4 dually feasible (all coefficients non-positive) x 5 leaves x 4 enters (min ratio)