Branch and bound Pasi Frnti 15 10 2018

Branch and bound Pasi Fränti 15. 10. 2018

General search strategies Backtracking Explore all alternatives • • Solution constructed by stepwise choices Decision tree Guarantees optimal solution Exponential time (slow)

Knapsack problem Simplified Input: Weight of N items {w 1, w 2, . . . , wn} Knapsack limit S Output: Selection for knapsack: {x 1, x 2, …xn} where xi {0, 1}. 5 3 Sample input: ci=wi={2, 3, 5, 7, 11} S=15 7 2 11 Max 15

Knapsack as decision tree wi={2, 3, 5, 7, 11} Yes Chooce 2 No 2 Yes No Yes Max 15 Chooce 3 5 Yes Chooce 5 2 No Yes 5 10 7 3 17 No Yes 2 3 8 No 5 No Yes 10 12 5 … 28 - No Yes Chooce 7 No 5 14 9 … 12 2 15 8 No 14 13 26 15 10 3 12 5 … 7 - … 12 -

Bounding criterion Brute force: search full tree N choices Greedy: take if fits in 2 O(2 N) 5 17 2 5 10 10 12 5 - 7 5 14 3 2 9 2 8 15 8 … 28 10 12 - 5 3 10 3 12 - 5 … 14 13 26 15 7 - … 12 -

Bounding criterion +wnext > S Must be sorted! wi={2, 3, 5, 7, 11} Chooce 2 2 Chooce 3 Chooce 5 Chooce 7 Chooce 11 5 10 12 5 5 5 14 - 2 7 3 2 9 2 13 8 15 8 - 5 3 10 3 14 12 - 5 7 - 11

Empty space for notes

Traveling salesman problem Input: Graph (V, E) Problem: Shortest tour visiting all nodes. Closed loop: Optimal Length = A-B-C-F-H-E-D-G-A = 22 Open loop: Optimal Length = D-G-A-B-C-F-H-E = 17

Brute force TSP Brute force: search full tree O(N!) 2 2 277 2 2 1 153 157 2 283 2 2

Bounding criterion 1 Best found so far 28 Best candidate: B 6 A 2 C E 6 New so far: 5 G 1 4 H 32 4 D F B 17 15 A C E G 5! = 120 H combinations F D

Bounding criterion 2 Nearest neighbors Best candidate: 28 All nodes must be visited. Find the smallest possible incoming link. B 17 A C Lower bound: 33 4 E 1 G 1 H 3 3 F 4 D 16 4+1+1+3+3+4

Empty space for notes

Branch and Bound Algorithm: Scheduling Problem Material by A. Mirhashemi Input of the problem: Ø A number of resources A B Ø A number of tasks Output of the problem: Ø A sequence of feeding the tasks to resources to minimize the required processing time C 1 2 3 4

Application 1 Digital processing: Each resource is a processor. All tasks need to pass trough all processors in the fix sequence A, B, C but depending on the task it takes different time for each processor to process them. For example : Processor A: Scanning Processor B: Making a PDF Processor C: Exporting a PDF Task 1: Task 2: Task 3: Task 4: A one page plain text document A 10 page document with pictures A 5 page html document. …

Application 2 Production line: Each product (task) need to pass trough all machines (resources) in the production line but, the time depends on what kind of customization the customer has ordered for that production. For example: Machine A: Machine B: Machine C: Solding Painting Packaging Task 1: Task 2: Task 3: Task 4: A black car with airbag A red car without airbag with CD player A white car with leather seats …

Different tasks take different time to be processed in each resource 1 2 3 4 A B C 7 5 6 3 6 5 4 4 7 2 1 3

Tasks can be done in any order N! Possible different sequences 3 1 4 P 2 2 N 1 4 4 2 3

Decision tree (Brute force) 2 d n a rt 1 or pa F 1 2 3 4 4 4 2 2 1 3 3 F 4 4 1 3 1 4 4 1 1 3 3 1 4 4 4 Start 2 2 3 3 1 4 3 2 4 d n a rt 3 or pa 4 2 3 4 2 Start 1 1 1 2 2 2 1 2 3 3 3 1 1 2 2 1

Greedy Algorithm A possible greedy algorithm might start with selecting the fastest tasks for processor A. A B C 1 7 6 7 2 5 5 2 3 6 4 1 4 3 A : 4 2 3 1

Greedy solution T(4, 2, 3, 1) = 34 4 2 3 4 2 4 A B C 1 7 6 7 2 5 5 2 3 6 4 1 4 3 1 1 3 2 3 1

Optimal solution T(4, 1, 2, 3) = 26 1 4 B C 1 7 6 7 2 5 5 2 3 6 4 1 4 3 3 2 4 A 2 3 1 2 3

Branch and bound Algorithm Define a bounding criteria for a minimum time required by each branch of the decision tree For level 1: For level 2:

Level 1 A B C 1 7 6 7 2 5 5 2 3 6 4 1 4 3 b(1)= 7+(6+5+4+4)+1=27 b(2)= 5+(6+5+4+4)+1=25 Minimum b(3)= 6+(6+5+4+4)+2=27 b(4)= 3+(6+5+4+4)+1=23 Start This next Bounds: 1 2 3 4 T ≥ 27 T ≥ 25 T ≥ 27 T ≥ 23

Level 2 b(4, 1)= (3+7)+(6+5+4)+1=26 b(4, 2)= (3+5)+(6+5+4)+1=24 b(4, 3)= (3+6)+(6+5+4)+2=26 A B C 1 7 6 7 2 5 5 2 3 6 4 1 4 3 Minimum This next Bounds: T ≥ 26 T ≥ 24 T ≥ 26

Solve the branch 4 -2 -x-x Bounds: T ≥ 26 T ≥ 24 T ≥ 26 Tmin(4, 2, x, x)= 29 Actual: T(4, 2, 1, 3) = 29 T(4, 2, 3, 1) = 34 A B C 1 7 6 7 2 5 5 2 3 6 4 1 4 3

Solve the branch 4 -2 -x-x Bounds: T ≥ 26 Tmin(4, 1, x, x)= 26 Actual: T ≥ 24 A B C 1 7 6 7 2 5 5 2 3 6 4 1 4 3 T ≥ 26 Tmin(4, 2, x, x)= 29 T(4, 1, 2, 3) = 26 T(4, 1, 3, 2) = 28 Tmin(4, 3, x, x)= 31 Actual: T(4, 3, 1, 2) = 31 T(4, 3, 2, 1) = 34

Solve the other branches Can be skipped Bounds: Actual Time: A B C 1 7 6 7 2 5 5 2 3 6 4 1 4 3 Start 1 2 3 4 T ≥ 27 T ≥ 25 T ≥ 27 T ≥ 23 T = 26 Must be solved

b(2, 1)= (5+7)+(6+4+4)+1=27 b(2, 3)= (5+6)+(6+4+4)+3=28 b(2, 4)= (5+3)+(6+4+4)+1=23 A B C 1 7 6 7 2 5 5 2 3 6 4 1 4 3 The only candidate that can outperform T(4, 1, 2, 3) is T(2, 4, …) so we calculate it: Actual T(2, 4, 1, 3) = 29 Actual T(2, 4, 3, 1) = 34

So the best time is T(4, 1, 2, 3) and we don’t need to solve the problem for any other branch because we now their minimum time, already. A B C 1 7 6 7 2 5 5 2 3 6 4 1 4 3 Start Bounds: Actual Time: 1 2 3 4 T ≥ 27 T ≥ 25 T ≥ 27 T ≥ 23 T = 29 T = 26 Bounds greater than 26!

Summary • Using only the first level criteria we reduce the problem by 50% (omitting 2 main branches). • Using the second level criteria we can reduce even more.

Empty space for notes

Leftover material:

Knapsack problem wi={2, 3, 5, 7, 11} Branch-and-bound 0 2 Max 15 2 3 3 5 2 5 3 5 12 5 10 7 5 0 5 9 7 12 0 3 7 7 7 9 2 13 3 10 11 14 0 11 11 11 2 0 3 11 0

Traveling salesman problem A 2 4 8 H 11 H 15 D F 12 F B C E 14 F 6 11 E 16 D 23 A B 6 G 9 C 13 17 13 G 7 F 15 15 C D 6 G 9 13 17 G 20 D G 2 D 3 11 F E 17 F F 20 H 20 22 G 24 27 D A H 22 G 24 A

Traveling salesman problem Input: graph (V, E) Problem: Find shortest path via all nodes and returning to start node.

Just in case if needed 117 216 110 246 199 121 79 182 315 142 170 242 191 148 231 136 78 120 126 149 89 116 234 170 51 112 79 131 109 86 73 163 143 72 63 90 53 191 178 59 58 116 105 135 27