MOEAD Use of Traditional Optimization Techniques in Evolutionary

MOEA/D: Use of Traditional Optimization Techniques in Evolutionary Multiobjective Optimization Qingfu Zhang Department of Computer Science City University of Hong Kong w: http: //www 6. cityu. edu. hk/stfprofile/qingfu. zhang. htm e: qingfu. zhang@cityu. edu. hk 1

q Most EC algorithms are for optimization. q Popular combinations: traditional opt methods = solution generators. q MOEA/D: traditional opt techniques are used in a different way. 2

Outline q q Basic Concepts Ideas in MOEA/D A Simple Version Design Issues o Decomposition Methods o Search Methods o Collaboration q Other Topics q Conclusions & Further Works 3

Outline q q Basic Concepts Ideas in MOEA/D A Simple Version Design Issues o Decomposition Methods o Search Methods o Collaboration q Other Topics q Conclusions & Further Works 4

Multiobjective Optimization Problem (MOP) objective vector search space Why multiobjective: o By nature, many real-life problems have multiple objectives. o Decision makers (DM) or modellers don’t know how to combine them into one. o DMs want to know trade-off relationship among these objectives. 5

Dominance : How to compare two solutions q y dominates x iff o y is no worse than x in any objs, and o y is better than x in at least one obj. Search space F(A) A Feasible objective region F(B) B F(C) C F B dominates A. B and C are not comparable. 6

Pareto Optimal Solutions= Best Trade-off Candidates Search space PS Feasible objective region PF q x is Pareto optimal iff no other solution dominates it. q A (rational) decision maker doesn’t like non-Pareto optimal solutions. q Pareto set (PS)= the set of all Pareto optimal solutions in the x-space. q Pareto front (PF) = the image of the PS in the F-space. 7

Basic Strategies in Multicriterion Decision Making q A-priori The final solution q A-posteriori A number of Pareto optimal solutions The final solution q Interactive ? 8

Evolutionary Multiobjective Optimization (EMO) q Generate a number of solutions to approximate the PF (PS). q Help the DM to understand her problem and find her preferred solutions. o Popular and well respected in multicriterion decision making area. o Hottest research area in EC. 9

Major EMO Algorithms q Pareto Dominance Based: NSGA-II, SPEA 2, PAES 2…. q Performance Indicator Based: SMS-EMOA , Hy. PE…. q Decomposition Based. 10

Outline q q Basic Concepts Ideas in MOEA/D A Simple Version Design Issues o Decomposition Methods o Search Methods o Collaboration q Other Topics q Conclusions & Further Works 11

Decomposition in EMO Decomposition has been used to some extent in EMO area for many years. Examples include: ü ü ü ü ü MOGLS (Ishibuchi, et al, 1998, Jaszkiewicz, 2002) MOSA[Ulungu , et al, 1999] MOTS [Hansen, 1997]. 2 PLS [Paquete & Stutzle, 2003] DWA (Jin et al, 2001 ) MSOPS (Hughes, 2003) MOTGA (Alves, 2007) CMOGA (Murata, et al, 2001) MOEA/D (Zhang & Li, 2007) …. These algorithms use traditional aggregation approaches. 12

MOEA/D=Decomposition + Collaboration q Decomposition (from traditional opt) o Decompose the task of approximating the PF into N subproblems. Each subproblem can be single objective or multiobjective. q Collaboration (from EC) o N agents (procedures) are used. Each agent is for a different subproblem. o These N subproblems are related to one another. N agents can solve these subproblems in a collaborative manner. 13

Major Issues q Problem decomposition and neighbourhood structure. q Search method used by each agent: memory and reproduction operators. q Collaboration. 14

Outline q q Basic Concepts Ideas in MOEA/D A Simple Version Design Issues o Decomposition Methods o Search Methods o Collaboration q Other Topics q Conclusions & Further Works 15

Problem Decomposition q Weighted Sum Approach • • Works for convex PF. Doesn’t work if the PF is not convex. Pareto optimal solution Approximation of the PF N single obj opt suproblems 16

q 17

q 18

19

q Neighbourhood: measure the relationship among subproblems ü Neighbouring subproblems should have similar objective functions and thus similar optimal solutions with high probability. ü The objective of a subproblem is continuous of its weight vector. Neighborhood of subproblem 5 ü Many different ways for defining neighbourhood structure.

Search Method q 21

Collaboration Among Different Agents q Mating selection Each agent borrows solutions from its neighbors for generating new solutions. q Replacement Each agent sends new solutions to its neighbors. 22

Decomposition, Neighborhood, Memory f 2 f 1 q At each generation, each agent does the following: 1. Mating selection: borrow some solutions from some neighbours (collaboration). 2. Reproduction: generate a new solution by using reproduction operators on its own solution and borrowed solutions. 3. Replacement: 1. replace its old solution by the new one if the new one is better than old one for its objective. 2. pass the new solution on to some of its neighbours, each of them replaces its old solution by this new one if it is better for its objective. (collaboration, neighbourhood) 23

Experimental Results q IGD CPU time 24

Test Instances with Complicated PS Shapes PS MOEA/D NSGA-II MOEA/D is better. 25

Dynamical Resource Allocation q Different subproblems (agents) require different amounts of computational resources. q Each subproblem (agent) has a utility value, which measures the likelihood of further improvement: q At each generation, a small number of agents are selected based on utility values and receive computational resources. Different utility functions have been proposed in [Zhang et al, 2009, Zhou and Zhang, 2016]. Related work: Unsolved (solved) subproblem in MOEA/D- AMS (Chiang & Lei, 2011) 26

Some Remarks q 27

Outline q q Basic Concepts Ideas in MOEA/D A Simple Version Design Issues o Decomposition Methods o Search Methods o Collaboration q Other Topics q Conclusions & Further Works 28

Penalty Based Boundary Interaction (PBI) (Zhang & Li 2007) q 29

q 30

q 31

Inverted PBI (IPBI, Sato, 2014) q 32

Constrained Decomposition x-space o Single objective decomposition (Wang et al, 2015) Good for diversity. o Multiple-objective to multiple-objective (M 2 M) (Liu et al, 2014) MOEA/D-M 2 M: Each agent can uses an EMO algorithm. 33

Disadvantage of Prefixed Decomposition If the PF shape is unknown, and very different from a simplex. Use of prefixed uniform search direction lines will waste lots of computational resources, and may not be able to find a good approximation to the PF. 34

Adaptive Decomposition Subproblem 1 Subproblem 2 estimated PF Subproblem 3 target point Subproblem 4 Subproblem 5 1. Estimate the PF shape. 2. Choose a set of target points in the estimated PF. § § Without the DM’s preference, the distribution of these target points should be as uniform as possible. The distribution of these target points can be determined by the DM. 3. For each target point, construct a subproblem such that its optimal solution = the target point. 35

q Qi. et al 2013 36

q 37

Outline q q Basic Concepts Ideas in MOEA/D A Simple Version Design Issues o Decomposition Methods o Search Methods o Collaboration q Other Topics q Conclusions & Further Works 38

Under MOEA/D framework, almost all the single objective heuristics have been generalized to EMO. 39

Probabilistic Model Based Search f 2 f 1 q Prob. model based search (EDA, ACO, Cross Entropy Search. . ): o At each generation, it maintains and updates a prob. model. o models can model the distribution of promising solutions. o New solutions are sampled from the model. q If each agent records and maintains a different model in MOEA/D, the space complexity will be too high. 40

q MOEA/D-ACO for multiple TSP (Ke et al 2013) o Decomposition ü ü A MOP a number single obj opt subproblems. Each ant is for different subproblem. Neighbourhood relationship is pre-defined. Each ant has 5 neighbours. Ants are divided into several groups. 14 ants / 2 groups. o Memory: ü Each group maintains a pheromone matrix. ü Each ant records its best solution found so far. ü Each ant has its own heuristic info matrix. 41

o Solution Construction: ü Each ant generates a new solution using: the pheromone matrix of its group+ its own heuristic info + its own current solution. o Memory Updating: ü The pheromone matrix of each group is updated using all new solutions generated by its ants. ü The current solution of each ant is updated by using its own new solutions and its neighbouring new solutions. 42

Experimental Results on Biobjective TSP: MOEA/D+ACO is better than Bicriterionant. 43

Outline q q Basic Concepts Ideas in MOEA/D A Simple Version Design Issues o Decomposition Methods o Search Methods o Collaboration: mating selection and replacement q Other Topics q Conclusions & Further Works 44

Decomposition, Neighborhood, Memory f 2 f 1 q At each generation, agent i does the following 1. Mating selection: obtain the current solutions of some neighbours. 2. Reproduction: generate a new solution by applying reproduction operators on its own solution and borrowed solutions. 3. Replacement: 1. its old solution by the new one if the new one is better than old one for its objective. 2. pass the new solution on to some of its neighbours, each of them replaces its old solution by this new one if it is better for its objective 45

Mating Selection: how to select parent solutions q One neighborhood (Zhang & Li, 2007) q One neighborhood+ Random Selection (Li & Zhang, 2009) The agent borrows solutions from its neighborhood. poor at diversity. With some probability, randomly selects one from the whole population. 46

q Ensemble of neighborhoods (Zhao et al, 2013) q Niche guided selection (Jiang & Yang, 2013) Several neighborhood sizes are used and dynamically adjusted. If its neighborhood is too crowded, selects one from the whole population.

Replacement Current solution Parent solution New solution replaced solution q 48

Current solutions parent solutions new solution replaced solution Case 1 Case 2 q Global replacement: (Wang et al 2016) o Find the best agent for the new solution. o Send the new solution to the best agent for solution replacement. 49

Current solution New solution before replacement after replacement q Stable matching based (Li et al, 2014) o Generational selection. Each of N agents selects one solution from M candidate solutions (M>N). o Model selection as matching problem: Let agent=girl, solution=boy, define their preferences. o Use stable matching algorithm 50

Outline q q Basic Concepts Ideas in MOEA/D A Simple Version Design Issues o Decomposition Methods o Search Methods o Collaboration q Other Topics q Conclusions & Further Works 51

Other Topics q Interactive MOEA/D q MOEA/D for many obj optimization q Higher-order approximation of the PF/PS by MOEA/D q Relationship with other EMO algorithms q Decomposition + other strategies such as Pareto Local Search q Parallel and distributed MOEA/D q Theory of MOEA/D q Applications 52

Resources q MOEA/D Web: https: //sites. google. com/view/moead 53

q Survey papers: 54

Outline q q Basic Concepts Ideas in MOEA/D A Simple Version Design Issues o Decomposition Methods o Search Methods o Collaboration q Other Topics q Conclusions & Further Works 55

Conclusion and Future Research Topics q Ideas from traditional optimization can be used for designing population based search algorithms. q MOEA/D= Decomposition + Collaboration q Single search methods can be generalized to multiobjective optimization in MOEA/D framework. q EMO algorithms can also used in MOEA/D framework. q MOEA/D is not mature. 56

Future Research Topics q What properties are common in real world MOPs? o Neighbouring subproblems have similar optimal solutions. o Regularity property: for a continuous m-obj optimization problem, dimension of PF (PS)=m-1. o ? ? Algorithm design = make the best use of problem properties. q Why collaboration (population) is useful? o In case of two agents for one problem, collaboration is useful (Huberman, et al, Science, 1997) How about N agents for N different problems as in MOEA/D? 57

q Many Obj Optimization o Using a finite number of points to approximate a high-D PF may not be realistic What role can EMO play in the interaction with decision makers? Objective reduction, trade-off info, higher order approximation, visualization. q Pareto Local Search o Local search is a basic building block for single obj optimization o Pareto local search is often very costly. Is it possible to develop an efficient PLS ? 58

This work is collaboration with Ahmad Alhindi, Robert Battiti, Xinye Cai, Kalyan Deb, Bilel Derbel, Juan J. Durillo, João A. Duro, Zhu Fan, Maogu Gong, Faqing Gu, Jalalian, Hamid, Hisao Ishibuchi, Muhammad Asif Jan, Licheng Jiao, Yong-Chang Jiao, Liangjun Ke, Wali Khan, Andreas Konstantinidis, Sam Kwong, Hui Li, Ke Li, Arnaud Liefooghe, Xi Lin, Bo Liu, Hailin Liu, Wudong Liu, Xiaoling Ma, Álvaro Rubio-Largo, Dhish Kumar Saxena, Arnab Sinha, P. N. Suganthan, Jianyoung Sun, Nasser Tairan, Edward Tsang, Miguel A. Vega-Rodraguez, Botond Virginas, Handing Wang, Luping Wang, Rui Wang, Zhenkun Wang, X Yao, Zexuan Zhu, Aimin Zhou. 59

Thank you very much! 60