Tutorial on Paradis EO Parallel Models for the

Tutorial on Paradis. EO: Parallel Models for the TSP

Contributions EO (A framework of Evolutionary Algorithms) Techniques related to multi-objective optimization Solution-based metaheuristics Paradis. EO (Parallel and distributed metaheuristics) http: //www. lifl. fr/~cahon/paradis. EO/ Hill Climbing Simulated Annealing Tabu search http: //www. lifl. fr/~cahon/paradis. EO/ls/ Parallelism (Partitioning solutions at several steps) Cooperation (Algorithms exchange solutions) Ex. Island cooperation

A case study : the «Traveling Salesman Problem» • From a full graph of vertices find the shortest hamiltonian cycle. • NP-hard • EO Designing a G. A. • Paradis. EO Illustrating some forms of parallelism

TSP Case-study – Components > cd $HOME/tsp/src/ > ls –al ~/tsp/ src/ main. cpp – main workbench file – GAs, LS etc. DEFINITION route. h – chromosome definition route_init. h – random chr. initialization operator EVALUATION part_route_eval. h – partial fitness eval. Operator merge_route_eval. h – fitness evaluation aggregation op. route_eval. h – full fitness evaluation operator OPERATORS order_xover. h – Order. Xover crossover operator edge_xover. h – Edge. Xover crossover operator city_swap. h – City. Swap mutation operator {two_opt_init, two_opt_next, two_opt_incr_eval}. h

An unifying view of three hierarchical levels n For both the population-based and solution-based metaheuristics, n n n The deployment of concurrent independent/cooperative metaheuristics The parallelization of a single step of the metaheuristic (based on distribution of the handled solutions) The parallelization of the processing of a single solution

DEFINITION TSP – The Initial Algorithm eo. Pop <Route> ox_pop (POP_SIZE, route_init); /* Population and initializer */ eo. Gen. Continue <Route> ox_cont (NUM_GEN); continuation criterion peo. Seq. Pop. Eval <Route> ox_pop_eval (full_eval); evaluation method eo. Stoch. Tournament. Select <Route> ox_select_one; eo. Select. Number <Route> ox_select (ox_select_one, POP_SIZE); selection strategy eo. SGATransform <Route> ox_transform (order_cross, CROSS_RATE, city_swap_mut, MUT_RATE); peo. Seq. Transform <Route> ox_seq_transform (ox_transform); ops. transformations eo. EPReplacement <Route> ox_replace (2); replacement strategy eo. Check. Point <Route> ox_checkpoint (ox_cont); /* Checkpoint */ peo. EA <Route> ox_ea (ox_checkpoint, ox_pop_eval, ox_select, ox_seq_transform, ox_replace); ox_ea (ox_pop); /* Application to the given population */

The parallel evaluation of the population Recombination, mutation Selection Replacement Computed values are coming back Full Eval. Partitionning Full Evaluating nodes Full Eval.

TSP – The Parallel Evaluation of the Population eo. Pop <Route> ox_pop (POP_SIZE, route_init); /* Population and initializer */ eo. Gen. Continue <Route> ox_cont (NUM_GEN); peo. Para. Pop. Eval <Route> ox_pop_eval (full_eval); eo. Stoch. Tournament. Select <Route> ox_select_one; eo. Select. Number <Route> ox_select (ox_select_one, POP_SIZE); eo. SGATransform <Route> ox_transform (order_cross, CROSS_RATE, city_swap_mut, MUT_RATE); peo. Seq. Transform <Route> ox_seq_transform (ox_transform); eo. EPReplacement <Route> ox_replace (2); eo. Check. Point <Route> ox_checkpoint (ox_cont); /* Checkpoint */ peo. EA <Route> ox_ea (ox_checkpoint, ox_pop_eval, ox_select, ox_seq_transform, ox_replace); ox_ea (ox_pop); /* Application to the given population */

The parallel evaluation of a single solution Recombination, mutation Computation f( f 1, f 2, …fn ) Partially evaluated solutions Partial Eval. e 1 Replication Partial Eval. e 2 Evaluating nodes Partial Eval. en Selection Replacement

The distributed evaluation step - provided peo. Pop. Eval Route peo. Para. Pop. Eval Route () eo. Pop & Route vector< eo. Eval. Func<EOT> > eo. Agg. Eval. Func<EOT> different partial evaluation functions aggregation of fitness values Features

TSP – The Parallel Evaluation of the Fitness Function Merge. Route. Eval merge_eval; std : : vector <eo. Eval. Func <Route> *> part_eval; for (unsigned i = 1 ; i <= NUM_PART_EVALS ; i ++) { part_eval. push_back (new Part. Route. Eval ((float) (i - 1) / NUM_PART_EVALS, (float) i / NUM_PART_EVALS)); } . . . eo. Gen. Continue <Route> ox_cont (NUM_GEN); peo. Para. Pop. Eval <Route> ox_pop_eval (part_eval, merge_eval); eo. Stoch. Tournament. Select <Route> ox_select_one; eo. Select. Number <Route> ox_select (ox_select_one, POP_SIZE); . . .

The cooperative island model n n n Several asynchronous GAs (coarse grain) n Different sub-populations Asynchronous migration of individuals n Different stopping criteria n Convergence n Frequency / generations Cycle of evolutions Miscellaneous n Heterogeneous parameters, operators, algorithms, … Migrations Cycle of evolutions

Manager of asynchronous migrations eo. Async. Island. Mig () Route Event « Immigration is required ? » eo. Continue Features eo. Selection of individuals to emigrate eo. Replacement eo. Topology - provided eo. Pop & Route Integration of immigrants into the local population Topology (Ring, Full, …)

TSP – Synchronous Island Model /* Migration occurs periodically */ eo. Periodic. Continue <Route> ox_mig_cont (MIG_FREQ); eo. Random. Select <Route> ox_mig_select_one; eo. Select. Number <Route> ox_mig_select (ox_mig_select_one, MIG_SIZE); /* Immigrants replace the worse individuals */ eo. Plus. Replacement <Route> ox_mig_replace; peo. Sync. Island. Mig <Route> ox_mig (MIG_FREQ, ox_mig_select, ox_mig_replace, topo, ox_pop); // migration frequency // strategy of selection // strategy of replacement // topology – ring topology // source population of emigrants // destination population for imigrants

TSP – Asynchronous Island Model /* Migration occurs periodically */ eo. Periodic. Continue <Route> ox_mig_cont (MIG_FREQ); eo. Random. Select <Route> ox_mig_select_one; eo. Select. Number <Route> ox_mig_select (ox_mig_select_one, MIG_SIZE); /* Immigrants replace the worse individuals */ eo. Plus. Replacement <Route> ox_mig_replace; peo. Async. Island. Mig<Route> ox_mig (ox_mig_cont , // migration frequency ox_mig_select, // strategy of selection ox_mig_replace, // strategy of replacement topo, // topology – ring topology ox_pop, // source population of emigrants ox_pop); // destination population for imigrants

Design of several levels of parallelization/hybridization Processing of a single solution (Objective / Data partitioning) Independent walks, Multi-start model, Hybridization/Cooperation of metaheuristics Parallel evaluation of the neighborhood/population Scalability Heuristic Population / Neighborhood Solution

An example of decomposition in tasks (High level) Starting Fork « Active messages » Shared_rw<EOT> Cooperative island model Fork Shared_rw<eo. Pop <EOT>> Fork Local searches Parallel evaluators

An example of decomposition in tasks (Low level) n Internal functions embedded in E. As Partitioning the « data » for the steps of n n Selection Crossover Evaluation Replacement Population Shared_r<EOT> Shared_w<EOT : : Fitness> Evaluation Shared_rw<EOT> Recombination

MPICH 2 – Environment An initial test – list the machines running MPDs (MPI daemons) > mpdtrace lxo 9 lxo 10 … An initial test – sending a message in the ring of machines > mpdringtest 100 number of loops time for 100 loops = 0. 0880811… seconds An initial test – launching processes on the machines > mpiexec –n 8 hostname no. of processes process name

TSP Compiling and Launching setting the correct directory for compilation: > cd $HOME/tsp/ compiling the application > make launching four processes – the tsp. param specifies the config. Params (there will be one process on each of the machines assigned to you) > mpiexec –n 4. /tsp @tsp. param cleanning the directory (erasing the obj. and core files, etc) > make clean

Real-world applications developed with Paradis. EO n n n Data Mining in Near-Infrared Spectroscopy Radio Network design Genomics

Conclusion n Paradis. EO is a white-box OO framework n n Clear conceptual separation of solution methods and problems Maximum design and code reuse High flexibility (fine-grained EO objects) Paradis. EO provides a broad range of features n Evolutionary algorithms, local searches and natural hybridization mechanisms n n n Invariant part provided Various transparent and portable parallel/distributed models Experimental evaluation on academic problems and industrial applications n High reuse capabilities, efficiency of the parallel/distributed models