Scheduling a Scheduling Competition Providence Sept 2007 Commentary

  • Slides: 8
Download presentation
Scheduling a Scheduling Competition, Providence, Sept 2007 Commentary on Session A J. Christopher Beck

Scheduling a Scheduling Competition, Providence, Sept 2007 Commentary on Session A J. Christopher Beck Department of Mechanical & Industrial Engineering University of Toronto, Canada jcb@mie. utoronto. ca © J. Christopher Beck 2007

1+3 Papers l Ghersi et al. focuses on an operational question – how do

1+3 Papers l Ghersi et al. focuses on an operational question – how do we judge the results? – important, technical issue – valuable for whoever runs the competition l Other 3 papers focus on a more strategic question – what problem types should the competition address? 2 © J. Christopher Beck 2007

Some Operational Issues l Automated verification of results l Entries need to run on

Some Operational Issues l Automated verification of results l Entries need to run on the same platform l Not just source code vs. binary – License issues? 3 © J. Christopher Beck 2007

Integration vs. Focus l Rich problems (Le Pape; Guerri et al. ) vs. a

Integration vs. Focus l Rich problems (Le Pape; Guerri et al. ) vs. a single fundamental issue (Cicirello) l Things to consider: – industry impact – potential for research progress/breakthroughs • will we understand the results? – barriers to entry • easier to participate in Cicirello-style track – is this an OR vs. AI issue? 4 © J. Christopher Beck 2007

Multiple Tracks? l Perhaps of increasing difficulty l Things to consider: – spreading the

Multiple Tracks? l Perhaps of increasing difficulty l Things to consider: – spreading the competition too much • one entry per track is not very interesting – “granularity” of tracks – creating a challenge – barriers to entry 5 © J. Christopher Beck 2007

Robustness as a Criteria l Bias evaluation toward good performance on all instances (Le

Robustness as a Criteria l Bias evaluation toward good performance on all instances (Le Pape) l Best all-round single machine scheduler across different opt. funcs (Cicirello) l Things to consider: – “jack of all trades, master of none” • give up on being “the” best on a given problem • marketing – industry vs. research – another OR vs. AI issue? 6 © J. Christopher Beck 2007

Competition or “Challenge” l Competition – like SAT or Planning competition – multiple tracks

Competition or “Challenge” l Competition – like SAT or Planning competition – multiple tracks l Challenge – one problem type (e. g. , one of Guerri et al. ’s, one of Le Pape’s) – long work horizon (e. g. , 6 months – 1 year) – like the CP Modeling Challenge (2005) 7 © J. Christopher Beck 2007

Competition or “Challenge” l Things to consider: – marketing – (end user) industry interest

Competition or “Challenge” l Things to consider: – marketing – (end user) industry interest and commitment – barriers to entry – potential for lack of community interest – organizational overhead 8 © J. Christopher Beck 2007