MultiCriteria Decision Analysis Tools Developed by Dr Martyn
Multi-Criteria Decision Analysis Tools Developed by Dr. Martyn Jones, The University of Manchester, January 2016.
About this module Content and learning objectives: This module is an introduction to MCDA. It gives a brief overview of MCDA and two frequently used methods.
What is multi-criteria decision analysis Multi-criteria decision analysis (MCDA) is a formal, structured and transparent decision making methodology. Its ism is to assist groups or individual decision makers to explore their decisions in the case of complex situations with multiple criteria. MCDA does not provide the ‘right’ answer. MCDA does not provide an objective analysis. MCDA does not relieve decision makers of the responsibility of making difficult judgments. MCDA assists the decision maker in confidently reaching a decision by: • enabling decision makers to gain a better understanding of the problem faced; • organising and synthesising the entire range of information; • integrating objective measurements with value judgements; • making explicit and managing the decision maker’s subjectivity; and • ensuring that all criteria and decision factors have been taken properly into account.
MCDA vs unstructured decision making process All decisions are subjective. All decision makers are biased. Avoiding bias and subjectivity may be feasible only if a problem is sufficiently simplified, such that some numeric objective criteria could be designed: We may simplify evaluation of a chemical process to an individual criteria, such as RISK, where risk is a product of HAZARD and EXPOSURE. If HAZARD is then quantified numerically by using e. g. , a toxicity scale, and EXPOSURE is known from materials flows, than RISK could be evaluated numerically and the process option with the minimum RISK value would be considered the best option. But the process with minimum RISK value is not automatically the best one, if we consider more criteria: energy, cost, resource efficiency, etc. Which of these criteria are more important?
MCDA vs unstructured decision making process Through the use of MCDA a larger number, of perhaps individually less important, indicators do not get ignored in the final decision during a simplification that relies too heavily on a small number of key criteria. Unstructured decision making often fails to make use of or consider the uncertainty surrounding different criteria. MCDA can be used in conjunction with Monte Carlo simulation to take account of modelled uncertainty of the criteria values, uncertainty of the subjective criteria preferences and provide a known level of certainty in the proposed decision, unachievable in unstructured decision making.
MCDA methodology MCDA is an umbrella term for a range of tools and methodologies. The level of complexity, interaction with the decision maker and level of detail utilised in the decision making process can vary substantially. In general the decision maker follows the same process: 1. Identify multiple criteria on which to base their decision; 2. Identify multiple alternative solutions to their decision; 3. Provide (subjective) ranking or weighting of criteria; and 4. Provide values, rankings or weighting of alternatives for each criteria.
MCDA in sustainability assessment Here we show a number of published studies where MCDA tools have been incorporated into sustainability assessment: Azapagic, A. and S. Perdan (2005 a) An integrated sustainability decision-support framework Part II: Problem analysis. International Journal of Sustainable Development & World Ecology, 12(2), 112 -131. Azapagic, A. and S. Perdan (2005 b) An integrated sustainability decision-support framework Part I: Problem structuring. International Journal of Sustainable Development & World Ecology, 12(2), 98 -111. Benetto, E. and C. Dujet (2003) Uncertainty analysis and MCDA: a case study from the life cycle assessment (LCA) practice. 57 th Meeting of the European working group on multicriteria decision aiding, Viterbo, 27– 29 March. Cinelli, M. , S. Coles and K. Kirwan (2014) Analysis of the potentials of multi-criteria decision analysis methods to conduct sustainability assessment. Ecological Indicators, 46, 138 -148.
MCDA methodology The Analytical Hierarchy Process (AHP) AHP was developed in the late 1970 s. Today it is the most widely used MCDA method. AHP generates all criteria weighting and alternative preference within each criteria by eliciting these values from the decision maker through a series of pairwise comparisons, as opposed to utilising numerical values directly. Thus, a complex decision is reduced to a series of simpler ones, between pairs of alternative values within criteria or between pairs of criteria. The decision maker’s preference is always explicit. However, the decision maker may be asked to make very many small decisions. Hence, it becomes important to generate an optimised hierarchy of criteria and alternatives, to reduce the number of pairwise decisions.
AHP Step 1: Construct the problem hierarchy Model, usually visually, the problem decision identifying relationships between criteria and alternatives. Step 2: Pairwise comparison of criteria Undertake pairwise comparison between criteria, identifying decision maker preference for criteria on which alternatives are evaluated. Step 3: Pairwise comparison of alternatives within each criterion Undertake pairwise comparison between alternatives based on their performance within each criterion. Step 4: Compute the vector of criteria weights From a matrix of pairwise comparison results AHP utilises a variety of matrix transformations to calculate criteria weight vectors representing normalised criteria weightings.
AHP Step 5: Compute the matrix of alternative scores From the results of the pairwise comparisons on alternatives within each criterion a nxm (where n is the number of criteria and m is the number of alternatives) matrix is constructed representing the normalised performance (score) of each alternative for each criteria. Step 6: Ranking the alternatives Utilising the vectors of criteria weights and the matrix of alternative scores a global score and hence ranking for each alternative is calculated using: where: a is the alternative, c is the criteria, g is the global score of the alternative, w is the criteria weight and s is the alternative score. A function of the ranking equation, aggregating across each criteria means that trade-offs between criteria in fundamental to the final ranking.
AHP Problem Hierarchy The problem hierarchy provides a structured, usually visual, means of modelling the decision being processed. As the first step in the analytical hierarchy process the creation of a hierarchy that models the decision problem enables decision makers to increase their understanding of the problem, its context and, in the case of group decision making, see alternative approaches to the problem across different stakeholders. The AHP problem hierarchy consists of a goal (the decision), a number of alternatives for reaching that goal, and a number of criteria on which the alternatives can be judged that relate to the goal. Here as an example a simple AHP hierarchy was generated in DECERNS tool.
AHP For most realistic analyses criteria are multi-tiered. Criteria weightings are calculated as pairwise comparisons. Consider a problem hierarchy with 12 criteria on which the three alternatives are to be judged. For pairwise comparison the first criteria will be compared against the remaining 11, the second criteria against the remaining 10, the third criteria against the remaining 9 etc. In total 66 (11+10+9+8+7+6+5+4+3+2+1) pairwise comparisons will need to be undertaken in order to determine the weighting of the 12 criteria.
AHP Pairwise Comparisons Within AHP pairwise comparison is the process of comparing entities in pairs so as to judge which is preferred and by how much. Comparisons are undertaken to determine criteria weighting and also assess the value or score of different alternatives within each criteria. A 9 -point scale to elicit the scale of preference from a decision maker: the more preferable entity within the pair scores 1 when it is showing no preference 3 when it is showing moderate preference 5 when it is showing strong preference 7 when it is showing very strong preference 9 when it is showing extreme preference. The less preferable entity within the pair scores the inverse, for example the less preferable entity where the more preferable entity shows very strong preference would score 1/7.
AHP Groups of pairwise comparisons are undertaken between every alternative value within a single criteria, and every criteria within the goal (or for multi-tier hierarchies within their parent criteria). For each group a matrix is completed with the results of the pairwise comparison, such as that shown in the table below, following the example from figure on Slide 12. The results of the matrix would provide the normalised criteria weights for criteria A 1 to A 4. Similar matrices would be completed for criteria B 1 to B 4, for C 1 to C 4 and also one comparing criteria A, B and C. Finally, pairwise comparisons would be undertaken to fill matrices for each criteria comparing the performance of each alternative within that criteria.
AHP Consistency across pairwise comparisons The consistency of the decision maker across a number of pairwise comparisons is a significant complexity. Consider the very simple comparison of three criteria: A, B and C. If the decision maker judges A to be more preferable than B, and A to be less preferable than C then the decision maker must not judge B to be more preferable than C. In a group that contains a large number of pairwise comparisons or where the difference is between moderate and very strong preference it can be seen that lack of consistency is a largely inevitable consequence of complex decision processes within AHP. The AHP method attempts to address the issue of consistency by implementing a consistency index that is a function of opposing comparisons. Above a threshold a lack of consistency is highlighted and no analysis results are presented. An unfortunate consequence is that decision makers begin to fulfil pairwise comparisons not on their actual judgements but rather in order to maintain acceptable consistency. An effective approach to limit the issue of consistency is to utilise a multi-tier hierarchy thereby reducing the number of pairwise comparisons undertaken within each group.
AHP Rank reversal If the inclusion or exclusion of a non-outperforming alternative, or duplicate alternative alters the ranking of the remaining alternatives a rank reversal occurs. AHP method and other MCDA methods are susceptible to rank reversal and experienced users must be aware of this.
AHP Recording the decision making process and decision makers subjectivity It is useful to have a record of decision making process. This gives some idea of how the decision was reached. The problem hierarchy gives insight into how the decision was structured. Most AHP tools allow to view the pairwise comparison matrices showing the preference values applied to each pair. However, this does not make explicit the subjectivity inherent in the judgements made by the decision maker; the reasoning and understanding behind those simple judgements is lost. For complex problems where a large number of pairwise comparisons have been undertaken it is unlikely that enough subjective detail could be recorded at the time of making the decision to suggest the information available could be considered transparent or suitable for any form of audit. Similarly it is unlikely that returning to this information in the future would enable the reasoning behind any mistakes to be observed.
PROMETHEE Preference Ranking Organization Method for Enrichment Evaluations This is a family of methods developed since 1980 s. PROMETHEE is classified as an outranking method allowing for a finite number of alternatives to be ranked based on a finite number of criteria, which are often conflicting. The PROMETHEE family includes a number of methods (PROMETHEE I, III, IV, V and VI) although PROMETHEE I for partial ranking of alternatives and particularly PROMETHEE II for complete ranking of alternatives are the most commonly used. Arguably, PROMETHEE II is the most relevant to decision making in process development and innovation. PROMETHEE II has been developed in order to provide a complete ranking of a finite set of alternatives from the best to the worst. The ranking is calculated using a pair-wise comparison of alternatives for each criteria utilising preference functions which are then aggregated using criteria weighting to provide a net outranking flow and hence a complete ranking of alternatives.
PROMETHEE
PROMETHEE Each criterion used within a PROMETHEE model to help rank alternatives is assigned a preference function by the decision maker. The preference function translates the difference (either positive or negative) in the value of a criterion between two alternatives in a pairwise comparison into a preference degree ranging from zero to one. Typically, 6 preference functions are used: Type 1: Usual criterion Indifference only exists between alternatives a and b when they are equal and in this case they will be assigned a preference degree of 0. However, the smallest difference in value represents a strict preference on behalf of the decision maker and is assigned a preference degree of 1. Type 2: Quasi-criterion In this case alternatives a and b are seen to be indifferent within a range defined by the decision maker and will be assigned a preference degree of 0. Beyond that area of indifference the preference becomes strict, with a preference degree of 1 being assigned.
PROMETHEE Type 3: Criterion with linear preference Extending the usual criterion, in this case the decision maker’s preference increases progressively for progressively larger deviations between alternatives a and b. Where a equals b the alternatives are indifferent and assigned a preference degree of 0. From that point the intensity of preference increases linearly to a threshold value defined by the decision maker at which point and beyond which the preference is strict and assigned a preference degree of 1. Type 4: Level criterion An extension to the quasi-criterion, in this case alternatives a and b are seen to be indifferent within a range defined by the decision maker and will be assigned a preference degree of 0. Beyond that a second range defined by the decision maker provides a weak preference where a preference degree of 0. 5 will be assigned. Beyond that range a strict preference with a preference degree of 1 is given.
PROMETHEE Type 5: Criterion with linear preference and indifference area A combination of types 2 and 3, in this case alternatives a and b are seen to be indifferent within a range defined by the decision maker and will be assigned a preference degree of 0. Beyond this range the decision maker’s preference increases progressively for progressively larger deviations between alternatives a and b. The intensity of preference increases linearly to a threshold value defined by the decision maker at which point and beyond which the preference is strict and assigned a preference degree of 1. Type 6: Gaussian criteria Similar to type 3, if a particular criterion is of the Gaussian type, the preference of the decision maker still grows with increasing deviation between alternatives a and b but the relationship is not linear. Through the decision maker providing a value of σ, representing the distance between the origin and point of inflection, the relationship between difference in alternatives a and b and the preference degree is given. The preference degree will vary from 0 where a equals b to approaching 1 where the difference is very large.
PROMETHEE Preference functions
PROMETHEE Criteria Weighting As with most MCDA methods it is possible, indeed preferred, to define the relative importance of each criteria within the decision making process. This is done using criteria weighting. Within the PROMETHEE model normalised weights, aggregating to 1 are used. There are many techniques to elicit the weightings from the decision maker with the choice often being one of the software being used or personal preference of the decision maker. As an example, the MCDA tool DECERNS provides 4 common methods of eliciting criteria weights within its PROMETHEE model: 1. Direct weighting The decision maker directly provides numerical weights for each criterion representing its perceived importance in the decision process. These weights will be normalised before being used in the PROMETHEE model. 2. Ranking The decision maker is asked to rank in order of preference the criteria. The rankings are converted to equally spaced numerical values, normalised to aggregate to 1.
PROMETHEE Criteria Weighting 3. Rating The decision maker attributes a score of 100 to the criteria perceived as most important. Subsequently scores of less than 100 are applied to each of the remaining criteria. The scores are normalised to aggregate to 1. 4. Pairwise comparison Using the same process and scale as AHP, the decision maker considers each possible pair of criteria and states a preference on a 9 point scale from equal preference through moderate, strong, very strong and extreme preference. Once all pairwise comparisons are complete scores for each criterion are aggregated and normalised.
PROMETHEE Multi-level criteria, multi-level weighting In this example the higher level criteria have been defined for economic, environmental and social impacts with the relevant criteria flowing into each. Criteria weighting occurs within the economic higher level criteria for criteria X and Y and positive, negative and net flows for the alternatives are calculated at that level. At the task level weighting of the three higher levels occurs and is applied to the flows calculated for the economic, environmental and social criteria to provide flows, and hence a ranking, for the overall task.
PROMETHEE Barriers to wide adoption In principle, PROMETHEE is a relatively simple ranking method in its idea and application. Key methodological difficulty is the definition of preference functions suitable for each criterion. This may require a specific guidance following statistical analysis of decision outcomes depending on preference functions used, as well as a guidance for preference functions suitable for specific situations. PROMETHEE tools Decision Lab DECERNS D-Sight Smart Picker Pro Visual PROMETHEE
PROMETHEE Recording the decision making process and decision makers subjectivity Due to the relative simplicity of the PROMETHEE approach it is possible to provide a simple and yet complete overview of the decision making process and decision maker’s subjective input. This information provides transparency and audit of the decision process and allows the decision to be recreated at a later date. The following data is readily available from all PROMETHEE software tools: Criteria/Alternative value matrix Criteria hierarchy map (if using multi-tier weighting) Preference functions for each criteria Criteria Weighting (for each tier if using multi-tier weighting)
Literature and further reading • Belton, V. and Stewart T. (2002) Multiple Criteria Decision Analysis: An Integrated Approach. New York, Kluwer Academic. • Azapagic, A. and S. Perdan (2005 a) An integrated sustainability decision-support framework Part II: Problem analysis. International Journal of Sustainable Development & World Ecology, 12(2), 112 -131 • Azapagic, A. and S. Perdan (2005 b) An integrated sustainability decision-support framework Part I: Problem structuring. International Journal of Sustainable Development & World Ecology, 12(2), 98 -111 • Benetto, E. and C. Dujet (2003) Uncertainty analysis and MCDA: a case study from the life cycle assessment (LCA) practice. 57 th Meeting of the European working group on multicriteria decision aiding, Viterbo, 27– 29 March. • Cinelli, M. , S. Coles and K. Kirwan (2014) Analysis of the potentials of multi-criteria decision analysis methods to conduct sustainability assessment. Ecological Indicators, 46, 138 -148.
- Slides: 29