Incorporating weighting into risk assessment can this make

  • Slides: 26
Download presentation
Incorporating weighting into risk assessment: can this make an overall risk rating more meaningful?

Incorporating weighting into risk assessment: can this make an overall risk rating more meaningful? Lihong Zhu 1, John Holt 2 & Rob Black 3 1. 2. 3. Ministry of Agriculture and Forestry, New Zealand Lihong. Zhu@maf. govt. nz Natural Resources Institute, University of Greenwich, UK Department of Law, University of Greenwich, UK

Outlines • • • Introduction Why weighting? How weighting? What can weighting do for

Outlines • • • Introduction Why weighting? How weighting? What can weighting do for us? Discussions and Conclusions Incorporating weighting into PRA

Pest risk assessment • Essential stage in PRA: risk assessment • Assessing the likelihood

Pest risk assessment • Essential stage in PRA: risk assessment • Assessing the likelihood and consequences of pest introduction • Evaluating whether the risk is significant and therefore should be regulated • Criteria including entry, establishment, spread and Incorporating weighting into PRA

Pest risk assessment • Risk factors identified by subdividing each criterion into a series

Pest risk assessment • Risk factors identified by subdividing each criterion into a series of tangible risk factors, • A risk score attributed to each risk factor • Overall assessment based on a synthesis of the scores: – Simple everage – Weighted average – High & biased weighted everage (Zhu etc 2000, Holt 2005) Incorporating weighting into PRA

Weighting the risk element • But how to achieve a more logical & biologically

Weighting the risk element • But how to achieve a more logical & biologically meaningful overall risk rating? – incorporating weighting i. e. a weighting is given to each risk factor/element – a weighting is a value given to a risk factor according to how important it is perceived to be, or how significant it contribute to the overall risk rating: the larger the value, the more important the factor • Why give weighting? • What weighting systems are adopted? • How to derive weighting? Incorporating weighting into PRA

Table 1. Disadvantage of simple average: A hypothetical example illustrates the impact of simple

Table 1. Disadvantage of simple average: A hypothetical example illustrates the impact of simple average of risk factor scores. Incorporating weighting into PRA

Why give weighting? • The risk factors are not all equally important • Weighting

Why give weighting? • The risk factors are not all equally important • Weighting should be given to each risk factor to reflect its perceived importance • Those more important should properly contribute more to the final result (i. e. estimated overall risk) than those less important Incorporating weighting into PRA

Weighting systems • How many risk elements being taken into account – 0 -2

Weighting systems • How many risk elements being taken into account – 0 -2 or 0 -3 weighting systems – 0 -1 weighting system • Associated combining formulae needed Incorporating weighting into PRA

How to derive weightings 0 -1 weighting system: • Weighting is set at 0

How to derive weightings 0 -1 weighting system: • Weighting is set at 0 -1 scale, the higher the weighting, the more important the risk element • Summation of weightings for all risk elements equals 1 • Therefore the overall risk score is normalised and is independent from the number of risk elements Incorporating weighting into PRA

How to derive weightings • Ranking the risk elements • Converting ranks to numerical

How to derive weightings • Ranking the risk elements • Converting ranks to numerical weightings between 0 and 1 • Effectively, a linear rescaling of the rankings is calculated such that the sum = 1 Incorporating weighting into PRA

How to derive weightings • Derived from expert opinion: Delphi technique is a tool

How to derive weightings • Derived from expert opinion: Delphi technique is a tool to achieve a consented set of rankings from a pool of experts • Derived from multivariate data analysis: principle components analysis (PCA) Incorporating weighting into PRA

Weighting derived from principal components analysis (PCA) • PCA - a data transformation technique,

Weighting derived from principal components analysis (PCA) • PCA - a data transformation technique, which can reduce the number of variables whilst accounts for the most of total variance • PCA applied to PRA: – High variability in a risk factor means that it has the potential to discriminate the level of risk between cases. Variance is a measure of dispersion of risk scores around the mean of each risk factor (variable) – Some risk factors are correlated Incorporating weighting into PRA

PCA • PCA was performed for data of 264 species and subset data of

PCA • PCA was performed for data of 264 species and subset data of individual pest groups, 7 risk factors used • Latent vector and percentage variation of principal component (PC) were computed for data of all species (Table 3) • The first 5 PC axes account for almost 90% of the total variation, which means number of principal risk factor can be reduced • Economic impact and host range have the heaviest loadings in PC 1 that counts for 38. 32% variation, which suggests they are the two most important factors considered while assessing pest risks Incorporating weighting into PRA

Table 3. Loadings & percentage variation explained by each PC Incorporating weighting into PRA

Table 3. Loadings & percentage variation explained by each PC Incorporating weighting into PRA

PCA – weightings for various pest categories Weightings of individual risk factors for various

PCA – weightings for various pest categories Weightings of individual risk factors for various pest categories (Table 4) • Risk factors were sorted by weighting rank for all species • Weightings of risk factors were not significantly different because of the small number of variables • Weightings for some pest categories (nematode, bacteria, phytoplasma and mite) were considered unacceptable because of Incorporating into PRAdata insufficient caseweighting of PRA

Table 4. Weightings for various pest taxa Incorporating weighting into PRA

Table 4. Weightings for various pest taxa Incorporating weighting into PRA

Is it possible to have a generalised weighting pattern? Rank correlation of weightings for

Is it possible to have a generalised weighting pattern? Rank correlation of weightings for various pest categories examined by Spearman test, it revealed that – No any pair of pest categories are significantly correlated on weighting rank, which means it is difficult to find a general pattern of weighting that suits all the pest categories – Insecta and Fungi have the most similar weighting rank; correlations exists between Insecta and nematodes; Nematodes and Bacteria have the most contrary weighting rank Incorporating weighting into PRA

What can weighting do? Summary assessment scheme • Summary assessment – a quick qualitative

What can weighting do? Summary assessment scheme • Summary assessment – a quick qualitative risk assessment for immediate action or with limited information: low, medium, or high risk? Scheme comprised (Figure 1): – Key risk factors identified producing most of the risk – Weightings derived showing the relative importance of each factor – Correlations analysed showing the relationships of key factors Incorporating weighting into PRA

Figure 1. Correlations & PCA derived weightings in a summary risk assessment Incorporating weighting

Figure 1. Correlations & PCA derived weightings in a summary risk assessment Incorporating weighting into PRA

Does summary scheme work? • Summary risk rating derived from weighted averaging (7 factors

Does summary scheme work? • Summary risk rating derived from weighted averaging (7 factors involved), in comparison with averaged overall risk and weighed averaged risk from detailed risk assessment (45 risk factors involved) (Figure 2) • Does this summary scheme work (Table 5 & Figure 3)? Incorporating weighting into PRA

Incorporating weighting into PRA

Incorporating weighting into PRA

Does summary scheme work? Incorporating weighting into PRA

Does summary scheme work? Incorporating weighting into PRA

Figure 3. Does summary scheme work? Incorporating weighting into PRA

Figure 3. Does summary scheme work? Incorporating weighting into PRA

Discussions • Incorporating weighting into PRA – Historical data of pest introductions and invasions

Discussions • Incorporating weighting into PRA – Historical data of pest introductions and invasions – Previous PRA cases – Expert opinion • Previous data do not necessarily apply to new situations; however, these can provide at least a starting point for new pests. Incorporating weighting into PRA

Conclusions Weightings can be derived for individual risk elements/factors by applying statistical techniques or

Conclusions Weightings can be derived for individual risk elements/factors by applying statistical techniques or assigning by expert judgement • identify the more important risk elements • filter out the factors that are low contributors to the overall assessment while retaining the important ones • Without compromising the rigor we striving for in risk assessment • It is difficult to develop a generic weighting pattern for different pest categories • A quick summary scheme can be developed, it gives a Incorporating weighting into PRA quick and precautious idea of risk rating

Acknowledgement Ministry of Agriculture and Forestry Higher Education Funding Council for England through University

Acknowledgement Ministry of Agriculture and Forestry Higher Education Funding Council for England through University of Greenwich Incorporating weighting into PRA