Project Planning Tools for Developing Plans for Risk

Project Planning Tools for Developing Plans for Risk Assessment Randall Ryti, Neptune and Company, Inc

Terminology Matters Scientific Term Public Meaning Better Choice Uncertainty Ignorance Range Error Mistake, wrong, incorrect Difference from exact true number Bias Distortion, political motive Offset from an observation Values Ethics, monetary value Numbers, quantity Manipulation Illicit tampering Scientific data processing Anomaly Abnormal occurrence Change from long-term average From Somerville and Hassol, 2011, Communicating the science of climate change Physics Today 64: 48 -53 Project Planning 2

Overview • Systematic Planning expected by EPA – Triad Approach Systematic Planning – Data Quality Objectives – Statistical Approaches • Planning highlights – Conceptual models – Scale – Uncertainty Project Planning 3

Baseline Risk Assessment Components • • Problem Formulation (Study Design) Exposure Assessment Toxicity Assessment Risk Characterization • Risk Management Project Planning 4

Systematic Planning expected by EPA • Policy and Program Requirements for the Mandatory Agency-wide Quality System, Order 360. 1, CHG 1, 1998 • Process goes by various names depending on the organization – Data Quality Objectives (DQO) – Technical Project Planning (TPP) Project Planning 5

Triad Approach ITRC Guidance SCM-3 SCM-1 Project Planning 6

Triad Approach Components From SCM-1 Project Planning 7

What is Systematic Planning? • Group effort to balance cost vs. amount of data needed to make decision • Understand how the data will be used • Ensure you get what you pay for by defining project needs in detail • Knowing what was not delivered and why • Documented outcome Project Planning 8

Systematic Planning Elements • Data collection and analysis process • Collection and analysis requirements • Quality assurance and quality control to assess data collection activities • Process for evaluation and assessment of collected data against the intended use and quality performance criteria Project Planning 9

Triad Approach • What’s different about the Triad approach – Emphasizes the Conceptual Site Model • More information on the Triad approach is available on the www – http: //www. triadcentral. org Project Planning 10

EPA DQO Steps EPA Data Quality Objective (DQO) Process Seven Steps 1. State the problem 2. Identify the goals of the study 3. Identify information input 4. Define the boundaries of the study EPA QA/G-4 February 2006 http: //www. epa. gov/quality/qa_docs. html Project Planning 11

EPA DQO Steps New EPA Data Quality Objective (DQO) Process 5. Develop the analytic approach (statistics) Decision making = hypothesis testing ü Estimation or other analytic approach ü 6. Specify criteria Decision = probability limits ü Estimation = ü ü Performance criteria for new data ü Acceptance criteria for existing data 7. Develop the plan for obtaining data EPA QA/G-4 February 2006 http: //www. epa. gov/quality/qa_docs. html Project Planning 12

Risk Assessment Planning Considerations • Screening level assessments – What is the source and which media – Do analytical methods provide adequate sensitivity • Baseline assessments – What are site-specific exposure rates – How bioavailable are the contaminants Project Planning 13

You Don’t Need To Be A Statistician To Set Error Tolerances! Identify decision errors and evaluate consequences • Failure to find a problem that exists (underestimate result) • Consequences – no remedial action will be taken prior to release of property – potential adverse ecosystem effects – loss of credibility and cost to redo if error is subsequently discovered Project Planning • Incorrect determination that a problem exists (overestimate result) • Consequences – cost of further assessment or action taken unnecessarily – money and time spent could have gone to a “real” problem abatement 14

Decision Error Are concentrations greater than reference? Conclusion Based on Site Data True State Site > Ref Correct Site < Ref False Acceptance Project Planning Site < Ref False Rejection Correct 15

Visual Sampling Plan (VSP) Screen Shot http: //dqo. pnl. gov Project Planning 16

All models are wrong, but some are useful - George E. P. Box Project Planning 17

Conceptual Models • Simplified representations of site characteristics • Show link between contaminants and receptors • Provide foundation for risk assessment • Used to identify – Priorities – Data needs

Example Conceptual Model Project Planning 19

Simplified? … Useful? Project Planning 20

Conceptual Site Model Physical Site Model External Effects Site Description Operational History Existing Data Physical Site Model Project Planning 21

Conceptual Site Model Exposure Model Receptors and Land Use Exposure Media and Intake Routes Exposure Area(s) Exposure Model Project Planning 22

Integrating Space into Conceptual Site Models • Exposure to soil occurs over some area, depth, and time • Contaminants are almost never uniform over space • Land use and associated habitat varies across space Project Planning 23

In the beginning was the bell curve. . . Project Planning 24

Uncertainty analysis. . . • Examines a risk assessor’s confidence in the risk estimate, • As it is affected by variability and incertitude: – Variability (heterogeneity), – Incertitude (lack of knowledge, ignorance). • Uncertainty = Incertitude + Variability – Characterized with narrative (qualitative) and/or quantitative methods. • Quantitative characterization assumes mathematical model(s). Project Planning 25

Conclusions • Project planning tools offer a graded approach from simple to complex • Some of these tools have been highlighted in previous ITRC guidance documents and are easily referenced Project Planning 26