FAKE DATA THE NEED FOR SAMPLING THEORY IN

FAKE DATA? THE NEED FOR SAMPLING THEORY IN ENVIRONMENTAL RESEARCH AND INVESTIGATIONS Roger Brewer, Ph. D, Senior Environmental Scientist Hawai´i Department of Health, Honolulu, Hawai´i, USA (roger. brewer@doh. hawaii. gov) July 16, 2019 1

Correlations Plant Uptake Vapor Intrusion Concentration Y Risk Assessment and “Highly Variable Data” ? Concentration X Indoor Air Quality Bacteria in Swimming Areas X X Concentration Trends Time 2

“Highly Variable” Discrete Soil Sample Data 4 -Acre Site (proposed hotel) Arsenic (mg/kg) 7. 2 4. 8 44 2. 5 162 507 520 217 4 164 42 208 184 211 243 45 30 48 46 36 36 1080 51 42 1680 925 385 254 994 285 363 737 624 419 467 1930 526 794 34 9. 2 101 1300 1420 789 191 202 209 55 66 323 1300 196 235 562 1280 282 431 194 Impetus for Multi Increment Sampling in Hawaii (2004) Solution? • Common: Collect samples until you run out of time and/or money and dump the data on a risk assessor; • Use the “maximum”? • Use Pro. UCL? • Toss the “outliers” (“maximums”)? 3

Toss the “Outliers”? (early red flag that something was wrong…) Gold nugget “outliers” “Max” Concentration = 100% All data not known to be in error should be considered valid… High concentrations are of particular concern for their potential health and environmental impact. USEPA 1989. Methods for Evaluating the Attainment of Cleanup Standards A common error has been to reject “outliers” that cannot be made to fit (a statistical) model. . . The tendency… has been to make the data fit a preconceived model instead of searching for a (more appropriate way to collect samples). Pitard 2009 4

Results of “Highly Variable” Data • • Repeated need to collect additional data with no clear end point; Manipulation of database to misleadingly reduce uncertainty; Uncertainty in potential risk; Uncertainty in adequacy of proposed, remedial actions; Failed confirmation samples following remediation; Abandoned Properties; Remediated sites later found to still be contaminated;

The Two Sides of Sampling Theory “Finite Element” Media “Infinite Element” Media (elements individually identified and randomly selected) (elements cannot be individually identified or randomly selected) room of people forest of trees cans of tuna bags of rice bags of soil bottles of water pile of flour pile of soil room of air pile of rice lake of water subslab vapors 6 6

Sampling Theory Basics Good Samples: Guidance on Obtaining Defensible Samples (2015) Good Test Portions: Guidance on Obtaining Defensible Test Portions (2018) Association of American Feed Control Officials: • Prepared by working group of state and federal regulators, university researchers, private consultants and industry sampling experts; • Basic concepts apply to all types of media. 7

Testing of “Finite Element” vs “Infinite Element” Media Testing of Finite Element Media (easiest media to sample): • Definition: Media composed of elements that can be individually identified and individually selected at random; • Common DQOs: Variability between elements, estimates of “minimum” or “maximum”, other quality control parameters; • Sample prepared by collecting and independently testing individual elements; • Only variable is the number of elements selected. Testing of Infinite Element Media (more complex to sample): • Definition: Media composed of elements that cannot be individually identified nor individually selected at random; • Common DQOs: Mean of targeted “Decision Unit” area & volume of targeted media; • Sample prepared by collecting & combining groups or “increments” of individual elements into single, bulk sample for processing & testing; • Variables include sample mass, # increments, collection method, etc. 8

Use of “Finite” vs “Infinite” Element Sampling Methods for Air, Subslab Vapors and Soil/Sediment air vapors soil sediment 9

Testing Indoor Air X Risk Question: Does the concentration of “X” in the air of this room during time period “Y” exceed screening level or risk “Z”? 10

Indoor Air Decision Unit Designation (for example only) Indoor Space Volume=244 m 3 slab Indoor Air Exchange Rate (L/minute) Indoor Volume x DU Volume = IAER x Designated Time Period DU Volume = 244, 000 L x 0. 35/hr x 24 hrs DU Volume = 2, 050, 000 L Data Needs: What is the mean concentration of “X” in indoor air over a specified, 24 hour period? Decision Unit: Volume of air circulating within structure during the specified time period.

Collection of Indoor Air Samples (Infinite Element Media Sampling Methods) Indoor Air DU Volume (2, 050, 000 liters) 16 m • • • Ideal: Entire DU volume collected and sent for testing; Reality: Representative sample continuously collected over 24 hours (e. g. , using a Summa canister or passive sampler); Assumes circulating air passing by collection point is representative of indoor air within the targeted area of the structure; Assumes targeted day is representative of long-term mean; Collect concurrent, replicate samples from other areas of structure and on different days to test assumptions and sampling precision.

Testing Subslab Vapors X SSAF Risk Question: Does the concentration of “X” in vapors intruding the building during time period “Y” exceed screening level or risk “Z”? Subslab Vapor SL = Indoor Air SL Attenuation Factor 13

Subslab Vapor Decision Unit Designation (for example only) Intruding Vapors X slab SSAF Indoor Air Exchange Rate (L/minute) Vapor Entry Rate DU Volume = x Designated Time Period DU Volume = 4. 5 L/min x 60 min/hr x 24 hrs 6, 480 L/day subslab vapors Data Needs: What is the mean concentration of “X” in intruding vapors over a 24 -hour period? Decision Unit: Volume of vapors intruding structure during the specified time period. 14

Collection of Subslab Vapor Samples (Finite vs Infinite Element Media Sampling Methods) • Example area covered subslab vapor sample; • Assumes cylindrical draw area from first 0. 5 m of subslab fill/soil and 20% air-filled porosity. Hypothetical Vapor Entry Point X m 5. 4 1 L Summa . 6 L Summa 6, 480 L Daily VER For example only • Ideal: Entire DU volume of subslab vapors collected and sent for testing; • Reality: Representative sample(s) must be collected; • Small-volume, “discreet” subslab vapor samples only representative of intruding vapors if subslab vapor plume is very uniform.

Subslab Vapor Plumes are Never “Uniform” (Finite Element Media Sampling Methods) Subslab AF = Conc. Indoor Air Conc. Soil Vapor X X (Luo 2009) X • VOC concentrations in vapor plumes can vary by orders of magnitude over short distances; • Small-volume, “discrete” subslab vapor samples not reliable for calculation of IA: Subslab attenuation factor; • Calculated subslab AF will vary depending on where the sample is taken; • Vapor entry point normally unknown; • Even if known, 1 -6 L “discrete” samples too small to represent intruding vapors; • Error in estimation of true subslab AF for a building not quantifiable. Cal. EPA 2011 (among others): “The default attenuation factors assume [that] …the subsurface is reasonably homogeneous (uniform). ”

Implications for USEPA “Emperical” VI Database Frequency “Fake” 95% UCL =0. 03 • SSAFs based on collection of a single, 1 -6 L vapor sample beneath each building slab; • Database not scientifically defensible for development of generic attenuation factors; • Can’t be “fixed” by statistical analysis; • Useful but failed scientific study, • It’s not “the model” (basic physics); it’s the sample data! Range of random noise/error in subslab vapor sample data Brewer, R. , Nagashima, J. , Rigby, M. , Schmidt, M. and O'Neill, H. (2014), Estimation of Generic Subslab Attenuation Factors for Vapor Intrusion Investigations. Groundwater Monitoring & Remediation, 34: 79– 92. http: //onlinelibrary. wiley. com/doi/10. 1111/gwmr. 12086/full

How to Collect Representative Subslab Vapor samples (Infinite Element Media Sampling Methods) • • Suspect or hypothetical vapor entry point(s) designated in slab; Targeted DU volume of vapors purged from point; Single, continuous sample collected in Summa from purge line; Data represents VOC concentration in vapors hypothetically intruding through that point over 24 hours (five purges = one week); • Exact source area of vapors under slab not important for initial assessment of vapor intrusion risk (building doesn’t know or care). Mc. Alary, T. A. , Nicholson, P. J. , Yik, L. K. , Bertrand, D. M. and G. Thrupp. 2010. High Purge Volume Sampling - A New Paradigm for Subslab Soil Gas Monitoring: Ground Water Monitoring and Remediation. 30 (2): pp 73– 85. 18

Collection of “Large Volume Purge (LVP)” Subslab Vapor Samples (PCE, former dry cleaner) Flow Meters Shop Vac Pump Summa Canister Connected to Purge Stream Vacuum Gauge Isopropyl Alcohol Rags (leak detection) LVP Sampling Point • Similar to an SVE pilot test; • Continuous, 6 L Summa sample collected from line during purge; • Five consecutive, 6, 480 L purges (five days of vapor entry). 19

Collection of LVP Subslab Vapor Samples (HDOH 2014 field study) X LVP Point LVP PCE Results: Sample #1: 17, 000 µg/m 3 Sample #2: 36, 000 µg/m 3 Sample #3: 50, 000 µg/m 3 Sample #4: 51, 000 µg/m 3 Sample #5: 54, 000 µg/m 3 • LVP collection point placed in center of slab (assumed worst-case subslab vapor plume scenario; • Commercial/Industrial subslab vapor screening level = 8, 000 µg/m 3; • Note increasing concentration with increasing purge volume (capturing “outlier” hot spots); • Detailed passive sampler study of PCE vapor plume and DU-MIS study of subslab soil also carried out; • Single LVP sample concentration often exceeds mean of surrounding, discrete sample data.

LVP References HDOH, 2017. Field Study of High-Density Passive Sampler and Large. Volume Purge Methods to Characterize Subslab Vapor Plumes: Hazard Evaluation and Emergency Response, Hawaii Department of Health, July 2017. Report and Recorded Webinar: Hawaii DOH Webinar Webpage, http: //eha-web. doh. hawaii. gov/eha-cma/Leaders/HEER/Webinar HDOH, 2016, Technical Guidance Manual: Hawai‘i Department of Health, Office of Hazard Evaluation and Emergency Response (Section 7: Indoor and Soil Vapor). http: //www. hawaiidoh. org/ 21

Soil and Sediment -The Challenge of Collecting a “Representative Sample”- X 1 kilogram (One million 1 mm particles) X X X 1 milligram (One single 1 mm particle) Risk Question: Does the concentration of “X” in area/volume of soil (or sediment) “Y” exceed the screening level or risk “Z”? 22

Step 1: Thorough “Systematic Planning” (HDOH Technical Guidance Manual Section 3) 1. Determine the COPCs and primary environmental concerns. 2. Specify the Site Investigation Objectives (aka “DQOs”). 3. Designate the data resolution (“Decision Unit”) needed to answer the investigation objectives. 4. Collect a representative sample(s) from each area. Every Spot? Each Playground? The Entire Park? • Ideal: Site investigation objectives and DUs discussed with risk assessors, remediation experts, etc. , prior to collection of samples in field; • Reality: Risk assessors and remediation specialists get stuck with the sample data given to them with no time or money to collect additional data. 23

It’s All About the Decision Units (DUs)… (HDOH Technical Guidance Manual Section 3, 4) • DU = An area and volume of soil you would send to the laboratory as a single sample for testing, if you could; • DUs should always be designated for testing, regardless of sampling method used. Spill Area DUs Exposure Area DUs Boundary DUs 24

Collection of Soil and Sediment Samples (Finite vs Infinite Element Media Sampling Methods) 1985 “Future changes in EPA policy may invalidate some of the discussions in this (guidance). ” • Discrete (“finite element”) soil sampling approaches developed in mid 1980 s; • Not used in any other sampling intensive, similar industry (mining, agriculture); • Risk assessment guidance and risk-based screening levels being concurrently developed; • Early sampling guidance documents confused applicability of screening levels to any testable mass of soil versus mean of targeted, exposure area. Brewer et al. , 2017, A critical review of discrete soil sample reliability: Journal of Soil and Sediment Contamination. What does a “discrete” sample represent? 25

Key Assumptions for Reliability of Discrete Soil and Sediment Sample Data “The… level is assumed to be uniform within (a contaminated area) and zero outside it. ” USEPA 1985: Verification of PCB Spill Cleanup “When there is little distance between points it is expected that there will be little variability between points. ” USEPA 1989: Methods for Evaluating the Attainment of Cleanup Standards If true: • A few samples collected from single points within a targeted property adequate to identify extent of contaminated areas and assess risk; • Data for individual points can be compared to risk-based screening levels. 26

Ever wonder… “What if I moved my sample point over a few feet? “What if the lab tested a different subsample of soil? ” 4 oz Jar: 100 grams PCBs, Pesticides, TPH, PAHs: 10 -30 grams VOCs: 5 grams Metals: 0. 5 -1. 0 grams

Field Study of Discrete Soil Sample Variability • • • (HDOH 2015) Three known-contaminated sites studied; 24 grid points designated at each site; 5 co-located discrete samples tested; One sample tested 10 time; Multi Increment samples (triplicates) collected from same areas. Study Site A (arsenic in wastewater) Study Site B (lead in incinerator ash) Study Site C (PCBs transformer oil) 13, 500 ft 2 area 1, 500 ft 2 area 6, 000 ft 2 area

Field Study Results Roger Brewer, John Peard & Marvin Heskett (2017) A critical review of discrete soil sample reliability: • Part 1 - Field study results • Part 2 - Implications • Supplement – Review of past USEPA guidance Journal of Soil and Sediment Contamination. Part 1: http: //dx. doi. org/10. 1080/15320383. 2017. 1244171 Part 2: http: //dx. doi. org/10. 1080/15320383. 2017. 1244172 2015 Field report and recorded webinars posted to HEER webpage http: //eha-web. doh. hawaii. gov/eha-cma/Leaders/HEER/Webinar 29

Field Study Results: Data for a discrete sample are random within a (largely) *unknown range Total Discrete Sample Variability *Range of variability similar within a single sample as between co-located samples.

“Compositional Heterogeneity” as a Source of Discrete Sample Data Variability 1 mm 100, 000 s ppm 2 mm Concentration #3 (PCB matrix) 1, 000 s ppm Concentration #2 (single nugget) 1 cm 1 -10 ppm Concentration #1 (whole sample) • Small, concentrated “nuggets” of contamination might or might not be included in subsample tested; • Concentration varies with volume/mass of soil tested (e. g. , 1 vs 30 g subsamples); • “Maximum” concentration always either 0 % (absent) or 100% (present); • Risk assessment objective is the mean or “true” concentration of the contaminant for the designated DU area/volume/mass of soil as a whole.

“Distributional Heterogeneity” as a Source of Discrete Sample Data Variability Discrete Samples (actual size) Jackson Pollock splatter painting Spilled milk following low areas What Contamination in Soil Would Look Like if You Could Actually See It

Primary Cause of “Failed” Confirmation Samples or Discovery of “New” Contamination After Cleanup • Lab data not reliably representative of sample; • Sample not reliably represented of area where it was collected. Artificial Hot & Cold Spots • “Variable” data leads to repeated investigations with no clear endpoint; • Problem can’t be fixed by collecting more discrete samples; • Problem can’t be fixed by statistical evaluation of discrete sample data.

Multiple Early Warnings About Discrete Sample Data Reliability… (refer to Part 2 and Supplement of Brewer et al. 2017) The so-called grab sample is not really a sample but a specimen of the material that may or may not be representative of the sampling unit (DU). Great care must be exercised when interpreting the meaning of these samples. USEPA (1992) Preparation of Soil Sampling Protocols 34

Risk Assessors (almost) Got it Right… Front Yard Exposure Area DU: Area: 400 ft 2 Depth: 2” Volume: 2. 5 cyds (2, 500 kg) Exposure Area DU • Concentration of contaminant at individual points within targeted area can be highly variable; • Assessment of risk based on mean (true) concentration of contaminant for specified area & volume of soil. 35

Ideal: 1) Excavate and send the entire volume of exposure area soil to lab for testing as a single “sample”; 2) Use data to assess risk. Exposure Area DU • • Entire mass of soil extracted by laboratory; Single concentration of contaminant reported; Represents the “true” mean; Not practical in most cases… 36

Alternative: 1) Collect 10+ discrete samples; 3) Test each sample individually; 4) Use statistical analysis to address variability and estimate mean. Exposure Area DU Problems: • 10+ samples often not available for single area (“use maximum”); • Laboratory analytical costs often limits number of samples collected; • Statistical analysis of data only evaluates the precision of the statistical test employed to estimate a mean for the data set provided; • Representativeness of single set of sample data unknown; 37 • False sense of data precision.

What do 95% UCLs Represent? -Comparison of 20 Sets of Replicate Discrete Sample Data. Range 95% UCL (mg/kg) Range RSD *Site A (arsenic) 403 to 776 34% to 67% *Site B (lead) 201 to 439 20% to 86% Site C (PCBs) 9. 4 to >1, 000 124% to 315% Study Site *XRF data (not directly comparable to MIS 6010 B data) • • • Pro. UCL used to calculate 95% UCLs for 20, random sets of discrete samples within each study area (10 samples per set); 95% UCL for an individual set of data is random within an unknown range; Field sampling error not directly considered in traditional risk assessments. 38

Better Approach: 1) Combine all “increments” into a single, bulk sample; 2) Report ONE concentration. (Infinite Element Media Sampling Methods) Exposure Area DU • Single, large sample prepared by collection and combining multiple “increments” of soil within targeted DU (“Multi Increment” sample); • Sample carefully processed and subsampled for testing; • Estimated mean concentration for DU area directly reported; • How many increments should be collected per sample? • How should the increments be collected (core-shaped)? 39 • What is the optimal, total mass of soil to be collected?

Gy’s Sampling Theory for Infinite Element Media (replaces finite element statistical sampling methods) Used in mining and agriculture industries since 1950 s. Considerations for Representative Samples: • FE Sample Mass (Compositional Heterogeneity) • Sample Increments (Distributional Heterogeneity); • Increment Collection (core-shaped); • Sample Processing & Subsample Collection (similar); • Subsample Analysis (least error); • Most Soil/Sediment Investigations: – Minimum sample mass = 1 -2 kg; – 50 -increments per sample; – Allows ppb (? ) to ppm data resolution. Variables: • Particle size/shape/density • Desired concentration resolution, etc. • Acceptable total error (FE) 40

Why Miners Figured it Out First -True Data Verification and Validation. Crushed Ore Sampled to Estimate Mean Commodity Concentration and Mass Gold Ore Commodity Extracted from Crushed Ore and True Mass Determined • Progressive isolation of sources of error (1800 s); • Development of “Theory of Sampling” (1950 s); • In French until 1990 s. Result of Poor Sample Collection Methods: • Underestimate Mean = Giving away your commodity; • Overestimating Mean = Cheating your customer; • Poor sampling methods most commonly underestimate true mean (missing small, high-concentration areas); • Either way – You go out of business…

Sampling Theory Training Classes for Finite and Infinite Element Media Chuck Ramsey (www. envirostat. org) (former USEPA; four-day, detailed introduction to sampling theory for food, soil, water, etc. ) Francis Pitard (www. fpscsampling. com) (advanced statistical sampling concepts with a focus on optimization of sampling protocols for mining) Best Training – Start practicing in the field! 42

Hawaii Technical Guidance Manual “Risk-Based Site Characterization” Section 3: Systematic Planning & DU Designation Section 4: Sampling Theory, Multi Increment Sample Collection Methods, Laboratory Processing (+use & misuse of discrete samples) Section 5: Field Implementations (clearing, DU ID, sampling tools, etc. ) HDOH, 2016, Technical Guidance Manual: Hawai‘i Department of Health, Office of Hazard Evaluation and Emergency Response, Sections 3 -5, http: //www. hawaiidoh. org/ Six-Part DU-MIS Training Webinar Series (You. Tube): http: //eha-web. doh. hawaii. gov/eha-cma/Leaders/HEER/technical-guidance-and-fact-sheets 43

But what about “acute” risk? 40 mg Pb Risk Question: Does the concentration of “X” in any (potential) discrete soil sample collected from the yard exceed “Y” mg/kg? (hypothetical acute toxicity) Sampling Considerations: Total Soil Mass: 2, 500 kg Discrete Sample Mass: 100 g Lab Aliquot Mass: 1 g (metals) What’s the DU? : • 25, 000 possible 100 g discrete samples (4 oz jar); • 2, 500, 000 potential 1 gram “samples” (laboratory subsample for metals); • 12, 500, 000 potential 200 mg “samples” (default child soil ingestion rate); • 250, 000 potential 10 gram “samples” (pica child ingestion rate); • Unanswerable question in terms of feasibility and cost; • Testing for acute risk with a reasonable degree of certainty is not practical; • Acute toxicity factors also not available for most chemicals; 44 • If a real concern then remove or cap suspect soil.

Are Discrete Sample Data Ever Useful? (Yes!) • • Nine-acre site proposed for residential development; Field XRF and grid of discrete samples used to identify spill area; Large-scale patterns likely real (Zones A & C); Initial removal of Areas A and B (targeted removal of possibly “fake” “hot spots” in Zone B not reliably); • Confirm with DU-MIS data. Soil contaminated by spillage of water-based arsenic herbicides DU-7 DU-4 DU-5 DU-6 DU-1 DU-2 DU-3 45 45

Are Discrete Sample Data Ever Useful? (No!) 9 Acres DU-MIS Confirmation Sample (entire site contaminated with PCBS) PCB “Hot Spots” 9 Acres <1. 1 mg/kg “Hot Spot” Removal >1. 1 to <10 mg/kg >10 mg/kg 46

ITRC “Incremental Sampling Methodology” Guidance (2012, update in progress) • Basic introduction to sampling theory; • HDOH staff and Hawaii consultants assisted in preparation; • Very few of 100+ original team members had training in Sampling Theory or experience in collection of “ISM” samples in the field; • Incomplete discussion of sampling theory & sample collection; • Lacks critical review of discrete sample data reliability; • Hypothetical evaluation of ISM data accuracy/precision misleading and inaccurate (attempts to use “finite element” statistical methods to evaluate sampling theory for infinite element media); • Currently being updated – Should be improved but still limited experience among most team members. 47

Questions? The future is already here – It’s just unevenly distributed. William Gibson (Neuromancer) 48