Eye of the Beholder UserOriented Data Quality Chris
Eye of the Beholder: User-Oriented Data Quality Chris Lynnes Goddard Earth Sciences Data and Information Services Center 1/7/14
(Some) Facets of Quality • Accuracy: closeness to Truth – Bias: systematic deviation – Uncertainty: non-systematic deviation • Completeness: how well data cover a domain – Spatial – Temporal • Consistency – Spatial: absence of spurious spatial artifacts – Temporal: absence of trend, spike and offset artifacts • Resolution – Temporal: time between successive measurements of the same volume – Spatial: distance between adjacent measurements • Ease of Use • Latency: Time between data collection and receipt 1/7/14
Pretend you’re a museum curator. . . and you’re putting together an exhibit on wildfires with some cool satellite data Which data quality facet is most important to you? A – B – C – D – 1/7/14 E – Accuracy Resolution (spatial and/or temporal) Completeness (spatial and/or temporal) Latency Ease of Use Museum Curator
Which data quality facet is most important to you? A – B – C – D – 1/7/14 E – Accuracy Resolution (spatial and/or temporal) Completeness (spatial and/or temporal) Latency Ease of Use Museum Curator Poll
You’re an operational user and. . . you want to use satellite wildfire data to direct Hot. Shot team deployments Which data quality facet is most important to you? A – B – C – D – 1/7/14 E – Accuracy Resolution (spatial and/or temporal) Completeness (spatial and/or temporal) Latency Ease of Use
Which data quality facet is most important to you? A – B – C – D – 1/7/14 E – Accuracy Resolution (spatial and/or temporal) Completeness (spatial and/or temporal) Latency Ease of Use Operational User / Hot. Shot
You’re an operational user and. . . you want to use satellite wildfire data to estimate burn scar areas for landslide prediction Which data quality facet is most important to you? A – B – C – D – 1/7/14 E – Accuracy Resolution (spatial and/or temporal) Completeness (spatial and/or temporal) Latency Ease of Use
Which data quality facet is most important to you? A – B – C – D – 1/7/14 E – Accuracy Resolution (spatial and/or temporal) Completeness (spatial and/or temporal) Latency Ease of Use Operational User / Landslide
You’re an ecology researcher and. . . you want to use satellite wildfire data to predict extinction risk of threatened species Which data quality facet is least important to you? A – B – C – D – 1/7/14 E – Accuracy Resolution (spatial and/or temporal) Completeness (spatial and/or temporal) Latency Ease of Use
Which data quality facet is least important to you? A – B – C – D – 1/7/14 E – Accuracy Resolution (spatial and/or temporal) Completeness (spatial and/or temporal) Latency Ease of Use Ecology Researcher
You’re a remote sensing researcher. . . you want to perfect an algorithm to detect and estimate active burning areas at night with visible and infrared radiances Which data quality facet is least important to you? A – B – C – D – 1/7/14 E – Accuracy Resolution (spatial and/or temporal) Completeness (spatial and/or temporal) Latency Ease of Use
Which data quality facet is least important to you? A – B – C – D – 1/7/14 E – Accuracy Resolution (spatial and/or temporal) Completeness (spatial and/or temporal) Latency Ease of Use Remote Sensing Researcher. . .
User Needs Analysis Working Group • Chartered by NASA’s Earth Science Data Systems Working Group • Goal: Understand what users need from an Earth science data system 1/7/14
Methodology • Develop User Model – User types and characteristics • Inventory Sources of User Input • Start with ACSI Survey comments – First topic: Discovery • Assess importance/utility of complaint or suggestion across User Model segments 1/7/14 • Look at relative scores for all comments
User Classes • • • General public K-12 Teachers Undergraduates Graduate students Professors Education / Public Outreach specialists Production Centers Internal Data Providers External Data Providers Science Team Cal/Va QA and Testingl 1/7/14 • • • Data Analyst Data Tech Computer Scientist* Domain Scientist Interdisciplinary Scientist Operational User Discipline-specific Modeler Assimilation modelers Climate Modelers Web Services Decision Support Systems Data Analysis and Visualization Systems *aka Data Scientist?
Assessment vs. User Model Assess comment/suggestion against the User Model (L=low importance, H=high importance) 1/7/14
Repurposing the User Needs Analysis Methodology • Step 1: Quality Needs Assessment – Spell out quality facets – Develop a User Model • Adapt ESDSWG model? – Map quality facet importance to user type • Step 2: Quality Communication – For each user type, how do you communicate the important quality aspects? 1/7/14 • Metrics? • Visuals?
- Slides: 17
