The Community for Data Integration CDI bringing together

The Community for Data Integration (CDI) bringing together people, tools, and information Leslie Hsu, CDI Coordinator June 15, 2016 U. S. Department of the Interior U. S. Geological Survey

CDI is what exists today because of… • Madison Langseth • Kevin Gallagher, Tim Quinn, Cheryl Morris, Linda Gunderson • Jen Carlino, Michelle Chang • CDI Coordinators past & present • CDI members and participants Community for Data Integration (CDI) 2

My path to CDI <2008 2009 2010 2011 Community for Data Integration (CDI) 3 2012 2013 2014 2015

Motivation for CDI • USGS is a distributed organization – practitioners may be isolated from those with similar interests • It is difficult to keep up with latest technology, tools, and policies. “USGS data resources are so vast that many scientists are unaware of the data holdings that may be directly relevant to their research. ” – Holl, 2011 • Some of the best innovative and practical ideas bubble up from the workforce, and cross formal funding boundaries Community for Data Integration (CDI) 4

The CDI Purpose • Advance understanding of Earth systems through enhanced use of data and information including associated tools and techniques • Provide a forum for people doing work with data integration to come together to share ideas as well as learn new skills and techniques • Advance overall USGS capabilities with data and information by increasing visibility of the work of many people throughout the USGS Community for Data Integration (CDI) 5

A brief CDI history Council for Data Integration 2009 In-person meeting 2010 2011 Community for Data Integration (CDI) USGS-wide Request for Proposals 2012 2013 In-person meeting 2014 Community voting on Statements of Interest Science Support Framework 6 2015 2016 Continued growth …

Who is CDI? CDI email list top level domains as of June 2016 (398 addresses including listservs) Community for Data Integration (CDI) 7

Who is CDI? Membership by email domain name (June 2016) *Note: log scale A long tail… Community for Data Integration (CDI) 8

Main CDI activities Working group activity Social media daily weekly monthly Opportunities notifications Community for Data Integration (CDI) Special workshops and trainings Monthly meeting 9 Request for Proposals yearly Annual meeting

Social media – twitter for announcements, links of interest, making connections Community for Data Integration (CDI) 10

Social media – wiki, blog, and forum for activity notices, webinar information, questions tinyurl. com/usgscdiwiki Community for Data Integration (CDI) 11

Env Modeling WG CDI Working Groups Dynamic groups that come together to solve common challenges Science Center Strategy Dev, Process and Analyze Pages Communication WG Earth Science Themes WG High level working groups Connected Devices WG Focus groups 3 D/4 D Viz Data Release Use Case, DOI Group, Data Management Policy, Science Data Lifecycle Mobile Apps, Cloud Computing, Web App Framework, Scientific Computing, Data Best Practices, Data Policy Semantic Web WG Citizen Science WG Data Management WG Tech Stack WG 2010 2011 joint with ESIP Tech Dive 2012 2013 2014 2015 2016 *dates are approximate as mentioned in CDI annual reports Community for Data Integration (CDI) 12

CDI Working Groups Contacts High-level Working Groups Citizen Science WG Communication WG Data Management WG Earth Science Themes WG Semantic Web WG Tech Stack WG Sophia Liu, Dave Govoni JC Nelson, Marcia Mc. Niff Viv Hutchison Roland Viger Fran Lightsom Rich Signell Focus Groups Connected Devices WG Science Center DM Strategy Dev Process and Analyze Pages Tim Kern, Lance Everette Cassandra Ladino Steve Tessler Community for Data Integration (CDI) 13

Monthly Meetings • • • 2 nd Wednesday, 11 -12: 30 p Eastern Virtual (Web. Ex) For general audience Announcements from sponsors Inform community of tools and projects Working group reports Community for Data Integration (CDI) 14

Trying to solve a challenging problem and wondering if another group is working on the same issue? The CDI community of practice has a powerful collective body of knowledge that could be tapped to form connections and collaborations. We’ll have Scientist’s Challenge “lightning talks” (5 minutes) to kick off Monthly Meetings and post slides on the CDI community forum to stimulate discussion and generate community connections. How it works: https: //goo. gl/pkx. Fv. P Community for Data Integration (CDI) 15 Image courtesy Sira Ananwong at Free. Digital. Photos. net Scientist’s challenge

• Web API using OPe. NDAP • Archiving Models for reproducibility • Data access and transfer with external partners • Best way to make SQL database accessible • Getting a multi-center geochemistry lab group running with data best practices • Mentoring support for programming languages • Are there field equipment databases? • Repurposing USGS applications for other data types • Best way to capture a workflow Community for Data Integration (CDI) 16 Image courtesy Sira Ananwong at Free. Digital. Photos. net Scientist’s challenge

Special trainings and workshops • • Community-selected topics Technical skills Hackathons Help meet USGS data policies Community for Data Integration (CDI) 17

Annual Meetings • • Trainings Presentations Collaborations Networking Community for Data Integration (CDI) 18

Request for Proposals • USGS-wide • Two-phase process – Statements of Interest and full proposals • Guiding principles • Science Support Framework • Encourage collaborative work Community for Data Integration (CDI) 19

Community voting on Statements of Interest • Allows a public dialog between evaluators and proposers • Allows proposers to improve their ideas before the full proposal Community for Data Integration (CDI) 20

Guiding Principles • Focus on targeted efforts that yield near-term benefits to Earth and biological science • Leverage existing capabilities and data • Implement and demonstrate innovative solutions (e. g. methodologies, tools, or integration concepts) that could be used or replicated by others at scales from project to enterprise • Preserve, expose, and improve access to Earth and biological science data, models, and other outputs • Develop, organize, and share knowledge and best practices in data integration Community for Data Integration (CDI) 21

CDI Science Support Framework Community for Data Integration (CDI) 22

FY 2016 Projects in the CDI Science Support Framework Data Management Training Clearinghouse Integration of Phenological Forecast Maps Visualization of landuse scenario data Integrating the SDM Foundation process into current scientific workflow systems Framework for Web portal development Publishing Digital Data for Dynamic Visualization 2016 Data at Risk CDI Proposal Community vulnerability to hazards National Stream Characterization Integration of National Soil and Wetland Datasets Crowd-sourced Earthquake Detections Hunting Invasive Species with HTCondor Birds and the Bakken Community for Data Integration (CDI) 23

Selected products Data and information management • • • Fastest way to access projects: www. usgs. gov/cdi Science Data Lifecycle Data Management Website Metadata Wizard Science. Base enhancements Improving access to UAS data Community for Data Integration (CDI) 24

Selected products Citizen Science and Crowdsourcing • My. Science – inspiration for citizenscience. gov 2016 2013 • Tweet Earth. Quake Dispatch enhancements • Citizen Science Workshop and report Community for Data Integration (CDI) 25

Selected products Applications and tools • • Mobile application checklist Science. Cache Digital Grain Size App Dam Removal Information Portal (DRIP) Community for Data Integration (CDI) 26

We’re listening – CDI Idea. Lab usgs 1. uservoice. com • Introduce community ideas • Let the community vote on new directions • Track progress (see USGS Idea. Lab at usgs. uservoice. com) Community for Data Integration (CDI) 27

Community for Data Integration (CDI) 28

Main CDI activities Working group activity Social media daily weekly monthly Opportunities notifications Community for Data Integration (CDI) Special workshops and trainings Monthly meeting 29 Request for Proposals yearly Annual meeting

CDI in the organizational landscape Community for Data Integration (CDI) 30

Internal connections • Community for Data Integration: grass roots community of practice to leverage and integrate existing data and new technologies for interdisciplinary research and innovative data management • Powell Center: opportunities for scientist-driven interdisciplinary collaboration around complex questions using existing information to produce new knowledge and hopefully, wisdom • Innovation Center: repurposing of technology (tools or techniques) for unintended but serendipitous ends. Community for Data Integration (CDI) 31

External connections to leverage outputs “Volunteer fatigue has become a serious obstacle” –L. Powers, Connecting Communities session abstract Community for Data Integration (CDI) 32

CDI @ upcoming meetings • • • ESIP Sci. Data. Con Research Data Alliance Geological Society of American Geophysical Union Community for Data Integration (CDI) 33

Two FAQ Q 2: Why should I participate? A 2: Q 1: Can CDI host my data/model? A 1: Sorry, no, CDI is not infrastructure, but the community may help you find solutions. Community for Data Integration (CDI) 34

What are we working toward? • Efficient knowledge exchange • Solving shared data integration and management challenges • Wherever the community leads us… Community for Data Integration (CDI) 35

What can you do 1. Follow the blog (tinyurl. com/usgscdiblog) and twitter (@USGS_CDI) 2. Bookmark wiki: tinyurl. com/usgscdiwiki 3. Use the Idea. Lab and the Forum 4. Present a topic or challenge at the CDI Monthly Forum 5. Propose a working group 6. Submit a statement of interest 7. Ask for things 8. Help spread the word 9. Join by emailing cdi@usgs. gov 10. Contact me at lhsu@usgs. gov Community for Data Integration (CDI) 36

Join us – see the CDI Calendar tinyurl. com/usgscdicalendar Calendar also linked here Community for Data Integration (CDI) 37

Review: What challenges does CDI address? • USGS is a distributed organization – practitioners may be isolated from those with similar interests • It is difficult to keep up with latest technology, tools, and policies • Some of the best ideas bubble up from the work force and cross formal funding boundaries Community for Data Integration (CDI) 38

Stay in touch with CDI Web: www. usgs. gov/cdi Email: cdi@usgs. gov Twitter: @USGS_CDI Wiki: tinyurl. com/usgscdiwiki Blog: tinyurl. com/usgscdiblog Calendar: tinyurl. com/usgscdicalendar Community for Data Integration (CDI) 39

Learn more • CDI wiki – tinyurl. com/usgscdiwiki, archive of everything CDI • Annual reports – USGS OFR from 2013 -2015 – Community for Data Integration 201 X Annual Report, on the wiki • Project pages – Science. Base Community – Community for Data Integration, accessible from Projects and Products subpage of www. usgs. gov/cdi Community for Data Integration (CDI) 40