What the Data Products Program Offers Users Todd

What the Data Products Program Offers Users Todd Boroson Dick Shaw Presentation to NOAO Users Committee October 23, 2003

Data Products Program – Mission/Vision • Shift away from low value-added software to reduce data • Shift towards providing calibrated data products, catalogs, services • Full commitment to enabling public access to NOAO data; place great value on legacy data products for new science • Full involvement with international efforts to standardize data access, products, and services

Data Products Program – Overview • Resources – Evolved from IRAF group – currently 12 programmers, 5 scientists – Developing relationships with supercomputer centers – Outside projects fund activities not directly supporting core mission • Structure/process – Group split between Tucson and La Serena – Project scientist/lead programmer – Formal management w/ schedules, plans, reviews • Guidelines – Priorities set by scientific oversight, community interaction – Integrated program, external commitments also affect priorities – Ultimate goal of supporting access to LSST data and tools has stepping stones of other large f-p arrays (NEWFIRM, ODI)

Components of DPP • Archive – “User Interface” for community to access NOAO data products – Must be extensible to LSST era – Vision is workspace/tools/NVO-compliance • Data Reduction Pipelines – Gain experience with pipeline for Mosaic imagers – Incorporate time-domain capabilities – Develop further for NEWFIRM • Data Reduction Tools (incl. Gemini) – Collaborative project with Gemini – These serve as modules for pipelines • IRAF System – Improvements evaluated in terms of supporting DPP infrastructure, providing support for pipelines, and making community more selfsufficient – Movement toward more “open” model – System improvements for better scripting environment

More components of DPP • Data Transport System – “glue” that makes system approach possible • NVO – Participate in development of international standards for data discovery and dissemination. – Provide tools to support data discovery and visualization with emphasis on data we are providing • New analysis tools – Link with NSA, NVO • Work on LSST – Most interested in ensuring community access to data and the services necessary to support community research

Evolution of DPP Capabilities • Obtain survey data products through NSA • Pipelines available to assist observers • All raw data available through archive – Immediately to observers (replace STB) – After proprietary period to community • Reduced data available through archive • Data access and tools for other/new big surveys & instruments – – MACHO/Super. MACHO NEWFIRM/ODI/? Pan-STARRS LSST

Issues for Users • Difficulties of calibration and data quality for ground-based O/IR data – Driven by science objectives for archived data – One strategy is to motivate through promise of archive, pipeline – Another is instrument support for prescribed protocols only – Should ultimate usefulness of data in archive be a criterion for TAC consideration? • Management of proprietary periods – 18 month default; exceptions? • Ongoing community interaction/guidance for DPP