LSST Survey Data Products Mario Juric LSST Data

LSST Survey Data Products Mario Juric LSST Data Management Project Scientist Radio Astronomy in the LSST Era, NRAO May 6 -8 th, 2013 Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 1

LSST Survey Data Products r<24. 5 (<27. 5@10 yr) 18000+ deg 2 10 mas astrom. Imaging the visible sky, once every 3 days, for 10 years (825 revisits) Telescope Images Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. Catalogs 2

LSST: Data Volume − One 6. 4 -gigabyte image every ~17 seconds − ~1000 visits (two back-to-back images), per night − 15 terabytes of raw scientific image data / night − ~2 million real time events per night, for 10 years − Alerts transmitted within 60 seconds of end of observation − A catalog of ~38 billion observed objects (24 B. galaxies, 14 B. stars) − A catalog of ~5 trillion detections (“sources”) and ~32 trillion measurements (“forced sources”) Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 3

LSST: Open Data, Open Source − LSST data, including images and catalogs, will be available with no proprietary period to the astronomical community of the United States, Chile, and International Partners − Transient alerts will be made available for world-wide distribution within 60 seconds, using standard VO protocols − LSST data processing stack will be free software (licensed under the GPL, v 3 -or-later) − All science will be done by the community (not the Project!), using LSST’s data products Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 4

Levels of Data Products http: //ls. st/srd Nightly Annual User-created Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 5

Data Rights Transients: Everyone Solar System: US/Chile/Int’l Partners Creation: US/Chile/International Partners, as granted by a TAC Use: As granted by the owner Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 6

Level 1: Transients Moving Objects Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 7

Level 1: Transients − Sized for: 2 M/night (average), 2 k/visit (average), 10 k/visit (peak) − Issued in 60 seconds (broadcast world-wide as VOEvents) − All measurements performed on image differenced against a deep template − Plan to include: – position – flux, size, and shape – light curves in all bands (up to a ~year; stretch: all) – variability characterization (eg. , low-order light-curve moments, probability the object is variable) – cut-outs centered on the object (template, difference image) − Do not plan to include: – variability classification (eg. , “this object is an RR Lyrae”) Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 8

Level 1: Moving Objects − Solar system objects are detected in difference images as sources with positive flux. These sources are then processed by the Moving Object Pipeline (MOPS), which links sources together into tracks of individual objects and determines orbits for them. − Availability: within 24 hrs of orbit determination − Planning to include: – Object identification – Orbital elements – Photometric properties in each band – Data quality flags – No shape reconstruction – No taxonomical classification Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 9

Level 2: Annual Data Processing Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 10

Level 2: Data Releases − Data releases – Performed annually, except in Year 1 • Two DRs in the first year (6 -mo) – No proprietary period (US/Chile/Partners) − Complete reprocessing – Every DR will reprocess all data taken up to the beginning of that DR − Major products – Images – Catalogs − Accessing the catalogs – Database and SUI – Remote access APIs, VO protocols (e. g. , Table Access Protocol) Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 11

Level 2 Processing LSST processing will begin with production of a series of coadds, where detection of objects will occur. Detections on co-adds will be deblended and associated to form a master object list. The objects will be simultaneously characterized in all observed epochs (Multi. Fit). Time variability will be characterized by independent measurement of sources in individual epochs. * * * draft * * * Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 12

Level 2 Catalog Guiding Principles − Defining the contents of Level 2 catalogs: 1. “Maximize science enabled by the catalogs” • The large majority of LSST science cases should be enabled by just the catalog. • Reprocessing images is costly and difficult for the end-user. 2. “Provide simple but useful, commonly used, external or derived, quantities” • Example: E(B-V) values for each object. • Example: Photo-z using well known, published, algorithms. Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 13

Example: Object Table − Object characterization (models): – Moving Point Source model (11 parameters) – Double Sérsic model (bulge-disk decomposition; 18 parameters) • Maximum likelihood peak • Samples of the posterior − Object characterization (model-free): – Centroid: (α, δ), per band – Adaptive moments and ellipticity measures (per band) – Petrosian and Kron fluxes and radii (per band) − Colors: – Seeing-independent measure of object color − Variability statistics: – Period, low-order light-curve moments − Metadata: Deblend status, flags, etc… Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 14

LSST Data Products Definition Document Coming soon (August 2013) LSST Final Design Data Products Definition document An overview of LSST data products LSST and associated software plans. Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 15

Level 3: User-created Data Products Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 16

Level 3 − “Level 3” products will be created by the community using LSST’s hardware and software. – For processing that would greatly benefit from being co-located with the LSST data • Likely weighted towards science cases involving the images • Envisioning a TAC – For alternative catalogs (e. g. , alternative photo-Z calculation) • Like “mydb” in SDSS – Can migrate to Level 2, if found to be universally useful • LSST will provide support for this migration (e. g. , help with the necessary software engineering) Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 17

Level 3 − Examples: – Searching for extremely faint Kuiper-Belt Objects using “shift-andstack” – Characterization of diffuse structures (eg. , ISM) – Extremely crowded field photometry (eg. , globular clusters) − LSST has a mandate to provide 10% of its total storage and computing resources for Level 3 – Example: DR 2 Level 3 allocation for all users (~in year 2022) is: • 20 TFLOPS compute • 300 TB database (includes the “mydb” space), 1 PB disk space − LSST archive will be located in the National Petascale Computing Facility at NCSA. Significant additional supercomputing is expected to be available at the same site (e. g. , NPCF currently hosts the Blue Waters supercomputer). Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 18

Example: Background-matched co-add ofw. Algorithms: Co-adds SDSS Stripe 82 in the vicinity of M 2, Generated wit LSST pipeline prototypes. background matching

Following LSST Development − General LSST news: − LSST Data Management news: @LSST @mjuric − Public LSST website: http: //lsst. org − Software development: http: //dev. lsstcorp. org Radio Astronomy in the LSST Era – NRAO, Charlottesville, VA – May 6 -8 th 2013. 20

Thank you for your attention! Questions?