Organizing the Extremely Large LSST Database for Real-Time Astronomical Processing
نویسندگان
چکیده
The Large Synoptic Survey Telescope (LSST) will catalog billions of astronomical objects and trillions of sources, all of which will be stored and managed by a database management system. One of the main challenges is real-time alert generation. To generate alerts, up to 100K new difference detections have to be cross-correlated with the huge historical catalogs, and then further processed to prune false alerts. This paper explains the challenges, the implementation of the LSST Association Pipeline and the database organization strategies we are planning to use to meet the real-time requirements, including data partitioning, parallelization, and pre-loading.
منابع مشابه
Large Synoptic Survey Telescope: Overview
A large wide-field telescope and camera with optical throughput over 200 m deg – a factor of 50 beyond what we currently have – would enable the detection of faint moving or bursting optical objects: from Earth threatening asteroids to energetic events at the edge of the optical universe. An optimized design for LSST is a 8.4 m telescope with a 3 degree field of view and an optical throughput o...
متن کاملar X iv : a st ro - p h / 03 02 10 2 v 1 6 F eb 2 00 3 Large Synoptic Survey Telescope : Overview
A large wide-field telescope and camera with optical throughput over 200 m 2 deg 2 – a factor of 50 beyond what we currently have – would enable the detection of faint moving or bursting optical objects: from Earth threatening asteroids to energetic events at the edge of the optical universe. An optimized design for LSST is a 8.4 m telescope with a 3 degree field of view and an optical throughp...
متن کاملScience Data Quality Assessment for the Large Synoptic Survey Telescope
LSST will have a Science Data Quality Assessment (SDQA) subsystem for the assessment of the data products that will be produced during the course of a 10 yr survey. The LSST will produce unprecedented volumes of astronomical data as it surveys the accessible sky every few nights. The SDQA subsystem will enable comparisons of the science data with expectations from prior experience and models, a...
متن کاملScientific Data Mining in Astronomy
We describe the application of data mining algorithms to research problems in astronomy. We posit that data mining has always been fundamental to astronomical research, since data mining is the basis of evidencebased discovery, including classification, clustering, and novelty discovery. These algorithms represent a major set of computational tools for discovery in large databases, which will b...
متن کاملDesigning a Multi-petabyte Database for LSST
The 3.2 giga-pixel LSST camera will produce approximately half a petabyte of archive images every month. These data need to be reduced in under a minute to produce real-time transient alerts, and then added to the cumulative catalog for further analysis. The catalog is expected to grow about three hundred terabytes per year. The data volume, the real-time transient alerting requirements of the ...
متن کامل