Computer Science – Databases
Scientific paper
Jul 2004
adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2004aspc..314...38t&link_type=abstract
Astronomical Data Analysis Software and Systems (ADASS) XIII, Proceedings of the conference held 12-15 October, 2003 in Strasbou
Computer Science
Databases
1
Scientific paper
For large astronomical databases like the SDSS Science Archive, data loading is potentially the most time-consuming and labor-intensive part of archive operations, and it is also the most critical: it is the last chance to examine the data before it is published. We attempted to automate this job as much as possible, and to make it easy to diagnose data and loading errors. We describe the sqlLoader --- a distributed workflow system of modules that check, load, validate and publish the data to the databases. The workflow is described by a directed acyclic graph whose nodes are the processing modules. It is designed for parallel loading and is controlled from a web interface (Load Monitor). The validation stage represents a systematic and thorough scrubbing of the data. Finally, the different data products are merged into a set of linked tables that can be efficiently searched with specialized indices and pre-computed joins.
Gray James
Szalay Alex S.
Thakar Ani
No associations
LandOfFree
From FITS to SQL - Loading and Publishing the SDSS Data does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with From FITS to SQL - Loading and Publishing the SDSS Data, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and From FITS to SQL - Loading and Publishing the SDSS Data will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-1076728