Computer Science – Databases
Scientific paper
1999-07-06
Computer Science
Databases
9 pages, original at research.microsoft.com/~gray/papers/MS_TR_99_30_Sloan_Digital_Sky_Survey.doc
Scientific paper
The next-generation astronomy digital archives will cover most of the universe at fine resolution in many wave-lengths, from X-rays to ultraviolet, optical, and infrared. The archives will be stored at diverse geographical locations. One of the first of these projects, the Sloan Digital Sky Survey (SDSS) will create a 5-wavelength catalog over 10,000 square degrees of the sky (see http://www.sdss.org/). The 200 million objects in the multi-terabyte database will have mostly numerical attributes, defining a space of 100+ dimensions. Points in this space have highly correlated distributions. The archive will enable astronomers to explore the data interactively. Data access will be aided by a multidimensional spatial index and other indices. The data will be partitioned in many ways. Small tag objects consisting of the most popular attributes speed up frequent searches. Splitting the data among multiple servers enables parallel, scalable I/O and applies parallel processing to the data. Hashing techniques allow efficient clustering and pair-wise comparison algorithms that parallelize nicely. Randomly sampled subsets allow debugging otherwise large queries at the desktop. Central servers will operate a data pump that supports sweeping searches that touch most of the data. The anticipated queries require special operators related to angular distances and complex similarity tests of object properties, like shapes, colors, velocity vectors, or temporal behaviors. These issues pose interesting data management challenges.
Gray Jim
Kunszt Peter
Szalay Alexander S.
Thakar Ani
No associations
LandOfFree
Designing and Mining Multi-Terabyte Astronomy Archives: The Sloan Digital Sky Survey does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Designing and Mining Multi-Terabyte Astronomy Archives: The Sloan Digital Sky Survey, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Designing and Mining Multi-Terabyte Astronomy Archives: The Sloan Digital Sky Survey will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-357590