Database Architecture for the Indra Cosmological Simulations

Details Database Architecture for the Indra Cosmological Simulations Database Architecture for the Indra Cosmological Simulations

Jan 2012

adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2012aas...21914513c&link_type=abstract

American Astronomical Society, AAS Meeting #219, #145.13

Statistics

Computation

Scientific paper

The Indra Simulation suite is a set of 512 cosmological N-body simulations in a 1Gpc/h-sided box producing over 100 TB of data. We present the data storage strategy developed to efficiently answer the most important questions being asked of this data. Some of these queries involve sampling all of the particle data for a particular snapshot, such as computation of particle topologies like filaments, voids, and clusters. The other queries involve searching a snapshot for information about a few particles, such as tracking halo member particle positions through time. We present a SQL database design to support the dataset by allowing efficient storage and querying. The particle data consists of positions and velocities for each particle, identified by a unique ID throughout the instance, as well as FOF halos and the density field on a power of two grid. In each timestep, the particle data is sorted along a Peano-Hilbert curve, and all particles within a single cell are placed into a single binary array stored in one row in the database using a custom SQL array library. To still allow for fast querying on these arrays, we associate a Bloom filter with each array to test whether a given particle is contained within it. A Bloom filter is a space efficient data structure to test set membership in constant time. Example query code and performance tests will be given.
The authors are grateful for support from the Gordon and Betty Moore and the W.M. Keck Foundations.

Affiliated with

Also associated with

No associations

Rating

Database Architecture for the Indra Cosmological Simulations does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Database Architecture for the Indra Cosmological Simulations, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Database Architecture for the Indra Cosmological Simulations will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-1576171