Computer Science – Performance
Scientific paper
Jan 2011
adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2011aas...21720504m&link_type=abstract
American Astronomical Society, AAS Meeting #217, #205.04; Bulletin of the American Astronomical Society, Vol. 43, 2011
Computer Science
Performance
Scientific paper
We have developed and tested a high performance computing enabled data management system for the Dark Energy Survey (DES). This highly automated system with built in data quality testing uses a central database at the National Center for Supercomputing Applications to store survey, image and processing metadata that enable automation, provenance tracking and data quality tagging at the image, catalog and object levels. Observed single and multi-epoch catalogs are also stored within the database where they are used during calibration and are available for immediate collaboration access. In addition, periodic data releases involve building new sets of tables that are distributed into external analysis databases that are located at NCSA as well as at Fermilab and other collaboration sites in the US, Brazil, UK, Spain and Germany. Processed images and other file data are distributed to Archive sites throughout the collaboration using high bandwidth grid tools. A set of portals for catalog and image access, for direct manipulation of catalog data within a users own database space, and for operation and monitoring of the data management system itself have been developed and are being used during extensive testing using SDSS scale datasets that are processed, calibrated and then served to the collaboration for scientific and data quality analyses.
The DESDM processing and calibration involves standard image detrending codes that have been specially built for DECam. Astromatic codes (E. Bertin) have been extended for DES and are being used for astrometric calibration, PSF modeling across all images, PSF homogenization during the image stacking and model fitting photometry of all detected objects. Photometric calibration uses both direct calibration for photometric data as well as cross calibration using common objects on overlapping images. Processing also includes shear measurements for all objects and a difference imaging pipeline to search for time variable objects.
Dark Energy Survey Data Management Team
Mohr Joseph John
No associations
LandOfFree
Processing, Calibration and Archiving for the Dark Energy Survey does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Processing, Calibration and Archiving for the Dark Energy Survey, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Processing, Calibration and Archiving for the Dark Energy Survey will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-1396832