Statistics – Applications
Scientific paper
Aug 2006
adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2006iauss...6e...8b&link_type=abstract
Astronomical Data Management, 26th meeting of the IAU, Special Session 6, 22 August, 2006 in Prague, Czech Republic, SPS6, #8
Statistics
Applications
Scientific paper
The Montage mosaic engine supplies on-request image mosaic services for the NVO astronomical community. A companion paper describes scientific applications of Montage. This paper describes one application in detail: the generation at SDSC of a mosaic of the 2MASS All-sky Image Atlas on the NSF TeraGrid. The goals of the project are: to provide a "value-added" 2MASS product that combines overlapping images to improve sensitivity; to demonstrate applicability of computing at-scale to astronomical missions and surveys, especially projects such as LSST; and to demonstrate the utility of the NVO Hyperatlas format. The numerical processing of an 8-TB 32-bit survey to produce a 64-bit 20-TB output atlas presented multiple scalability and operational challenges. An MPI Python module, MYMPI, was used to manage the alternately sequential and parallel steps of the Montage process. This allowed us to parallelize all steps of the mosaic process: that of many, sequential steps executing simultaneously for independent mosaics and that of a single MPI parallel job executing on many CPUs for a single mosaic. The Storage Resource Broker (SRB) developed at SDSC has been used to archive the output results in the Hyperatlas. The 2MASS mosaics are now being assessed for scientific quality. The input images consist of 4,121,440 files, each 2 MB in size. The input files that fall on mosaic boundaries are opened, read, and used multiple times in the processing of adjacent mosaics, so that a total of 14 TB in 6,275,494 files are actually opened and read in the creation of mosaics across the entire survey. Around 130,000 CPU-hours were used to complete the mosaics. The output consists of 1734 6-degree plates for each of 3 bands. Each of the 5202 mosaics is roughly 4 GB in size, and each has been tiled into a 12x12 array of 26-MB files for ease of handling. The total size is about 20 TB in 750,000 tiles.
Berriman Bruce G.
Brieger L.
Good John C.
Jacob Joseph C.
Katz Sandor D.
No associations
LandOfFree
An All-Sky 2MASS Mosaic Constructed on the TeraGrid does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with An All-Sky 2MASS Mosaic Constructed on the TeraGrid, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and An All-Sky 2MASS Mosaic Constructed on the TeraGrid will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-1625745