Computer Science – Performance
Scientific paper
Oct 2007
adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2007aspc..376...38f&link_type=abstract
Astronomical Data Analysis Software and Systems XVI ASP Conference Series, Vol. 376, proceedings of the conference held 15-18 Oc
Computer Science
Performance
Scientific paper
High data rates from modern telescopes are requiring high performance computing. New large survey projects in particular, like the Dark Energy Survey (DES) and the Large Synoptic Survey Telescope (LSST), are placing high demands on computing and data management systems not only in their need for fast turn-around or real-time processing, but in the expectation of multiple reprocessings of the collection as software and algorithms improve. In these applications, I/O and data movement can often incur the largest performance overhead. Even when high network bandwidth is available, filling the network pipe to its capacity is non-trivial. We present a prototype system for a data deployment tool that can make intelligent use of high-performance data transfer software to move data in a fast and efficient way. It accomplishes this by asking the application not only what data is needed and where, but also for what purpose (e.g. last night's observations on that cluster for calibration). We are developing this tool to support automated processing systems for DES and LSST.
Fleming D.
Plante Ray
No associations
LandOfFree
High Performance Data Deployment and Processing does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with High Performance Data Deployment and Processing, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and High Performance Data Deployment and Processing will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-955624