The NOAO Data Cache Initiative - Building a Distributed Online Datastore

Details The NOAO Data Cache Initiative - Building a Distributed Online Datastore The NOAO Data Cache Initiative - Building a Distributed Online Datastore

Dec 2005

adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2005aspc..347..679s&link_type=abstract

Astronomical Data Analysis Software and Systems XIV ASP Conference Series, Vol. 347, Proceedings of the Conference held 24-27 Oc

Other

4

Scientific paper

The Data Cache Initiative (DCI) of the NOAO Data Products Program is a prototype Data Transport System for NOAO and affiliate facilities. DCI provides pre-tested solutions for conveying data from our large suite of instrumentation to a central mountain data cache. The heart of DCI is an extension of the Save-the-Bits safestore, running for more than a decade (more than 4 million images saved, comprising more than 40 Tbytes). The iSTB server has been simplified by the removal of STB's media handling functionality, and iSTB has been enhanced to remediate each incoming header with information from a database of NOAO instrumentation and an interface to the NOAO proposal database. Each mountain data cache has been implemented on commodity hardware running Redhat 9.0. Software RAID 1 runs over hardware RAID 5 to provide maximum storage reliability for each copy of the data. Each image is transferred from Kitt Peak or Cerro Tololo to the corresponding datastore at the Tucson or La Serena data centers using an rsync-based queue adopted from NCSA. From each data center, the files are transported to the other NOAO data center and also to NCSA for off-site storage using the Storage Resource Broker (SRB) of the San Diego Supercomputer Center. Thus we have three copies of each file on spinning disks or near-online. Major institutional users will be given access to the datastores.

Affiliated with

Also associated with

No associations

Rating

The NOAO Data Cache Initiative - Building a Distributed Online Datastore does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with The NOAO Data Cache Initiative - Building a Distributed Online Datastore, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and The NOAO Data Cache Initiative - Building a Distributed Online Datastore will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-1035080