redhat and hdf5

I've recently been informed that we're going to be changing operating systems and hardware architecture to redhat on an x86 blade system (or something of the sort). I'm starting to investigate what all the implications might be and am digging through the archives for items of interest, but in the mean-time I was wondering if anyone out there was using HDF5 on RedHat (no, I don't know which flavor, either, we haven't been given much information), and what complications or advantages they ran across in the process.

One thing of particular interest to me is the parallel I/O - I'd be interested in hearing comments about that WRT RH. I'm not entirely sure it's relevant to our system but it's worth researching anyway.

Hi John,

Redhat (equivalent to CentOS, sort of) is one of the HDF5 development platforms.
You should be quite okay running HDF5 in Redhat on x86 hardware.

Parallel I/O is a different issue. It depends what parallel file systems you use.
Two commercial systems, GPFS (by IBM) and Lustre (now owned by Oracle)
are possible options. Their performance varies and are highly
dependent on the characteristics of the I/O pattern of your applications.

-Albert Cheng

···

On 7/26/2011 12:28 PM, John Knutson wrote:

I've recently been informed that we're going to be changing operating systems and hardware architecture to redhat on an x86 blade system (or something of the sort). I'm starting to investigate what all the implications might be and am digging through the archives for items of interest, but in the mean-time I was wondering if anyone out there was using HDF5 on RedHat (no, I don't know which flavor, either, we haven't been given much information), and what complications or advantages they ran across in the process.

One thing of particular interest to me is the parallel I/O - I'd be interested in hearing comments about that WRT RH. I'm not entirely sure it's relevant to our system but it's worth researching anyway.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Hi Albert, thanks for the response...

Albert Cheng wrote:

Parallel I/O is a different issue. It depends what parallel file systems you use.
Two commercial systems, GPFS (by IBM) and Lustre (now owned by Oracle)
are possible options. Their performance varies and are highly
dependent on the characteristics of the I/O pattern of your applications.

Unfortunately I had some misconceptions in my thinking. To do parallel I/O, HDF5 apps actually need to set up an MPI file access property list, which would be in exclusion to any other FAPL type (e.g. sec2, direct, etc.). Is that correct?

We are currently doing writes using the iRODS VFD I developed, though reading apps are using the (potentially faster) sec2 driver.

Hi John,

Hi Albert, thanks for the response...

Albert Cheng wrote:

Parallel I/O is a different issue. It depends what parallel file systems you use.
Two commercial systems, GPFS (by IBM) and Lustre (now owned by Oracle)
are possible options. Their performance varies and are highly
dependent on the characteristics of the I/O pattern of your applications.

Unfortunately I had some misconceptions in my thinking. To do parallel I/O, HDF5 apps actually need to set up an MPI file access property list, which would be in exclusion to any other FAPL type (e.g. sec2, direct, etc.). Is that correct?

  Yes, that's correct. (Since the "bottom" layer needs to make MPI-I/O calls, instead of POSIX, etc)

  Quincey

···

On Jul 26, 2011, at 1:51 PM, John Knutson wrote:

We are currently doing writes using the iRODS VFD I developed, though reading apps are using the (potentially faster) sec2 driver.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org