hl_region and parallel I/O

While I respect hdf5 hyperslabs I never have warmed to them, so I was
excited to see the announcement of hl_region

some comments:

- what is this 'h5cc' your makefile looks for? serial i/o? bah, who
  wants that? :>

- just wanted to make sure I did not miss anything, but it looks like
  you can only do structured i/o in the file, but must have a
  contiguous region of memory.

- in the low-level api I turn on collective I/O with a call to
  H5Pset_fapl_mpio and H5Pset_dxpl_mpio, but there's no plist
  parameter to H5LTread_region.

I guess NPOESS has no need for parallel I/O . Well, back to the
low-level API for me, then :>

==rob

···

--
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA

Rob,

Thank you for your comments!

While I respect hdf5 hyperslabs I never have warmed to them, so I was
excited to see the announcement of hl_region

some comments:

- what is this 'h5cc' your makefile looks for? serial i/o? bah, who
wants that? :>

yes, serial :slight_smile:

- just wanted to make sure I did not miss anything, but it looks like
you can only do structured i/o in the file, but must have a
contiguous region of memory.

yes, this is correct.

- in the low-level api I turn on collective I/O with a call to
H5Pset_fapl_mpio and H5Pset_dxpl_mpio, but there's no plist
parameter to H5LTread_region.

I guess NPOESS has no need for parallel I/O . Well, back to the
low-level API for me, then :>

Currently, it is only sequential I/O.

Based on your email, it looks like there is a need for a better interface to handle hyperslabs in HDF5 (in both sequential and parallel modes).

All,

Please share with us your use cases and desired interfaces.

Elena

···

On Sep 3, 2010, at 3:55 PM, Rob Latham wrote:

==rob

--
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Oh, I don't have any better suggestions. I started to think of some
changes that would make it more parallel-io friendly and then I
quickly ended up right back at the regular hdf5 parallel I/O
interface.

==rob

···

On Fri, Sep 03, 2010 at 04:10:34PM -0500, Elena Pourmal wrote:

> I guess NPOESS has no need for parallel I/O . Well, back to the
> low-level API for me, then :>
>
Currently, it is only sequential I/O.

Based on your email, it looks like there is a need for a better interface to handle hyperslabs in HDF5 (in both sequential and parallel modes).

--
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA