Hyperslab ordering

Hi there,

I have an application that needs to read hyperslabs of a dataset in a
different order to what they are on disk. For example, if I have a
dataset as such:

1,1,1,2,2,1,1,5,5,5,1,1

And I want my read data to be:

5,5,5,2,2

I can use two calls to H5Sselect_hyperslab to select the parts of the
dataset I want - the first call to select the 5s and a second to select
the 2s. However, hyperslab selection with SET and OR doesn't retain the
order of selection and reading straight into memory will simply give me
2,2,5,5,5. To get around this I can select by element instead of
hyperslab, or construct a selected-element memory dataspace to re-order
them (, or worse, read each hyperslab separately).

In reality my datasets are much larger and more complicated than this
and so are the individual hyperslabs (with strides, etc). It seems a
bit overkill to have to select by elements when the hyperslab
description is much more elegant. Does anyone know of any way to keep
the hyperslab selection order in HDF5 1.6.x? Is this a feature that is
in / could be added to 1.8.x?

Also, I am currently re-ordering the hyperslabs by using an
element-selected memory dataspace. Any idea if I would get better
performance if I selected by elements on the file instead and read
contiguously into memory (the datasets aren't chunked)? Or would it
make little difference?

With thanks,

Matt

···

--
_______________________________________________________________________
Matt Street MSc MBCS
Parallel Technology Support Tel: +44 (0) 118 982 4528
Applied Computer Science / HPC AWE, Aldermaston, Reading, RG7
4PR. UK.

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.

Hi Matt,

Hi there,

I have an application that needs to read hyperslabs of a dataset in a
different order to what they are on disk. For example, if I have a
dataset as such:

1,1,1,2,2,1,1,5,5,5,1,1

And I want my read data to be:

5,5,5,2,2

I can use two calls to H5Sselect_hyperslab to select the parts of the
dataset I want - the first call to select the 5s and a second to select
the 2s. However, hyperslab selection with SET and OR doesn't retain the
order of selection and reading straight into memory will simply give me
2,2,5,5,5. To get around this I can select by element instead of
hyperslab, or construct a selected-element memory dataspace to re-order
them (, or worse, read each hyperslab separately).

In reality my datasets are much larger and more complicated than this
and so are the individual hyperslabs (with strides, etc). It seems a
bit overkill to have to select by elements when the hyperslab
description is much more elegant. Does anyone know of any way to keep
the hyperslab selection order in HDF5 1.6.x? Is this a feature that is
in / could be added to 1.8.x?

  Currently, the only way to reorder elements when selecting them is to select them by element, as you are doing.

Also, I am currently re-ordering the hyperslabs by using an
element-selected memory dataspace. Any idea if I would get better
performance if I selected by elements on the file instead and read
contiguously into memory (the datasets aren't chunked)? Or would it
make little difference?

  I'm guessing it will not make much difference, for this sort of use.

  Quincey

···

On Jul 17, 2008, at 5:40 AM, Matthew Street wrote:

With thanks,

Matt

--
_______________________________________________________________________
Matt Street MSc MBCS
Parallel Technology Support Tel: +44 (0) 118 982 4528
Applied Computer Science / HPC AWE, Aldermaston, Reading, RG7
4PR. UK.

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.