Write a multi-dim dataset as one-d

Hdf-forum archives
1

Is there any way to make a selection from a multi-dim dataset but treat it
as a one-d array with the same memory layout. I want to stream data
linearly (in chunks) into a dataset without needing to figure out the
multi-dim coords or having rectangular chunks.

Thanks,
David

2

Pretty sure that is possible though I haven't looked at doing this kind of thing in detail.

I think you wind up having to create a different datapsace for the memory buffer than the one you get from the dataset when you open it.

https://www.hdfgroup.org/HDF5/doc/UG/HDF5_Users_Guide-Responsive%20HTML5/index.html#t=HDF5_Users_Guide%2FDataspaces%2FHDF5_Dataspaces_and_Partial_I_O.htm%23TOC_7_4_Dataspaces_and_Databc-6&rhtocid=7.2

Mark

···

From: Hdf-forum <hdf-forum-bounces@lists.hdfgroup.org<mailto:hdf-forum-bounces@lists.hdfgroup.org>> on behalf of David <list@aue.org<mailto:list@aue.org>>
Reply-To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Date: Thursday, January 7, 2016 12:00 PM
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: [Hdf-forum] Write a multi-dim dataset as one-d

Is there any way to make a selection from a multi-dim dataset but treat it as a one-d array with the same memory layout. I want to stream data linearly (in chunks) into a dataset without needing to figure out the multi-dim coords or having rectangular chunks.

Thanks,
David

3

Mark,

I see how it's possible to reinterpret data in memory but it seems like the
dataspace for the dataset is fixed to how it was created. You can only
hyperslab it but not write using a reinterpretation of its shape. I can't
find how the API allows that at least. I'm hoping there is something I
missed or a tricky way around it. I wouldn't be surprised if there wasn't
though since it breaks encapsulation of the dataset's memory layout,
although that would be sort of academic since we need to write things
knowing that datasets are row-major.

Thanks,
David

···

On Thu, Jan 7, 2016 at 12:54 PM, Miller, Mark C. <miller86@llnl.gov> wrote:

Pretty sure that is possible though I haven't looked at doing this kind of
thing in detail.

I think you wind up having to create a different datapsace for the memory
buffer than the one you get from the dataset when you open it.

https://www.hdfgroup.org/HDF5/doc/UG/HDF5_Users_Guide-Responsive%20HTML5/index.html#t=HDF5_Users_Guide%2FDataspaces%2FHDF5_Dataspaces_and_Partial_I_O.htm%23TOC_7_4_Dataspaces_and_Databc-6&rhtocid=7.2

Mark

From: Hdf-forum <hdf-forum-bounces@lists.hdfgroup.org> on behalf of David
<list@aue.org>
Reply-To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
Date: Thursday, January 7, 2016 12:00 PM
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
Subject: [Hdf-forum] Write a multi-dim dataset as one-d

Is there any way to make a selection from a multi-dim dataset but treat it
as a one-d array with the same memory layout. I want to stream data
linearly (in chunks) into a dataset without needing to figure out the
multi-dim coords or having rectangular chunks.

Thanks,
David

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5