So I'm wondering if there is a good way to write an irregular shaped 2D
array into hdf5. And example of this would be like storing vtk node
connections for an unstructured grid. First number noting the cell type and
the next numbers denoting the nodes.
So the array has some rows that are length 5 and others that are length 4
(or arbitrary). I understand how to do this with C++ vectors and push_back,
but those don't create contiguous arrays. Is there another way to create
this in a way that HDF would accept?
We actually do this exact operation for our project. What I decided to do was to first flatten from a "vector of vectors" into a single array, then write that array as a "normal" array to HDF5. Add an attribute to the HDF5 dataset to state what the array represents so that when you read it back from HDF5 you know that you will need to recreate your own data structure.
Another way that we tackled the issue was to write the data into a contiguous array and then write each of the "length" values into another contiguous array. The write the data to an HDF5 data set and the "Length" array as an attribute array of the value array. I can send a link to this implementation if you want. With your data you would end up with the following arrays:
So I'm wondering if there is a good way to write an irregular shaped 2D
array into hdf5. And example of this would be like storing vtk node
connections for an unstructured grid. First number noting the cell type
and the next numbers denoting the nodes.
So the array has some rows that are length 5 and others that are length
4 (or arbitrary). I understand how to do this with C++ vectors and
push_back, but those don't create contiguous arrays. Is there another
way to create this in a way that HDF would accept?
________________________________
From: Hdf-forum <hdf-forum-bounces@lists.hdfgroup.org> on behalf of Steven Walton <walton.stevenj@gmail.com>
Sent: Monday, October 17, 2016 12:34 PM
To: HDF Users Discussion List
Subject: [Hdf-forum] Irregular 2D array write (C++)
So I'm wondering if there is a good way to write an irregular shaped 2D array into hdf5. And example of this would be like storing vtk node connections for an unstructured grid. First number noting the cell type and the next numbers denoting the nodes.
So the array has some rows that are length 5 and others that are length 4 (or arbitrary). I understand how to do this with C++ vectors and push_back, but those don't create contiguous arrays. Is there another way to create this in a way that HDF would accept?
So I'm wondering if there is a good way to write an irregular shaped 2D array into hdf5. And example of this would be like storing vtk node connections for an unstructured grid. First number noting the cell type and the next numbers denoting the nodes.
So the array has some rows that are length 5 and others that are length 4 (or arbitrary). I understand how to do this with C++ vectors and push_back, but those don't create contiguous arrays. Is there another way to create this in a way that HDF would accept?
So I'm wondering if there is a good way to write an irregular shaped 2D array into hdf5. And example of this would be like storing vtk node connections for an unstructured grid. First number noting the cell type and the next numbers denoting the nodes.
9 23 41 54 12
9 46 29 19 60
5 93 18 58
5 29 58 17
9 50 38 58 95
So the array has some rows that are length 5 and others that are length 4 (or arbitrary). I understand how to do this with C++ vectors and push_back, but those don't create contiguous arrays. Is there another way to create this in a way that HDF would accept?
-Steven
Your corresponding C++ type for this array would be
std::vector<std::vector<int>>
such as for storing cells in mixed meshes, like quads and triangles mixed?
If so, you need a variable-length array data type of type integer, i.e., HDF5 will see it as
vector<hvl_t>
It's not overly efficient to use variable-length data types, so it would be better to sort the cells into groups 3-element and 4-elements, and save that as two datasets of constant-length data type, but if the size of elements must be mixed in one dataset, then a variable length data type will be a direct match of that structure.
Werner
···
On 17.10.2016 18:34, Steven Walton wrote:
So I'm wondering if there is a good way to write an irregular shaped 2D array into hdf5. And example of this would be like storing vtk node connections for an unstructured grid. First number noting the cell type and the next numbers denoting the nodes.
So the array has some rows that are length 5 and others that are length 4 (or arbitrary). I understand how to do this with C++ vectors and push_back, but those don't create contiguous arrays. Is there another way to create this in a way that HDF would accept?
--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362
------------------------------
*From:* Hdf-forum <hdf-forum-bounces@lists.hdfgroup.org> on behalf of
Steven Walton <walton.stevenj@gmail.com>
*Sent:* Monday, October 17, 2016 12:34 PM
*To:* HDF Users Discussion List
*Subject:* [Hdf-forum] Irregular 2D array write (C++)
So I'm wondering if there is a good way to write an irregular shaped 2D
array into hdf5. And example of this would be like storing vtk node
connections for an unstructured grid. First number noting the cell type and
the next numbers denoting the nodes.
So the array has some rows that are length 5 and others that are length 4
(or arbitrary). I understand how to do this with C++ vectors and push_back,
but those don't create contiguous arrays. Is there another way to create
this in a way that HDF would accept?
Also, in reading that, using unlimited dimensions *also*requires* chunking too.
But, my suggestion #2 replaces use of unlimited dimensions with a max and relies upon ability to set chunk size such that wasted space (due to partially written chunks if any) might be small. Note, HDF5 will only ever allocate space in the file for chunks that are actually written. So, you can have a very large, sparse 2D array (like a banded matrix or something) that really doesn't take up that much space on disk.
So I'm wondering if there is a good way to write an irregular shaped 2D array into hdf5. And example of this would be like storing vtk node connections for an unstructured grid. First number noting the cell type and the next numbers denoting the nodes.
So the array has some rows that are length 5 and others that are length 4 (or arbitrary). I understand how to do this with C++ vectors and push_back, but those don't create contiguous arrays. Is there another way to create this in a way that HDF would accept?
So I'm wondering if there is a good way to write an irregular shaped 2D array into hdf5. And example of this would be like storing vtk node connections for an unstructured grid. First number noting the cell type and the next numbers denoting the nodes.
So the array has some rows that are length 5 and others that are length 4 (or arbitrary). I understand how to do this with C++ vectors and push_back, but those don't create contiguous arrays. Is there another way to create this in a way that HDF would accept?
