nesting compound data types

Hdf-forum archives
1

Hi folks -
Back in the summer, I had some questions about dealing with compound
datatypes and got some helpful advice. I had to put the project on the
shelf for awhile but have finally resumed working on it.

To summarize: I'm working on an application that processes various sorts of
messages, where for our purposes a message can be considered to be a
collection of name-value pairs. The values are mostly numeric, but can be
strings. I've been working on extending the application so that it can
produce HDF output. The HDF subsystem is told when a message is starting
and stopping, and it's given the name-value pairs one at a time. The
messages can be arbitrarily large, and it's not safe to assume they can be
kept in memory.

The approach I eventually settled on is to handle each message in two
passes. The first pass is used to tell HDF the names and sizes of fields,
and the second pass is used to actually write out the values. Because I'm
handling one value at a time, the writing part is a bit involved. Here's a
notional snippet:
// name, value, dataType, dtSize, ctDataSet and ctSpace have been defined
elsewhere

hid_t valueDT = H5Tcreate(H5T_COMPOUND, dtSize);
herr_t status = H5Tinsert(valueDT, name, 0, dataType);
hid_t filespace = H5Dget_space(ctDataSet);
hsize_t offset[] = { 0 };
hsize_t dim[] = [ 1 };
status = H5Sselect_hyperslab(filespace, H5S_SELECT_SET, offset, NULL,
                                             dim, NULL);
status = H5Dwrite(ctDataSet, valueDT, ctSpace, filespace, H5P_DEFAULT,
&value);
.
.
.
I'm doing this so that valueDT will get associated with an existing
datatype in ctDataSet (via name matching).

The refinement I'm working on now is to provide support for nested compound
datatypes. There's enough information in the names to let me figure out the
parent-child relationships. I've got part of it working; I can communicate
the structure to HDF, and get an output file that reflects the structure.
Writing out the actual data is what's challenging, and I'm looking for
advice on:
1) the name-matching step
2) the arguments to H5Sselect_hyperslab - are any of them after the first
two actually appropriate?

Sorry about the length, but the context is pretty complicated and I would
rather lay it out in advance than end up having to fill in missing details
(which I'll doubtless end up doing anyway :)).

Thanks for your patience!

-Josiah Slack

2

In case I went overboard with detail in the original post, I'll try to boil
down my question to its essential part. There is existing documentation
that shows how to write individual fields in a compound type, but it's not
clear to me how to extend the approach to multiple levels of compound type.
Let's say that I've got a compound type that contains an element called
position, itself a compound type. position has elements called x, y, and z.
Is there a way of writing out just x?

Thanks for your attention.

···

On Mon, Nov 18, 2013 at 6:14 PM, Josiah Slack <josiahnmi@gmail.com> wrote:

Hi folks -
Back in the summer, I had some questions about dealing with compound
datatypes and got some helpful advice. I had to put the project on the
shelf for awhile but have finally resumed working on it.

To summarize: I'm working on an application that processes various sorts
of messages, where for our purposes a message can be considered to be a
collection of name-value pairs. The values are mostly numeric, but can be
strings. I've been working on extending the application so that it can
produce HDF output. The HDF subsystem is told when a message is starting
and stopping, and it's given the name-value pairs one at a time. The
messages can be arbitrarily large, and it's not safe to assume they can be
kept in memory.

The approach I eventually settled on is to handle each message in two
passes. The first pass is used to tell HDF the names and sizes of fields,
and the second pass is used to actually write out the values. Because I'm
handling one value at a time, the writing part is a bit involved. Here's a
notional snippet:
// name, value, dataType, dtSize, ctDataSet and ctSpace have been defined
elsewhere

hid_t valueDT = H5Tcreate(H5T_COMPOUND, dtSize);
herr_t status = H5Tinsert(valueDT, name, 0, dataType);
hid_t filespace = H5Dget_space(ctDataSet);
hsize_t offset[] = { 0 };
hsize_t dim[] = [ 1 };
status = H5Sselect_hyperslab(filespace, H5S_SELECT_SET, offset, NULL,
                                             dim, NULL);
status = H5Dwrite(ctDataSet, valueDT, ctSpace, filespace, H5P_DEFAULT,
&value);
.
.
.
I'm doing this so that valueDT will get associated with an existing
datatype in ctDataSet (via name matching).

The refinement I'm working on now is to provide support for nested
compound datatypes. There's enough information in the names to let me
figure out the parent-child relationships. I've got part of it working; I
can communicate the structure to HDF, and get an output file that reflects
the structure. Writing out the actual data is what's challenging, and I'm
looking for advice on:
1) the name-matching step
2) the arguments to H5Sselect_hyperslab - are any of them after the first
two actually appropriate?

Sorry about the length, but the context is pretty complicated and I would
rather lay it out in advance than end up having to fill in missing details
(which I'll doubtless end up doing anyway :)).

Thanks for your patience!

-Josiah Slack