Writing variables of different types to a dataset (table?)


#1

Hi, I’m new to hdf5 and I would like to write values to a table. I’m using Hdf5cpp I have data that I want to structure in a dataset as follows:

Name, ID, ParentID
Phil, 1, 3.0
Fred, 4, 5.0

So basically column headers and data with different types.
how would I go about this? It needs to be variable in size and I’m adding a new row every now and again…

Thanks in advance,


#2

Hi Paul,

you might be interested in these slides or the upcoming presentation did you wish to sign up/attend on Tue, Jun 30 · 6:30 PM EDT, C++TO online event.

If the data is from a sensor network / stream look at this packet table example

Below is the one from the lightning talk, this is the code you have to write, the header file is given – which contains a valid POD struct type, with arbitrary complex fields; the rest is machine generated.

01: #include <h5cpp/core>
02:     #include "generated.h"
03: #include <h5cpp/io>
04: int main(int argc, char *argv[] ){
05:	h5::fd_t fd = h5::create("lightning-talk-example.h5",H5F_ACC_TRUNC);
06:     try {
07:         h5::pt_t pt = h5::create<sn::example::complicated_struct_t>(fd, "dataset name",
08:             h5::max_dims{H5S_UNLIMITED}, h5::chunk{1024} );
09:         sn::example::complicated_struct_t event;
10:         for(size_t i=0; i<size; i++ )
12:             h5::append(pt, event);
13:     } catch ( const h5::error::any& e ){ ... }
14: }
01: namespace sn::example {
02:	struct complicated_struct_t {
03:		unsigned long idx;
        ... };
12: }

For the hmm: record? here is the header file for the struct:

#ifndef  H5TEST_STRUCT_01 
#define  H5TEST_STRUCT_01
typedef size_t type_alias_t;
namespace sn {
 namespace other {
    struct struct_t {                    // POD struct with nested namespace
        type_alias_t                idx; // aliased type 
        double              field_02[3]; // const array mapped 
    };
 }
 namespace example {
    struct complicated_struct_t {        // POD struct with nested namespace
        type_alias_t                idx; // typedef type 
        float               field_02[7];  // array of elementary types
        sn::other::struct_t field_03[5]; // embedded struct 1D array
        other::struct_t  field_04[3][8]; // array of arrays 
    };
 }
}
#endif

And finally the machine generated compound data type descriptor:

#ifndef H5CPP_GUARD_kCTha
#define H5CPP_GUARD_kCTha
namespace h5 {
    template<> hid_t inline register_struct<sn::example::complicated_struct_t>(){
        hsize_t at_00_[] ={7};            hid_t at_00 = H5Tarray_create(H5T_NATIVE_FLOAT,1,at_00_);
        hsize_t at_01_[] ={3};            hid_t at_01 = H5Tarray_create(H5T_NATIVE_DOUBLE,1,at_01_);
 
        hid_t ct_00 = H5Tcreate(H5T_COMPOUND, sizeof (sn::other::struct_t));
        H5Tinsert(ct_00, "idx",	HOFFSET(sn::other::struct_t,idx),H5T_NATIVE_ULONG);
        H5Tinsert(ct_00, "field_02",	HOFFSET(sn::other::struct_t,field_02),at_01);
        hsize_t at_02_[] ={5};            hid_t at_02 = H5Tarray_create(ct_00,1,at_02_);
        hsize_t at_03_[] ={8};            hid_t at_03 = H5Tarray_create(ct_00,1,at_03_);
        hsize_t at_04_[] ={3};            hid_t at_04 = H5Tarray_create(at_03,1,at_04_);
 
        hid_t ct_01 = H5Tcreate(H5T_COMPOUND, sizeof (sn::example::complicated_struct_t));
        H5Tinsert(ct_01, "idx",	HOFFSET(sn::example::complicated_struct_t,idx),H5T_NATIVE_ULONG);
        H5Tinsert(ct_01, "field_02",	HOFFSET(sn::example::complicated_struct_t,field_02),at_00);
        H5Tinsert(ct_01, "field_03",	HOFFSET(sn::example::complicated_struct_t,field_03),at_02);
        H5Tinsert(ct_01, "field_04",	HOFFSET(sn::example::complicated_struct_t,field_04),at_04);
 
        H5Tclose(at_00); H5Tclose(at_01); H5Tclose(ct_00); H5Tclose(at_02); H5Tclose(at_03);
        H5Tclose(at_04); 
 
        return ct_01;
    };
}
H5CPP_REGISTER_STRUCT(sn::example::complicated_struct_t);
#endif

best wishes: steven


#3

H Thanks. Just a quick question, Is h5cpp/core different to the H5Cpp.h libs? Seperate? Newer?


#4

Hi Paul,
yes h5cpp is a separate header only project, it has a requirement of libhdf5 but not the high level library.

It is rather recent: about 2 years old, based on a thin performance templates I had been privately maintaining since 2011 for high frequency trading data management. The library has been heavily influenced by Elena Pourmal’s HDF5 C++ meeting group participants: Marc Paterno, Chris Green, Eugen Wintersberger, Martin Shetty … this user group, and most importantly by Gerd Heber who has been collaborating with me on this project since its inception.

best: steve