That title isn’t the greatest, but here’s the general sense of it.
I have a bunch of HDF5 files all linked together through a root file. We’ll call the root file Root.h5 and the others File1.h5, File2.h5, … (The reasons for splitting it up basically come down to some specific use case stuff).
What I do is create Root.h5 and (I.E) File1.h5. The code to do so is the same for both and as shown below.
h5file = H5Fcreate(path.c_str(), H5F_ACC_EXCL, H5P_DEFAULT, H5P_DEFAULT);
I then create an external link from Root.h5:["/some/path/here"] to the root node of File1.h5. The idea being that now I can open Root.h5 and interact with it as if it were one monolithic file even though the data actually lives in another file.
hid_t lcpl_id = H5Pcreate(H5P_LINK_CREATE); H5Pset_create_intermediate_group(lcpl_id, 1); status = H5Lcreate_external(relative_output_sub_path.c_str(), "/", root_h5file, exported_group_path.c_str(), lcpl_id, H5P_DEFAULT); status = H5Lcreate_external(relative_output_sub_proc_path.c_str(), "/", root_h5file, proc_group_path.c_str(), lcpl_id, H5P_DEFAULT);
A little bit later, I use the Root.h5 file handle to write my dataset to a subgroup of “here”, setting the flag to create intermediate groups in the process. I.E Root.h5:["/some/path/here/subgroup/data"]
hid_t lcpl_id = H5Pcreate(H5P_LINK_CREATE); H5Pset_create_intermediate_group(lcpl_id, 1); herr_t status; hid_t dcpl_id = H5Pcreate(H5P_DATASET_CREATE); // Set compression, chunking, etc // Create the dataspace hid_t dataspace = H5Screate_simple(dims.size(), dims.data(), NULL); // "file_id" is Root.h5, and path is "/some/path/here/subgroup/data" target_dset = H5Dcreate(file_id, path.c_str(), H5T_IEEE_F64LE, dataspace, lcpl_id, dcpl_id, H5P_DEFAULT); // Now write the data status = H5Dwrite(target_dset , H5T_NATIVE_FLOAT, H5S_ALL, H5S_ALL, H5P_DEFAULT, &data);
Now then - this all succeeds just fine. The data is written to the correct file and if I open that subfile in HDFView the data is there. What’s weird, however, is if I open the root file in HDFView and traverse that same link, I can only see up to “subgroup” - “data” is invisible when viewed this way, even though it’s actually there (“subgroup” appears to have no children). If I go back and just write “/some/path/here/data” (I.E to the root of File1.h5 rather than a subgroup) then the data is visible when traversing through Root.h5
The data is accessible as expected as well. Opening Root.h5 in some other program (in my case, Julia) and requesting “/some/path/here/subgroup/data” returns the correct information. Additionally, making modifications to either the root element or “subgroup” using HDFView, such as adding attributes, by directly accessing “File1.h5” are reflected in Root.h5, or by accessing “subgroup” from Root.h5 are reflected in File1.h5. Additionally, opening Root.h5 in Julia and writing some data to “/some/path/here/subgroup/otherdata” is accessible as expected.
I’m guessing I must have some sort of error in how I’m setting my creation properties, but reading through the documentation I can’t see anything I might’ve forgotten to set, especially since creating data through Julia works fine. Am I missing something here, or could this be a bug?