I am writing code to acquire data from an X-ray detector and write to an hdf5 file.
The code is to write data from a circular buffer (in an FPGA accelerator card) to one or more hdf5 files. Although the hdf5 library is not internally thread safe, using multiple threads with different hdf files for each thread, would allow some parallelism with one thread using libhdf5 to write the data while another thread is reading data from the FPGA card.
The firmware allows more than one way to read data from the card, so I wanted to structure the code with file and DataSet opening and data writing in separate functions, with the H5File and H5:Datset classes stored as class members, rather than constructed into local variables just before use.
The main thread creates all the hdf files, creates the data sets and sets the property to allow chunking.
Worker threads then extend the data set and the write the data to their respective files when data arrives.
The first attempts were with 1.14.0 built without thread safe (by mistake).
This created the file, created the data set and started writing data. When the 2nd thread triggered errors were caused – presumably due to me not building with thread safe.
Realising my mistake I downloaded 1.14.4-3 and built with thread safe. Note since I am using the C++ wrapper I have to add –enable-unsupported.
Now I get errors much earlier in the programe, when I try to create the dataset:
Calling createDataSet
HDF5-DIAG: Error detected in HDF5 (1.14.4-3) thread 1:
#000: …/…/hdf5-1.14.4-3/src/H5D.c line 187 in H5Dcreate2(): unable to synchronously create dataset
major: Dataset
minor: Unable to create file
#001: …/…/hdf5-1.14.4-3/src/H5D.c line 111 in H5D__create_api_common(): can’t set object access arguments
major: Dataset
minor: Can’t set value
#002: …/…/hdf5-1.14.4-3/src/H5VLint.c line 2602 in H5VL_setup_acc_args(): invalid location identifier
major: Invalid arguments to routine
minor: Inappropriate type
#003: …/…/hdf5-1.14.4-3/src/H5VLint.c line 1733 in H5VL_vol_object(): invalid identifier
major: Invalid arguments to routine
minor: Inappropriate type
terminate called after throwing an instance of ‘H5::FileIException’
Investigating I discovered that assiging the new H5File object into a class member was stopping the following call to createDataSet() working.
This is reaching the limits of my C++ understanding, but this seems to be a situation where the C++ 11 move semantics would be useful, moving the h5 file identifier from the newly constructed class into the target class and then setting it to invalid in the source class to prevent the destructor closing it. However, H5C++ predates C++11, so I don’t understand how I am should approach this. Some googling showed std::swap existed before std::move. I also hoped the reference counting within HDF5 would ensure that the file was not closed before I had finished with it.
Experimenting to make minimal demonstration of the problem, I just need to try to assign the H5File object to another local variable to stop it working:
#include
#include
#include
#include
#include
#include
#include
#include <condition_variable>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <poll.h>
#include <unistd.h>
#include “H5Cpp.h”
#include “hdf5_hl.h”
int main()
{
const char *fName = “test1.h5”;
hsize_t dimsFile[6], maxDimsFile[6], dimsChunk[6];
H5::H5File h5File;
H5::H5File tmpH5File(fName, H5F_ACC_TRUNC);
H5::DataSet dataSet;
dimsFile[0] = 1;
dimsFile[1] = 80;
dimsFile[2] = 80;
dimsFile[3] = 16;
H5::DataSpace dataSpaceFile1(4, dimsFile);
printf("Creating dataset in file %s using variable tmpH5File\n", fName);
dataSet = tmpH5File.createDataSet("/mapped", H5::PredType::NATIVE_UINT32, dataSpaceFile1);
dataSpaceFile1.close();
dataSet.close();
tmpH5File.close();
#if 1
fName = “test2.h5”;
h5File = H5::H5File(fName, H5F_ACC_TRUNC);
H5::DataSpace dataSpaceFile2(4, dimsFile);
printf("Creating dataset in file %s using variable H5File\n", fName);
dataSet = h5File.createDataSet("/mapped", H5::PredType::NATIVE_UINT32, dataSpaceFile2);
dataSpaceFile2.close();
dataSet.close();
h5File.close();
#endif
fName = “test3.h5”;
H5::H5File tmpH5File3(fName, H5F_ACC_TRUNC);
std::swap(h5File, tmpH5File3);
H5::DataSpace dataSpaceFile3(4, dimsFile);
H5::DSetCreatPropList pList3(H5P_DATASET_CREATE);
printf("Creating dataset in file %s using variable H5File\n", fName);
dataSet = h5File.createDataSet("/mapped", H5::PredType::NATIVE_UINT32, dataSpaceFile3);
dataSpaceFile3.close();
dataSet.close();
h5File.close();
}
Compiling against hdf5 1.14.4-3 using
CFLAGS=-fPIC …/hdf5-1.14.4-3/configure --prefix=/usr/local/hdf5 --enable-cxx --enable-threadsafe --enable-unsupported
I get errors:
./hdf_tests_simpler
Creating dataset in file test1.h5 using variable tmpH5File
Creating dataset in file test2.h5 using variable H5File
HDF5-DIAG: Error detected in HDF5 (1.14.4-3) thread 1:
#000: …/…/hdf5-1.14.4-3/src/H5D.c line 187 in H5Dcreate2(): unable to synchronously create dataset
major: Dataset
minor: Unable to create file
#001: …/…/hdf5-1.14.4-3/src/H5D.c line 111 in H5D__create_api_common(): can’t set object access arguments
major: Dataset
minor: Can’t set value
#002: …/…/hdf5-1.14.4-3/src/H5VLint.c line 2602 in H5VL_setup_acc_args(): invalid location identifier
major: Invalid arguments to routine
minor: Inappropriate type
#003: …/…/hdf5-1.14.4-3/src/H5VLint.c line 1733 in H5VL_vol_object(): invalid identifier
major: Invalid arguments to routine
minor: Inappropriate type
terminate called after throwing an instance of ‘H5::FileIException’
Aborted (core dumped)
ifdef out test2.h5 to test using std::swap we get
./hdf_tests_simpler
Creating dataset in file test1.h5 using variable tmpH5File
Creating dataset in file test3.h5 using variable H5File
HDF5-DIAG: Error detected in HDF5 (1.14.4-3) thread 1:
#000: …/…/hdf5-1.14.4-3/src/H5D.c line 187 in H5Dcreate2(): unable to synchronously create dataset
major: Dataset
minor: Unable to create file
#001: …/…/hdf5-1.14.4-3/src/H5D.c line 111 in H5D__create_api_common(): can’t set object access arguments
major: Dataset
minor: Can’t set value
#002: …/…/hdf5-1.14.4-3/src/H5VLint.c line 2602 in H5VL_setup_acc_args(): invalid location identifier
major: Invalid arguments to routine
minor: Inappropriate type
#003: …/…/hdf5-1.14.4-3/src/H5VLint.c line 1733 in H5VL_vol_object(): invalid identifier
major: Invalid arguments to routine
minor: Inappropriate type
terminate called after throwing an instance of ‘H5::FileIException’
However, if I compile and link against 1.14.0 compiled without --enable-threadsafe this test passes:
./hdf_tests_simpler
Creating dataset in file test1.h5 using variable tmpH5File
Creating dataset in file test2.h5 using variable H5File
Creating dataset in file test3.h5 using variable H5File
How should I approach this?
I have a work around where I use:
H5::H5File tmpH5File(fNameExt, H5F_ACC_TRUNC); // Create the file so I can open it
tmpH5File.close();
myFileVariable.openFile(fNameExt, H5F_ACC_RDWR);
Is this a reasonable think to do?
William.