C++ Thread unable to open file


#1

Hello,
I am trying to read from two different hdf5 files with two different threads (using std::thread) of a thread pool.
Each thread has its dedicated hdf5 file.
However it seems that sometimes the file cannot be opened: According to the visual studio debugger, the error below is raised when the file is opened. Other messages that vary are logged without the debugger.

Exception thrown at 0x00007FFCBBE5EFE2 (hdf5.dll) in HDFMultiThreadTest.exe: 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF

Below is the relevant portion of code that opens the file to read the data. The error does not appear when the thread pool only uses 1 thread. I do not understand why there are thead related issues as each thread has its own hdf5 file.

Is there an element that I missed ? Do you know where this error could come from ?

I am using hdf5 1.12.1 on windows. I can provide the full toy example with the Thread Pool if necessary.

Thanks in advance!

double* read_data(int i,int id) {
    using namespace H5;
    double* points;
    // Dataset names are numbers (as strings) in the range 0 to 0 (checked with HDFView)
    std::string dataset_name = std::to_string(i);
    // Format name depending on thread id
    const char* fmt = "benchmark%d.hdf5";
    int sz = std::snprintf(nullptr, 0, fmt, id);
    std::vector<char> buf(sz + 1); // note +1 for null terminator
    std::snprintf(&buf[0], buf.size(), fmt, id);
    std::string NAME_DATA_FILE(buf.begin(), buf.end());
    // Create filename of file to open
    const H5std_string FILE_NAME_BENCHMARK(NAME_DATA_FILE);
    H5std_string DATASET_NAME(dataset_name);
    // Read data
    try {
        H5File  file(FILE_NAME_BENCHMARK, H5F_ACC_RDONLY); // <- Debugger error here
        DataSet dataset_points = file.openDataSet(DATASET_NAME);

        DataSpace dataspace_points = dataset_points.getSpace();
        hsize_t dims_points[1];
        hsize_t rank_points = dataspace_points.getSimpleExtentDims(dims_points);

        points = new double[dims_points[0]];
        dataset_points.read(points, PredType::NATIVE_DOUBLE);
    }
    catch (Exception error) {
        error.printErrorStack();
        exit(1);
    }
    return points;
}

#2

Even if you are accessing two different files, the HDF5 API is not thread safe by default. You can enable a thread safety feature using the option, --enable-threadsafe.

https://docs.hdfgroup.org/hdf5/v1_12/_m_t.html


#3

Thank you for your answer
I will try to recompile the library with the option
Just one question: I see that the documentation link uses pthread when the library is compiled in thread safe mode. Is pthread also usable on windows ? (I rarely program with multithreading and more often on linux)


#4

Under CMake, Windows only uses Win32 threads when thread-safety is selected. This could be changed (PRs accepted!) but we’d have to add testing for that and it’s not a high priority given the low number of people who would use Pthreads on Windows.


#5

Which HDF5 C++ bindings are you using? I believe H5File & friends are not threadsafe even with a threadsafe C-build. G.


#6

Thanks for your answers
Yes I am using H5File…
Is there an other way to go?


#7

You should be using the C API from C++ if you want thread-safety. The global lock we use to provide thread-safety is only enabled at the C API level so the C++ components are not thread-safe.


#8

Ok I will try to setup the C API.