I’m very new to the HDF5 library and I’d like to see whether H5 is a good file format for a project I have. I’m interested in writing multiple h5 files at once with different threads so each thread writes a different file, but I’m having some trouble testing this out. Currently I’m just trying to add threading to one of the H5D example files given with HDF5Examples, but I get this error:
#000: C:\autotest\hdf5114-StdRelease-dist-10vs17\build\hdfsrc\src\H5D.c line 1390 in H5Dwrite(): can’t synchronously write data
Through some other discussions on this forum it seems like what I want to do should be possible (Reading and Writing Multiple Files Using Multiple Threads C++ - #3 by gheber), so I want to confirm - is it possible to synchronously write multiple h5 files with multiple threads? I’m also currently trying to do threading with using win32 - is there a better way to use threading with h5 in a windows environment?
I’ve also attached the file I’m currently running in Visual Studio 2022 if anyone wants to take a look, though it’s very messy and likely wrong in various places - I’m not too familiar with C.
Assuming you do, this will give you the illusion of multiple threads writing to different files or the same file. I’m saying illusion because only one thread at a time can execute code in the HDF5 library. In other words, these threads will not be executing library code concurrently.
I don’t think I built with --enable-threadsafe - how would I use H5is_library_threadsafe? According to what you said, even with the threadsafe build the threads are NOT actually operating concurrently, just appearing to? But there still shouldn’t be a big issue with writing multiple files with multiple threads?
Building with --enable-threadsafe is necessary to have a threadsafe library. The argument to H5is_library_threadsafe is hbool_t* is_ts. If it’s 0, that means you are dealing with a non-threadsafe library build.
By ‘operating concurrently,’ I mean potentially ‘executing library code simultaneously.’ Even with thread safety enabled, this is NOT happening. Your multithreaded program will behave correctly, but despite appearances, no two (or more) threads will simultaneously execute H5Dwrite code on different cores.
As I said, your multithreaded program will behave correctly (no races, no corruption, etc.) if you use a multithreaded library build.
