Reading Image data from single file and writing to two h5 files using threads . Getting run time error doing this . What could be the reason ?
Are you using thread-safe HDF5?
You may check the libhdf5.settings file (it will be in the lib directory under the installation point or in the src directory), Features section, Threadsafety line (yes/no).
Elena
I am working on windows platform and I am using static libraries
Did you build the libraries yourself, or did you download them from somewhere?
Which version are you using? There is also a function H5is_library_threadsafe to test if you are dealing w/ a threadsafe build.
G.
Yes , I am building libraries using CMake and the version is 1.12.2 , I am facing issues while enabling threadsafety .
I think you have to be a little more specific so that we can help you. What is the error message / the error behavior? Can you give us a minimal reproducible example?
G.
Basically reading from a image file and trying to store those image data in two different files in h5 format at two different locations . Using two file handles but it is crashing and i am using h5Image24bit(,) to write at respective files .
Error : Read violation sometimes .
write violation
stack overflow sometimes
heap issue somettimes
Unfortunately, HDF5 HL libraries are not thread-safe. You will need API locking in your application to avoid HDF5 data structures corruption.
If you need to write images according to HDF5 Image specification, just use HDF5 C APIs.
On Windows, thread-safety needs a dll build as we have to put the thread-local storage teardown in DllMain. If you use a static HDF5 library you’ll leak resources. CMake should warn about this is you try to build a static thread-safe Windows HDF5 library.
1 Like
I have used API locking as well , But still it is crashing . What calls could be used ? , As of now I am using libhdf5.lib and libhdf5_hl.lib
Please provide an example to reproduce the problem. It may be a bug in HDF5, it may be an issue with your application.
Elena
// Writing a sample code which is causing issue
Creating h5 file and a group to store data
//Inside thread function using critical section
///// This part of code down is being used by threads
Enter Critical section
Using write Image Api call ; //
Critical section exit ()
interesting information
Sorry, C code please.
A single instance with ten threads is writing images properly and no issues there . When two instances which tends to create 2 files where 10 threads write to one file and other ten threads write to other file is causing issue .
Without the code it is still hard to say what is going on. Please create a reproducer and post it here.
Have you tried to use just one thread per file when you have two files, and say 24 threads and one file? Just curious…but this may help you to create a reproducer.
Bottom line, your application corrupts HDF5 data structures, but it is impossible to tell without seeing the code if this is a bug in the thread-safe HDF5 library or in the application.
fid =H5FCreate(file, H5F_ACC_TRUNC,H5P_DEFAULT,H5P_DEFAULT);
gid = HFGCreate2(fid , "name ", H5P_DEFAULT,H5P_DEFAULT,H5P_DEFAULT);
thread code
Ccomsinglelock locksec(hdfwrite);
H5Imake_image_24bit(gid ,“stringname”,width , height ,INTERLACE_PIXEL,buffer_data);
locksec.Unlock();
Working with latest version of HDF5 1.12.2 and built code with threadsafe .
H5is_threadsafe is returning positive number but its boolean parameter is changing val to false … What could be the reason ?
Can you check the libhdf5.settingsfile (it should be under the lib directory of the installation point) and check that thread-safety was enabled? You should see something like this
Threadsafety: yes (recursive RW locks: no)
The pseudo code you provided should work.
Actually this code will be executed twice like two instances of this code … Two h5 files created , because of which I am facing issues