write simultaneously to different files

HDF5 Library
1

I want to create multiple threads to simultaneously write content to different HDF files, but I’ve encountered some issues. Below is the error message,
HDF5-DIAG: Error detected in HDF5 (1.13.2) thread 0:
#000: H5Iint.c line 1325 in H5I_inc_ref(): can’t locate ID
major: Object ID
minor: Unable to find ID information (already closed?)
#001: H5Iint.c line 1325 in H5I_inc_ref(): can’t locate ID
major: Object ID
minor: Unable to find ID information (already closed?)
HDF5-DIAG: Error detected in HDF5 (1.13.2) thread 0:
#000: (null) line 1325 in (null)()
major: Object ID
minor: Unable to find ID information (already closed?)
#001: H5Iint.c line 534 in H5I__register(): invalid type
major: Object ID
minor: Unable to find ID group information
HDF5-DIAG: Error detected in HDF5 (1.13.2) thread 0:
HDF5-DIAG: Error detected in HDF5 (1.13.2) #000: H5S.c line 1395 in H5Screate_simple(): unable to register dataspace ID
major: Object ID
minor: Unable to register new ID
#001: H5Iint.c line 590 in H5I_register(): unable to register object
major: Object ID
minor: Unable to register new ID
#002: H5Iint.c line 534 in H5I__register(): invalid type
major: Object ID
minor: Unable to find ID group information
HDF5-DIAG: Error detected in HDF5 (1.13.2) thread 0:
#000: H5D.c line 186 in H5Dcreate2(): unable to synchronously create dataset
major: Dataset
minor: Unable to create file
#001: H5D.c line 110 in H5D__create_api_common(): can’t set object access arguments
major: Dataset
minor: Can’t set value
#002: H5VLint.c line 2669 in H5VL_setup_acc_args(): invalid location identifier
major: Invalid arguments to routine
minor: Inappropriate type
#003: H5VLint.c line 1787 in H5VL_vol_object(): invalid identifier type to function
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.13.2) thread 0:
#000: H5D.c line 1227 in H5Dwrite(): can’t synchronously write data
major: Dataset
minor: Write failed
#001: H5D.c line 1163 in H5D__write_api_common(): dset_id is not a dataset ID
major: Invalid arguments to routine
thread 0:
#000: (null) line 110 in (null)()
major: Object ID
minor: Unable to register new ID
#000: H5D.c line 1227 in H5Dwrite(): can’t synchronously write data
major: Dataset
minor: Write failed
#001: H5D.c line 1163 in H5D__write_api_common(): dset_id is not a dataset ID
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.13.2) thread 0:
#000: H5D.c line 186 in H5Dcreate2(): unable to synchronously create dataset
major: Dataset
minor: Unable to create file
#001: H5D.c line 110 in H5D__create_api_common(): can’t set object access arguments
major: Dataset
minor: Can’t set value
#002: H5VLint.c line 2669 in H5VL_setup_acc_args(): invalid location identifier
major: Invalid arguments to routine
minor: Inappropriate type
#003: H5VLint.c line 1787 in H5VL_vol_object(): invalid identifier type to function
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.13.2) thread 0:
#000: H5D.c line 1227 in H5Dwrite(): can’t synchronously write data
major: Dataset
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.13.2) thread 0:
#000: H5D.c line 474 in H5Dclose(): not a dataset ID
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.13.2) thread 0:
#000: H5S.c line 451 in H5Sclose(): not a dataspace
major: Invalid arguments to routine
minor: Inappropriate type
minor: Write failed
#000: H5F.c line 1055 in H5Fclose(): not a file ID
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.13.2) thread 0:
#000: H5D.c line 474 in H5Dclose(): not a dataset ID
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.13.2) thread 0:
#000: H5S.c line 451 in H5Sclose(): not a dataspace
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.13.2) thread 0:
#000: H5F.c line 1055 in H5Fclose(): not a file ID
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.13.2) thread 0:
#000: H5F.c line 1055 in H5Fclose(): not a file ID
major: Invalid arguments to routine
minor: Inappropriate type
HDF5: infinite loop closing library
L,T_top,P,P,Z,FD,VL,VL,PL,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E

along with an example of the code I’m trying to implement:
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <hdf5.h>

#define NUM_THREADS 3
const char* file_names[NUM_THREADS] = {“file1.h5”, “file2.h5”, “file3.h5”};

void* write_to_hdf5(void* arg) {
int thread_id = ((int)arg);

hid_t file_id = H5Fcreate(file_names[thread_id], H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT);

hsize_t dims[2] = {2, 2};
hid_t dataspace = H5Screate_simple(2, dims, NULL);
hid_t dataset = H5Dcreate2(file_id, "dataset", H5T_NATIVE_INT, dataspace, H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);

int data[2][2] = {{1, 2}, {3, 4}};

H5Dwrite(dataset, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT, data);

H5Dclose(dataset);
H5Sclose(dataspace);
H5Fclose(file_id);

pthread_exit(NULL);

}

int main() {
pthread_t threads[NUM_THREADS];
int thread_ids[NUM_THREADS];

for (int i = 0; i < NUM_THREADS; i++) {
    thread_ids[i] = i;
    int result = pthread_create(&threads[i], NULL, write_to_hdf5, (void*)&thread_ids[i]);
    if (result) {
        fprintf(stderr, "Error creating thread %d: %d\n", i, result);
        exit(-1);
    }
}

for (int i = 0; i < NUM_THREADS; i++) {
    pthread_join(threads[i], NULL);
}

return 0;

}

1 Like
2

Hello @zhengweibin,

Maybe a silly question. Did you make use of thread-safe version of the HDF5 library? If I remember correctly, the default version of HDF5 is not thread-safe.

Best regards,
Jan-Willem

1 Like
3

Yes, i did recompile with --enable-thread-safe option, and then make && make install before run the codes

4

You are correct. I forgot to add the prefix when recompiling, so it didn’t overwrite the HDF5 installation I had before. As a result, when compiling the program, I was still using the original non-threadsafe version. Thank you for your response.

5

You are welcome.

In the past, I got similar errors when I used a non-threadsafe HDF5 version in threaded application.