How should I extend datasets in parallel HDF5?

HDF5 Library
1

Dear HDF5 experts…

I’m writing a data output routine of radiation transport simulation code using HDF5 library. In the code, a primary particle is initiated and transported through matters, and the primary particle as well as the resultant secondary radiations induced by the primary are followed. A primary and the associated secondary radiations are called a history.

The simulation is repeated over millions of histories, and therefore it is done in parallel with MPI.
A history is processed within a single PE. The data obtained in a history (the length is fixed) is recorded when the history ended; the length of a history varies one by one, and so does the timing of data writes with PEs.
The number of histories is not fixed, and the data file should be extendable.

How should I extend a dataset in the parallel environment?

According to the description of H5Dextend and H5Dset_extent of the Collective Calling Requirements in Parallel HDF5 Applications page,

All processes must participate only if the number of chunks in the dataset actually changes.
All processes must use the same dataspace dimensions.

In my case, the timing of the end of a history in a PE (and therefore the extension dataset followed by data write) is asynchronous to other PEs.
How can I make all process participate the extension of a dataset?

Is there a parallel version of h5_extend.f90 or something like that?

Thanks in advance.
Kazuyoshi

2

All processes that collectively opened the HDF5 file need to collectively call H5Dset_extent() with the same parameters. It’s up to you to ensure that your processes coordinate so they can all do this correctly.

While you could technically call H5Dset_extent() with only one process if the number of chunks doesn’t change, this isn’t a good idea since your processes will then have differing views regarding the extent of the dataset. For example if you extend on process 0 and read from the new area on process 1 it will fail because process 1 thinks the dataset still has the old extent. In addition, only one process writes the extent to the file, and, depending on y our configuration, it might not be obvious which one.

1 Like