Does parallel HDF5 support SWMR?


#1

Hi,

I’m running test with SWMR. Does parallel HDF5 supports SWMR? For example, I run my simulation code with multiple MPI processes and each of these processes writes a block of a 2D global array. At each simulation step, a new 2D global array is generated and it needs to be appended to the previous one. So basically, the simulation output is a 3D array with an unlimited dimension that can grow during runtime. If parallel HDF5 supports SWMR, is there an example code I can look at?

Thanks,
L.W.


#2

This will not work with parallel HDF5. Using the serial library, I can think of at least one approach that might work well for you. HDF5 1.10 introduced a Single-writer multi-reader (SWMR) mode = to open a file.
Here are how parallel HDF5 works: https://docsbay.net/parallel-hdf5