When I create a filtered chunked dataset, I noticed that the dataset gets filtered twice: when the empty dataset gets created, and when it gets written.
For efficiency, I would like to remove the first filter call (dataset creation) and if possible, avoid the initial disk allocation with empty data.
I’ve tried options such as H5D_ALLOC_TIME_INCR, but without success so far.
I’m using the C++ wrappers of HDF5 1.10.3 with collective calls.
The file is being created collectively. I create the dataset like this:
ds_creatplist.setFilter( (h5Z_filter_t)32768, H5Z_FLAG_OPTIONAL);
createDataSet(“ds”,H5::PredType::NATIVE_INT, dataspace, ds_creatplist);
As far as I can see, DSetCreatPropList has no options related to a sequential or collective call.
Do you mean create the dataset and the file sequentially on rank 0, and reopen them collectively?
No, if the file is opened via an MPI communicator, all processes in that communicator need to “witness” the dataset creation (or attribute or group creation, etc.). That’s part of the parallel HDF5 etiquette.
Unfortunately, the openDataSet calls the filter to compress data. I don’t know how to avoid this call.
A possible work-around: test for a zero-filled buffer and write a “magic number” instead of calling an expensive compression algorithm. This way, the filter is called twice, but relatively fast the first time.
I believe that there is currently no way to get around the first call to the filter.
If you look here https://support.hdfgroup.org/HDF5/doc_resource/H5Fill_Values.html under section VI., the first part of the first bullet point mentions that certain VFDs (such as the MPI-I/O one) require that the space for the dataset be allocated upon creation time. From what I understand, it seems that in your case the library is realizing upon dataset open that the space wasn’t allocated. At that point, the MPI-I/O driver forces the space to be allocated, the side effect of which will be that the chunks in the dataset get filtered during allocation time. In this manner, later reads of those chunks will see filtered data as expected, instead of unfiltered data.
You may be able to optimize by using your suggestion of testing the buffer, but in that case, if fill values weren’t written, you may not be guaranteed that the data buffer for the chunk is actually zero-filled.