H5Pset_deflate error on dataset property list

HDF5 Library
1

I’m trying to set gzip/ “deflate” compression on a dataset, with custom Cython bindings for HDF5.

# Create the dataset property list
dcpl = H5Pcopy(H5P_DATASET_CREATE)
H5Pset_layout(dcpl, H5D_CHUNKED)
H5Pset_chunk(dcpl, 2, chunk_size)
H5Pset_fill_value(dcpl, H5T_IEEE_F32LE, fill_value)
# Specify gzip compression on output dataset
if H5Pset_deflate(dcpl, 3) < 0:
    print('ERROR in setting gzip filter')
dset_id = H5Dcreate( # Using H5P_DEFAULT for lcpl_id, ..., dapl_id
    dest, dset_name.encode('UTF-8'), dtype, dspace,
    H5P_DEFAULT, dcpl, H5P_DEFAULT)
H5Dwrite(dset_id, dtype, H5S_ALL, H5S_ALL, H5P_DEFAULT, buff)
H5Dclose(dset_id)

However, the return value of H5Pset_deflate() is negative. I consulted these resources:

And I’ve confirmed that gzip compression is available in my HDF5 build. I’m wondering how I can further investigate this issue. I haven’t been able to find any resources online about what a negative return value here indicates. The Troubleshooting guide (above) only covers cases where compression was “not applied” or “not effective” but says nothing about the return value of these filter functions.

As I suspect that my build uses MPI, I also tried this again, building with a dataset transfer property list and the H5FD_MPIO_COLLECTIVE option. The build was successful but I’m still getting a negative return value from H5Pset_deflate():

dxpl = H5Pcreate(H5P_DATASET_XFER)
H5Pset_dxpl_mpio(dxpl, H5FD_MPIO_COLLECTIVE)
...
# Same as above
...
H5Dwrite(dset_id, dtype, H5S_ALL, H5S_ALL, dxpl, buff)
2

Could you please include the error stack here?
BTW, have you’ve read HDF5: Dataset Creation Properties?

3

One issue I caught is that I should have been using H5Pcreate() and instead of H5Pcopy() for the dataset creation property list. Fixing this, I was getting the errors:

HDF5-DIAG: Error detected in HDF5 (1.10.7) thread 0:
  #000: ../../../src/H5Dio.c line 335 in H5Dwrite(): can't write data
    major: Dataset
    minor: Write failed
  #001: ../../../src/H5Dio.c line 701 in H5D__write(): collective access for MPI-based driver only
    major: Dataset
    minor: Feature is unsupported
ERROR in writing HDF5 file

I’m building against OpenMPI, as I’ve been able to define MPI_Init(), and I am using H5Pset_fapl_mpio() to set up the file access for MPI, but something else is wrong.

If I give up on using MPI, i.e., get rid of the data transfer property list and H5Pset_dxpl_mpio(), then everything works fine.

5

Could you please look at the examples here and see if you can figure out something different in your code?