Deflate filter is not available

HDF5 Library
1

Hi there! I’m new to HDF5. I’m trying to use h5repack, but I get a lot of warnings saying
Warning: dataset <a_dataset> cannot be read, deflate filter is not available

This page says that ZLIB is the deflate filter. If I understood correctly, the HDF group portal also says that ZLIB (and SZIP) already come within the pre-built binaries, which was the way I installed HDF5 at an Ubuntu distribution in WSL.

Is there something I understood incorrectly? How can I use the deflate filter in h5repack?
Thank you!

2

First, which version and from where did you you install hdf5. There may be issues with a certain version/download, which we may need to investigate.

1 Like
3

Thanks for replying!

Version: 1.14.2
Installed from: Index of /ftp/HDF5/releases/hdf5-1.14/hdf5-1.14.2/bin/unix (hdfgroup.org)
File used: hdf5-1.14.2-Std-ubuntu2204_64.tar.gz

4

Yes it should be there, you could check the lib/libhdf5.settings file and around line 80 there should be a line like:
I/O filters (external): DEFLATE DECODE ENCODE

1 Like
5

That’s strange. When I execute h5repack --enable-error-stack -l "/GROUP/ENTITY/Data:CHUNK=1024x1x1" "input_file" "outputs/test", the following error is printed:

Warning: dataset </GROUP/ENTITY/Data> cannot be read, deflate filter is not available
6

Does h5dump show the same failure?

1 Like
7

I’ve tried h5dump -d /GROUP/ENTITY/Data input_file and the output was:

HDF5 "input_file" {
DATASET "/GROUP/ENTITY/Data" {
   DATATYPE  H5T_IEEE_F64LE
   DATASPACE  SIMPLE { ( 25097, 162, 89 ) / ( H5S_UNLIMITED, 162, 89 ) }
   DATA {h5dump error: unable to print data

   }
}
}
8

One more info: h5repack works if installed in another way — through apt-get —, but the version installed is 1.10.8 instead of 1.14.2, which is the one I’d like to use if possible (because generally it’s more secure and efficient to use updated software).

9

Could you try the binary from the Github Releases page: Release HDF5 release 1.14.2 · HDFGroup/hdf5 · GitHub

1 Like
10

It worked like a charm. Thank you very much for all your replies!!
Would you mind taking a look at the post I’ve written a few minutes ago, please? Thanks!

11

I will leave that post to someone more experienced in that issue.

1 Like
12

Ok, thank you anyway!

13

@byrn hi there, again! Actually, I need to understand better why the Github version worked and the one on the ftp didn’t, please. I thought all HDF5 1.14.2 binaries/sources would come with GZIP/SHUFFLE filters. Could you clarify this, please?

14

We have updated the binaries on the download page. There was a configuration error in our binary build script - not anything in the source or the library. We fixed the issue and everything built properly. We also had to regenerate the HDFView binaries as well.

1 Like
15

Thank you very much for your reply! That’s great news that you’ve fixed that! Can you just confirm this to me, then, please: any install of HDF5 1.14 will have GZIP/SHUFFLE filters, is that correct?

16

The binaries that we provide on the download page should have zlib and szip support builtin. Other compression filters are provided as plugins, with some subset provided in the downloaded binaries.
The releases on Github also should have zlib and szip support builtin - the plugins are not currently in the Github released binaries.

1 Like
17

Ok, thank you very much!