[Help] Creating a dataset with a filter that is not registered

HDF5 Library
1

Hey!

I’m currently extending the Golang binding of HDF5 because of a special use case which I want to try out (you can see my code here). The goal is to introduce the H5DOwrite_chunk and H5DOread_chunk to the gonum package.

Technically they work, but to be useful I also have to set a filter on the dataset, because my data is arriving pre-compressed (hence the write chunk). But, when I set the filter of the dataset it complains that the filter is not registered, which it isn’t.

#013: H5Z.c line 1205 in H5Z_find(): required filter 32008 is not registered

However, I don’t have the implementation of the filter (bitshuffle + lz4 in my case), but I also don’t need it. My Golang application wont be the one reading the data, I just intend to write the chunk that is given to me, but it looks like I can’t create the dataset with the correct properties.

Is this the case? That I can’t create a dataset with a filter that I don’t have?

Thank you!

2

Efficient filtering is nontrivial, with credits intact, you may port H5CPP filtering pipeline to GO Language for direct chunk IO.
H5Zpipeline_basic.hpp
H5Zpipeline.hpp
H5Zall.hpp

Best wishes: steven varga

3

I think you should be able to get the shareable library for the filter from the HDF5 web site, and put it in a directory you point to with HDF5_PLUGIN_PATH.

Mark