Hello, we would like to to be able to obtain a pointer to the filter functions to postpone the execution of some filters, detaching them from H5Dread_chunk. Is there an official way of doing this through the hdf5 public interface?
Accessing the function pointers is currently possible by calling H5Z_find defined in H5Zprivate.h. The function is exported by the library and works as expected, but it is marked as “internal” to the library. Assuming that the pointer to the filter (or at least to the filter function) does not change (at least as long as no filters are registered nor unregistered), is this a safe and sane thing to do?
In general, is it allowed to use functions defined in the private headers of hdf5 and what are the argument against it? They are “only” not documented, not officially supported and not guaranteed to exist in a future version, or is there also any concrete argument against it with a definite release (i.e. 10.4)?
Igor, how are you? When you use the direct chunk I/O functions H5D[read,write]_chunk, then that’s a bit like saying, “I want to read/write the bytes on those chunks and bypass the HDF5 library as much as possible. I’m taking control.” And there are several use cases when that’s the right thing to do. At the same time you seem to suggest, that you want to come back to the library and use it’s built-in filter (compression) routines to interpret the bytes you’ve obtained via direct chunk I/O. While that’s possible, I can’t see a good reason why you would do that. Leaving aside the chunk cache, performance is going to be as good (or bad) as if you had followed the usual channels (chunked I/O pipeline). Tools like H5CPP have demonstrated that you can get better performance by using your custom pipeline, but that’s precisely because you DON’T use H5Z internals.
The risk with using any HDF5 internals is completely yours. No reason to ask for permission. We won’t send the sheriff. They are private/internals precisely because we don’t make any guarantees about their form or future.
If you can describe your goal/use case a bit, maybe someone can come up with a more helpful response.
