Hello HDF5,
I am using HDF5 since years now and would like to extend it with a custom compression filter now.
HDF5 supports dynamically loaded filters since 1.8.11 (http://www.hdfgroup.org/HDF5/doc/Advanced/DynamicallyLoadedFilters/HDF5DynamicallyLoadedFilters.pdf)
and the BZIP2 example is easy to understand and works fine for me (https://svn.hdfgroup.uiuc.edu/hdf5_plugins/trunk/BZIP2/src/H5Zbzip2.c)
--- Status ---
I read (http://www.hdfgroup.org/HDF5/doc/RM/RM_H5Z.html#Compression-Register), that for compressing a dataset
1) 'can_apply' function of filter is called once for each dataset before its creation
2) 'set_local' function of filter is called once for each chunk before 'filter' call
3) 'filter' function of filter is called once for each chunk to compress
'can_apply' and 'set_local' are optional.
--- Question ---
My compression filter will use an algorithm, which needs to know the dataspace describing the chunk to be compressed.
This information has to be known by the 'filter' function but is only available in 'set_local' function.
How am I supposed to pass values from 'set_local' to the 'filter' function?
Any hint would be a great help - Thanks.
Best
Jens Henrik
···
--
Dipl.-Ing. Jens Henrik Göbbert
IT Center - Computational Science & Engineering
Computer Science Department - Virtual Reality Group
Jülich Aachen Research Alliance - JARA-HPC
IT Center
RWTH Aachen University
Seffenter Weg 23
52074 Aachen, Germany
Phone: +49 241 80-24381
goebbert@vr.rwth-aachen.de<mailto:goebbert@vr.rwth-aachen.de>
www.vr.rwth-aachen.de<http://www.vr.rwth-aachen.de>
http://www.jara.org<http://www.jara.org/>
Hello HDF5,
I found good examples to solve the problem here
https://github.com/kiyo-masui/bitshuffle
http://www.hdfgroup.org/ftp/HDF5/releases/hdf5-1.6/hdf5-1.6.7/src/unpacked/src/H5Zshuffle.c
Thanks and Bye
Jens
···
From: Göbbert, Jens Henrik
Sent: Tuesday, November 18, 2014 5:26 PM
To: 'hdf-forum@lists.hdfgroup.org'
Subject: HDF5 custom filter -- missing dataspace id in filter function
Hello HDF5,
I am using HDF5 since years now and would like to extend it with a custom compression filter now.
HDF5 supports dynamically loaded filters since 1.8.11 (http://www.hdfgroup.org/HDF5/doc/Advanced/DynamicallyLoadedFilters/HDF5DynamicallyLoadedFilters.pdf)
and the BZIP2 example is easy to understand and works fine for me (https://svn.hdfgroup.uiuc.edu/hdf5_plugins/trunk/BZIP2/src/H5Zbzip2.c)
--- Status ---
I read (http://www.hdfgroup.org/HDF5/doc/RM/RM_H5Z.html#Compression-Register), that for compressing a dataset
1) 'can_apply' function of filter is called once for each dataset before its creation
2) 'set_local' function of filter is called once for each chunk before 'filter' call
3) 'filter' function of filter is called once for each chunk to compress
'can_apply' and 'set_local' are optional.
--- Question ---
My compression filter will use an algorithm, which needs to know the dataspace describing the chunk to be compressed.
This information has to be known by the 'filter' function but is only available in 'set_local' function.
How am I supposed to pass values from 'set_local' to the 'filter' function?
Any hint would be a great help - Thanks.
Best
Jens Henrik
--
Dipl.-Ing. Jens Henrik Göbbert
IT Center - Computational Science & Engineering
Computer Science Department - Virtual Reality Group
Jülich Aachen Research Alliance - JARA-HPC
IT Center
RWTH Aachen University
Seffenter Weg 23
52074 Aachen, Germany
Phone: +49 241 80-24381
goebbert@vr.rwth-aachen.de<mailto:goebbert@vr.rwth-aachen.de>
www.vr.rwth-aachen.de<http://www.vr.rwth-aachen.de>
http://www.jara.org<http://www.jara.org/>
