Custom filter registration to HDF filter

Hdf-forum archives
1

Hello Everyone,

I am trying to register my custom compression filter with HDF library and
create data sets using my compression technique.

I am following the gzip example from the following link as reference:

http://www.hdfgroup.org/HDF5/faq/compression.html

Can some one point/share a different example (probably simple), which
explains the registering of filter, creation of dataset and reading of
dataset.

···

--
With Best Regards,
Sravan Kumar

2

If you don't know how to read/write to/from a dataset use the HDF5 tutorial
[1]. Once this is clear move to filters.

Here [2] some more documentation about filters. Take care that in the
filter code there is an error: H5Z_REVERSE is H5Z_FLAG_REVERSE and that is
part of an old documentation.
I also suggest to read this document here[3]. It is really well done.

When you start with filter you have to keep in mind that:
- The dataset has to be chunked.
- Each chunk selected, by the hyperslab selection, are elaborated by
filters as one and independent piece of data.

Steps needed to create a dataset with filters are:
- Register the filter, H5Zregister();
- Create a dataset creation property list (dcpl), H5Pcreate();
- Set chunk to the dcpl, H5Pset_chunk();
- Set filter to the dcpl. H5Pset_filter();
- Create the dataset with the creation property list.

Finally, when you work with filters you could find useful to have HDF5
debugging option enabled.

Kind Regards,
Nicola.

[1] http://www.hdfgroup.org/HDF5/Tutor/rdwt.html
[2] http://www.hdfgroup.org/HDF5/doc/H5.user/Filters.html
[3] http://www.hdfgroup.org/HDF5/doc/Advanced/DataFlow_H5Dread/DataFlow
_H5Dread.pdf
[4] http://www.hdfgroup.org/HDF5/doc/H5.user/Debugging.html

···

On 14 May 2014 11:25, sravan kumar <withusravan@gmail.com> wrote:

Hello Everyone,

I am trying to register my custom compression filter with HDF library and
create data sets using my compression technique.

I am following the gzip example from the following link as reference:

http://www.hdfgroup.org/HDF5/faq/compression.html

Can some one point/share a different example (probably simple), which
explains the registering of filter, creation of dataset and reading of
dataset.

--
With Best Regards,
Sravan Kumar

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org

http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

--
Nicola Cadenelli
Jülich Supercomputing Centre
Wilhelm-Johnen-Straße
52425 Jülich, Germany
Phone (IT) +39 334 6550271
Office (DE) +49 2461 6196 475
Skype: nicolacdnll

3

I am sure this DOES NOT satisfy your criterion of 'simple', but if you download the 'Legacy Licensed' Silo 4.9.1 tarball from here,

https://wci.llnl.gov/codes/silo/downloads.html

untar it and go to silo-4.9.1/src/hdf5_drv and look at silo_hdf5.c and search for 'hzip' OR 'fpzip', you'll see how Silo does it.

Mark

···

From: sravan kumar <withusravan@gmail.com<mailto:withusravan@gmail.com>>
Reply-To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Date: Wednesday, May 14, 2014 2:25 AM
To: "hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>" <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: [Hdf-forum] Custom filter registration to HDF filter

Hello Everyone,

I am trying to register my custom compression filter with HDF library and create data sets using my compression technique.

I am following the gzip example from the following link as reference:

http://www.hdfgroup.org/HDF5/faq/compression.html

Can some one point/share a different example (probably simple), which explains the registering of filter, creation of dataset and reading of dataset.

--
With Best Regards,
Sravan Kumar

4

Hello Everyone,

Thanks for the good suggestions towards the knowledge base of HDF. I am
aware of most of the stuff, but this thread also helped me to know more
about HDF.

Coming to the main point, attached is small piece of code, in which i
register a custom filter and then trying to write a dataset using the
custom filter. For some reason i get a segmentation fault after the
application i.e, DWrite completes. Any help in this regard is welcome.

Thanks
Sravan

DWrite.cpp (2.4 KB)

RLE.cpp (1.29 KB)

RLE.h (806 Bytes)

RLE_Filter.cpp (2.13 KB)

···

On Wed, May 14, 2014 at 9:48 PM, Miller, Mark C. <miller86@llnl.gov> wrote:

I am sure this DOES NOT satisfy your criterion of 'simple', but if you
download the 'Legacy Licensed' Silo 4.9.1 tarball from here,

https://wci.llnl.gov/codes/silo/downloads.html

untar it and go to silo-4.9.1/src/hdf5_drv and look at silo_hdf5.c and
search for 'hzip' OR 'fpzip', you'll see how Silo does it.

Mark

  From: sravan kumar <withusravan@gmail.com>
Reply-To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
Date: Wednesday, May 14, 2014 2:25 AM
To: "hdf-forum@lists.hdfgroup.org" <hdf-forum@lists.hdfgroup.org>
Subject: [Hdf-forum] Custom filter registration to HDF filter

   Hello Everyone,

I am trying to register my custom compression filter with HDF library and
create data sets using my compression technique.

I am following the gzip example from the following link as reference:

http://www.hdfgroup.org/HDF5/faq/compression.html

Can some one point/share a different example (probably simple), which
explains the registering of filter, creation of dataset and reading of
dataset.

--
With Best Regards,
Sravan Kumar

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org

http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

--
With Best Regards,

Y. Sravan Kumar

http://meetsravan.blogspot.com/