Are there any plans to support Lzf natively for HDF5?
http://h5py.alfven.org/lzf/ seems to already implement it but it would
be nice if we had it natively...
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
Mag,
The HDF Group doesn't have plans to support any new types of compression except GZIP and SZIP supported now, but we have been working on defining a process of registering new filters and making them available to the HDF5 users’ community. Compressions used by PyTables and H5Py (i.e., LZF) were our "guinea pigs" and have been registered already. More information is coming soon.
Francesc and Andrew,
Thank you for your patience 
Elena
On Jul 5, 2009, at 7:48 AM, Mag Gam wrote:
Are there any plans to support Lzf natively for HDF5?
http://h5py.alfven.org/lzf/ seems to already implement it but it would
be nice if we had it natively...----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
Thanks for the reply and keep up the great work!
On Mon, Jul 6, 2009 at 12:45 PM, Elena Pourmal<epourmal@hdfgroup.org> wrote:
Mag,
The HDF Group doesn't have plans to support any new types of compression
except GZIP and SZIP supported now, but we have been working on defining
a process of registering new filters and making them available to the HDF5
users’ community. Compressions used by PyTables and H5Py (i.e., LZF) were
our "guinea pigs" and have been registered already. More information is
coming soon.
Francesc and Andrew,
Thank you for your patience
ElenaOn Jul 5, 2009, at 7:48 AM, Mag Gam wrote:
Are there any plans to support Lzf natively for HDF5?
http://h5py.alfven.org/lzf/ seems to already implement it but it would
be nice if we had it natively...----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to
hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org._______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Good morning,
The HDF Group has been working on setting up the process of registering the third party filters to be used with the HDF5 library (to see the process description go to http://wiki.hdfgroup.org/Communite-HDF5-filters-policy\)
Please visit http://wiki.hdfgroup.org/ --> "Community Support for HDF5" (it is the first link on the Front Page) --> HDF5 Filters
to find more about the process and the third party filters LZO, BZIP2, LZF and BLOSC that have been registered with The HDF Group using the proposed process.
Comment, suggestions are more than welcome.
Thank you!
Elena
On Jul 6, 2009, at 11:45 AM, Elena Pourmal wrote:
Mag,
The HDF Group doesn't have plans to support any new types of compression except GZIP and SZIP supported now, but we have been working on defining a process of registering new filters and making them available to the HDF5 users’ community. Compressions used by PyTables and H5Py (i.e., LZF) were our "guinea pigs" and have been registered already. More information is coming soon.
Francesc and Andrew,
Thank you for your patience
Elena
On Jul 5, 2009, at 7:48 AM, Mag Gam wrote:
Are there any plans to support Lzf natively for HDF5?
http://h5py.alfven.org/lzf/ seems to already implement it but it would
be nice if we had it natively...----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org._______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Hi Elena,
I'll provide my comments, here, via email.
First, great idea. I like this (service) HDF5 group is trying to offer.
The policy discusses the identifier and what information is required to
'request' an identifier. Are there any requirements on the language its
written it (I assume it has to be C), the platorm(s) it is known to
compile/run on?
Finally, it would be good to establish a small set of benchmark datasets
that compression filters can be tested on. These (uncompressed) HDF5
files could be provided for download from the community filters page
along with the same files with datasets compressed by each of the
filters published there. And, for each filter performance (both time to
compress/decompress as well as amount of compression) of the filter on a
canonical cpu (some machine HDFgroup owns -- all thats important is that
performance results are all obtained from the same machine) be given
along with the link to the filter 'details'
Mark
On Thu, 2009-08-13 at 08:59, Elena Pourmal wrote:
Good morning,
The HDF Group has been working on setting up the process
of registering the third party filters to be used with the HDF5
library (to see the process description go to
http://*wiki.hdfgroup.org/Communite-HDF5-filters-policy)Please visit http://*wiki.hdfgroup.org/ --> "Community Support for
HDF5" (it is the first link on the Front Page) --> HDF5 Filters
to find more about the process and the third party filters LZO, BZIP2,
LZF and BLOSC that have been registered with The HDF Group using the
proposed process.Comment, suggestions are more than welcome.
Thank you!
Elena
On Jul 6, 2009, at 11:45 AM, Elena Pourmal wrote:
> Mag,
>
> The HDF Group doesn't have plans to support any new types of
> compression except GZIP and SZIP supported now, but we have been
> working on defining a process of registering new filters and making
> them available to the HDF5 users� community. Compressions used by
> PyTables and H5Py (i.e., LZF) were our "guinea pigs" and have
> been registered already. More information is coming soon.
>
> Francesc and Andrew,
>
> Thank you for your patience
>
> Elena
>
> On Jul 5, 2009, at 7:48 AM, Mag Gam wrote:
> > Are there any plans to support Lzf natively for HDF5?
> >
> > http://*h5py.alfven.org/lzf/ seems to already implement it but it
> > would
> > be nice if we had it natively...
> >
> > ----------------------------------------------------------------------
> > This mailing list is for HDF software users discussion.
> > To subscribe to this list, send a message to
> > hdf-forum-subscribe@hdfgroup.org.
> > To unsubscribe, send a message to
> > hdf-forum-unsubscribe@hdfgroup.org.
> >
> >
> >
>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> Hdf-forum@hdfgroup.org
> http://*mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org______________________________________________________________________
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://*mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
Mark C. Miller, Lawrence Livermore National Laboratory
email: mailto:miller86@llnl.gov
(M/T/W) (925)-423-5901 (!!LLNL BUSINESS ONLY!!)
(Th/F) (530)-753-8511 (!!LLNL BUSINESS ONLY!!)
Hi Elena,
One other thing it might be worth mentioning about a compression filter
is whether the compresser supports ONLY lossless compression or both
lossy and lossless compression. Sometimes lossy is useful. Most times
not.
On Thu, 2009-08-13 at 08:58, Mark Miller wrote:
Hi Elena,
I'll provide my comments, here, via email.
First, great idea. I like this (service) HDF5 group is trying to offer.
The policy discusses the identifier and what information is required to
'request' an identifier. Are there any requirements on the language its
written it (I assume it has to be C), the platorm(s) it is known to
compile/run on?Finally, it would be good to establish a small set of benchmark datasets
that compression filters can be tested on. These (uncompressed) HDF5
files could be provided for download from the community filters page
along with the same files with datasets compressed by each of the
filters published there. And, for each filter performance (both time to
compress/decompress as well as amount of compression) of the filter on a
canonical cpu (some machine HDFgroup owns -- all thats important is that
performance results are all obtained from the same machine) be given
along with the link to the filter 'details'Mark
On Thu, 2009-08-13 at 08:59, Elena Pourmal wrote:
> Good morning,
>
> The HDF Group has been working on setting up the process
> of registering the third party filters to be used with the HDF5
> library (to see the process description go to
> http://*wiki.hdfgroup.org/Communite-HDF5-filters-policy)
>
> Please visit http://*wiki.hdfgroup.org/ --> "Community Support for
> HDF5" (it is the first link on the Front Page) --> HDF5 Filters
> to find more about the process and the third party filters LZO, BZIP2,
> LZF and BLOSC that have been registered with The HDF Group using the
> proposed process.
>
> Comment, suggestions are more than welcome.
>
> Thank you!
>
> Elena
>
>
> On Jul 6, 2009, at 11:45 AM, Elena Pourmal wrote:
> > Mag,
> >
> > The HDF Group doesn't have plans to support any new types of
> > compression except GZIP and SZIP supported now, but we have been
> > working on defining a process of registering new filters and making
> > them available to the HDF5 users� community. Compressions used by
> > PyTables and H5Py (i.e., LZF) were our "guinea pigs" and have
> > been registered already. More information is coming soon.
> >
> > Francesc and Andrew,
> >
> > Thank you for your patience
> >
> > Elena
> >
> > On Jul 5, 2009, at 7:48 AM, Mag Gam wrote:
> > > Are there any plans to support Lzf natively for HDF5?
> > >
> > > http://*h5py.alfven.org/lzf/ seems to already implement it but it
> > > would
> > > be nice if we had it natively...
> > >
> > > ----------------------------------------------------------------------
> > > This mailing list is for HDF software users discussion.
> > > To subscribe to this list, send a message to
> > > hdf-forum-subscribe@hdfgroup.org.
> > > To unsubscribe, send a message to
> > > hdf-forum-unsubscribe@hdfgroup.org.
> > >
> > >
> > >
> >
> >
> > _______________________________________________
> > Hdf-forum is for HDF software users discussion.
> > Hdf-forum@hdfgroup.org
> > http://*mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>
>
>
> ______________________________________________________________________
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> Hdf-forum@hdfgroup.org
> http://*mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
Mark C. Miller, Lawrence Livermore National Laboratory
email: mailto:miller86@llnl.gov
(M/T/W) (925)-423-5901 (!!LLNL BUSINESS ONLY!!)
(Th/F) (530)-753-8511 (!!LLNL BUSINESS ONLY!!)
I apologize for the multiple emails.
There was a typo in the page name that was brought to my attention. It was fixed. Here is the new link to the policy description https://hdf5dev.pbworks.com/Community-HDF5-filters-policy
Elena
On Aug 13, 2009, at 10:59 AM, Elena Pourmal wrote:
Good morning,
The HDF Group has been working on setting up the process of registering the third party filters to be used with the HDF5 library (to see the process description go to http://wiki.hdfgroup.org/Communite-HDF5-filters-policy\)
Please visit http://wiki.hdfgroup.org/ --> "Community Support for HDF5" (it is the first link on the Front Page) --> HDF5 Filters
to find more about the process and the third party filters LZO, BZIP2, LZF and BLOSC that have been registered with The HDF Group using the proposed process.Comment, suggestions are more than welcome.
Thank you!
Elena
On Jul 6, 2009, at 11:45 AM, Elena Pourmal wrote:
Mag,
The HDF Group doesn't have plans to support any new types of compression except GZIP and SZIP supported now, but we have been working on defining a process of registering new filters and making them available to the HDF5 users’ community. Compressions used by PyTables and H5Py (i.e., LZF) were our "guinea pigs" and have been registered already. More information is coming soon.
Francesc and Andrew,
Thank you for your patience
Elena
On Jul 5, 2009, at 7:48 AM, Mag Gam wrote:
Are there any plans to support Lzf natively for HDF5?
http://h5py.alfven.org/lzf/ seems to already implement it but it would
be nice if we had it natively...----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org._______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
