Alt. compression filter implementation

paramon · November 15, 2016, 1:20pm

Hello!

I'd like to use "bitshuffle" preconditioning filter for HDF5 datasets. Luckily, the page

points me to Filter ID 32008:

However, there are technical limitations that restrict me from usage of above plugin. I'm going to implement something simpler, probably not even *fully* compatible on cd_nelmts/cd_values.
In this case, should I still use ID 32008 for my filter?
Or, I'd rather contact
https://support.hdfgroup.org/services/support.html
and try register another ID?

Best wishes,
Andrey Paramonov

···

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

miller86 · November 15, 2016, 4:46pm

Hi Andrey,

Here is my take on filter ids

You should use a different filter id.

That said, you have two options in so doing. Use a non-registered id with value below 32000 that *only*you* care about.

Or, try to get a new registered filter id.

Do you anticipate many other people/organizations using your filter and sharing data that was filtered with it?

If so, yes, it would be very important to go the "registered filter id" route. Otherwise, its less important.

Mark

···

--
Mark C. Miller, LLNL

From: Hdf-forum <hdf-forum-bounces@lists.hdfgroup.org<mailto:hdf-forum-bounces@lists.hdfgroup.org>> on behalf of Андрей Парамонов <paramon@acdlabs.ru<mailto:paramon@acdlabs.ru>>
Reply-To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Date: Tuesday, November 15, 2016 at 5:20 AM
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: [Hdf-forum] Alt. compression filter implementation

Hello!

I'd like to use "bitshuffle" preconditioning filter for HDF5 datasets.
Luckily, the page
https://support.hdfgroup.org/services/contributions.html#filters
points me to Filter ID 32008:

However, there are technical limitations that restrict me from usage of
above plugin. I'm going to implement something simpler, probably not
even *fully* compatible on cd_nelmts/cd_values.
In this case, should I still use ID 32008 for my filter?
Or, I'd rather contact
https://support.hdfgroup.org/services/support.html
and try register another ID?

Best wishes,
Andrey Paramonov

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

paramon · November 16, 2016, 12:27pm

15.11.2016 19:46, Miller, Mark C. пишет:

Hi Andrey,

Here is my take on filter ids

You should use a different filter id.

That said, you have two options in so doing. Use a non-registered id
with value below 32000 that *only*you* care about.

Or, try to get a new registered filter id.

Do you anticipate many other people/organizations using your filter and
sharing data that was filtered with it?

If so, yes, it would be very important to go the "registered filter id"
route. Otherwise, its less important.

Hello!

Thank you for your reply.
The files will be created by other people and probably passed around, so I'd like to ensure the numbers can still be readable.

Best wishes,
Andrey Paramonov

···

From: Hdf-forum <hdf-forum-bounces@lists.hdfgroup.org
<mailto:hdf-forum-bounces@lists.hdfgroup.org>> on behalf of Андрей
Парамонов <paramon@acdlabs.ru <mailto:paramon@acdlabs.ru>>
Reply-To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org
<mailto:hdf-forum@lists.hdfgroup.org>>
Date: Tuesday, November 15, 2016 at 5:20 AM
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org
<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: [Hdf-forum] Alt. compression filter implementation

    Hello!

    I'd like to use "bitshuffle" preconditioning filter for HDF5 datasets.
    Luckily, the page
    https://support.hdfgroup.org/services/contributions.html#filters
    points me to Filter ID 32008:
    https://github.com/kiyo-masui/bitshuffle
    However, there are technical limitations that restrict me from usage of
    above plugin. I'm going to implement something simpler, probably not
    even *fully* compatible on cd_nelmts/cd_values.
    In this case, should I still use ID 32008 for my filter?
    Or, I'd rather contact
    https://support.hdfgroup.org/services/support.html
    and try register another ID?

    Best wishes,
    Andrey Paramonov

    --
    This message has been scanned for viruses and
    dangerous content by MailScanner, and is
    believed to be clean.

    _______________________________________________
    Hdf-forum is for HDF software users discussion.
    Hdf-forum@lists.hdfgroup.org <mailto:Hdf-forum@lists.hdfgroup.org>
    http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
    Twitter: https://twitter.com/hdf5

--
This message has been scanned for viruses and
dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is
believed to be clean.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

miller86 · November 16, 2016, 5:29pm

Ok, so you'll wanna follow the steps here...

https://support.hdfgroup.org/services/contributions.html#filters

Mark

···

--
Mark C. Miller, LLNL

From: Hdf-forum <hdf-forum-bounces@lists.hdfgroup.org<mailto:hdf-forum-bounces@lists.hdfgroup.org>> on behalf of Андрей Парамонов <paramon@acdlabs.ru<mailto:paramon@acdlabs.ru>>
Reply-To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Date: Wednesday, November 16, 2016 at 4:27 AM
To: "hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>" <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: Re: [Hdf-forum] Alt. compression filter implementation

15.11.2016 19:46, Miller, Mark C. пишет:
Hi Andrey,

Here is my take on filter ids

You should use a different filter id.

That said, you have two options in so doing. Use a non-registered id
with value below 32000 that *only*you* care about.

Or, try to get a new registered filter id.

Do you anticipate many other people/organizations using your filter and
sharing data that was filtered with it?

If so, yes, it would be very important to go the "registered filter id"
route. Otherwise, its less important.

Hello!

Thank you for your reply.
The files will be created by other people and probably passed around, so
I'd like to ensure the numbers can still be readable.

Best wishes,
Andrey Paramonov

From: Hdf-forum <hdf-forum-bounces@lists.hdfgroup.org<mailto:hdf-forum-bounces@lists.hdfgroup.org>
<mailto:hdf-forum-bounces@lists.hdfgroup.org>> on behalf of Андрей
Парамонов <paramon@acdlabs.ru<mailto:paramon@acdlabs.ru> <mailto:paramon@acdlabs.ru>>
Reply-To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>
<mailto:hdf-forum@lists.hdfgroup.org>>
Date: Tuesday, November 15, 2016 at 5:20 AM
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>
<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: [Hdf-forum] Alt. compression filter implementation

Hello!

     I'd like to use "bitshuffle" preconditioning filter for HDF5 datasets.
     Luckily, the page
     https://support.hdfgroup.org/services/contributions.html#filters
     points me to Filter ID 32008:
     https://github.com/kiyo-masui/bitshuffle
     However, there are technical limitations that restrict me from usage of
     above plugin. I'm going to implement something simpler, probably not
     even *fully* compatible on cd_nelmts/cd_values.
     In this case, should I still use ID 32008 for my filter?
     Or, I'd rather contact
     https://support.hdfgroup.org/services/support.html
     and try register another ID?

Best wishes,
Andrey Paramonov

     --
     This message has been scanned for viruses and
     dangerous content by MailScanner, and is
     believed to be clean.

     _______________________________________________
     Hdf-forum is for HDF software users discussion.
     Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org> <mailto:Hdf-forum@lists.hdfgroup.org>
     http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
     Twitter: https://twitter.com/hdf5

--
This message has been scanned for viruses and
dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is
believed to be clean.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

Attention! https://support.hdfgroup.org is the NEW home for documentation from The HDF Group. (Details)

Alt. compression filter implementation