Hi Mark,
Thanks for the interesting results on FPC compression.
You probably already saw this, but FPC is licensed for academic
use only. So if you compress data using FPC and distribute
it to a commercial entity, conceivably they could not legally
read it. But then I'm not a lawyer! The FPC license that
accompanies the software reads ...
4. SOFTWARE is made available under this Agreement to allow
certain non-commercial research and teaching use. CRF reserves
all commercial rights to SOFTWARE and these rights may be
licensed by CRF to third parties.
Steve
···
On Wed, Jul 25, 2012 at 12:00:07PM -0400, hdf-forum-request@hdfgroup.org wrote:
Send Hdf-forum mailing list submissions to
hdf-forum@hdfgroup.orgTo subscribe or unsubscribe via the World Wide Web, visit
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
or, via email, send a message with subject or body 'help' to
hdf-forum-request@hdfgroup.orgYou can reach the person managing the list at
hdf-forum-owner@hdfgroup.orgWhen replying, please edit your Subject line so it is more specific
than "Re: Contents of Hdf-forum digest..."Today's Topics:
1. Re: Experience with 'FPC' compressor in concert with HDF5
(Mark Miller)
2. Re: help (Elena Pourmal)----------------------------------------------------------------------
Message: 1
Date: Tue, 24 Jul 2012 18:22:53 -0700
From: Mark Miller <miller86@llnl.gov>
To: HDF Users Discussion List <hdf-forum@hdfgroup.org>
Subject: Re: [Hdf-forum] Experience with 'FPC' compressor in concert
with HDF5
Message-ID: <1343179373.3278.16.camel@skywalker.llnl.gov>
Content-Type: text/plainIn case anyone winds up following 'FPC' to this archive, I thought I
would just mention experience from a simple test I did with FPC (apart
from HDF5 though).I created three binary files of 1,000,000 samples of 4 cycles of a sin
wave. One with a magnitude of 1.0, one with a magnitude of 1e+15 and one
with magnitude of 1e-15. So, its very smooth data!Test FPC compression ratio gzip --best ratio
1.0 mag 2.00 1.27
1e-5 mag 2.00 1.27
1e+5 mag 2.04 1.28So, FPC does better than gzip for this relatively smooth data and FPC
appears to compress and decompress *much* faster (5-10x) than gzip too.Finally, I tried an integer example with a magnitude of 100000 (fitting
well within the range of a 32-bit int) and compressed that with gzip and
get a compression ratio of about 3.25.On Fri, 2012-07-13 at 10:09 -0700, Mark Miller wrote:
> I searched the HDF5-Forum archives and didn't get any hits on 'FPC'
>
> I am curious if anyone has any experience adapting the 'FPC' floating
> point compression code,
> http://www.csl.cornell.edu/~burtscher/research/FPC/
> for use as an internal HDF5 compression 'filter'?
>
> If so, what where your experiences? Would you be willing to share your
> code? Do you know of anything 'better' for floating point data?
>
> Thanks for any info.
>
> Mark
>
>
--
Mark C. Miller, Lawrence Livermore National Laboratory
<<<<<<<<<<<<<<<<!!LLNL BUSINESS ONLY!!>>>>>>>>>>>>>>>>
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511------------------------------
Message: 2
Date: Tue, 24 Jul 2012 21:33:38 -0500
From: Elena Pourmal <epourmal@hdfgroup.org>
To: HDF Users Discussion List <hdf-forum@hdfgroup.org>
Subject: Re: [Hdf-forum] help
Message-ID: <45868C85-DD59-4DC7-812F-FDC9B41A3C71@hdfgroup.org>
Content-Type: text/plain; charset="us-ascii"Hi Andrea,
Thank you for your report. This is definitely an oversight on our part. I will file a bug report (it is an easy fix).
Elena
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Elena Pourmal The HDF Group http://hdfgroup.org
1800 So. Oak St., Suite 203, Champaign IL 61820
217.531.6112
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~On Jul 23, 2012, at 10:40 AM, Andrea Negri wrote:
> Hi, I'm using the last stable release of the gfortran compiler (4.7.1)
> with the flags -Wconversion -Wconversion-extra.
> I'm trying to build an extensible dataset, by using the command
>
> use hdf5
> implicit none
> integer(hsize_t), dimension(7), parameter:: dims_scalar =
> INT((/1,0,0,0,0,0,0/),hsize_t)
> integer, parameter :: rank = 1
> integer(hid_t) :: dataspace_id
> integer :: error
> integer(hsize_t), dimension(1) :: maxdims
>
>
> maxdims = H5S_UNLIMITED_F
> CALL h5screate_simple_f(RANK, dims, dataspace_id, error, &
> & maxdims)
>
>
>
> and the compiler returns the warning:
>
> maxdims = H5S_UNLIMITED_F
> 1
> Warning: Conversion from INTEGER(4) to INTEGER(8) at (1)
>
>
>
> I have checked in the file H5f90global.f90 and H5S_UNLIMITED_F is
> declared as integer, whereas h5screate_simple_f actually requests an
> integer(hsize_t) for the maxdims argument.
>
> I have "fixed" the warning by converting the H5S_UNLIMITED_F variable:
> maxdims = INT(H5S_UNLIMITED_F, HSIZE_T)
>
>
> but the inconsistency still remains.
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> Hdf-forum@hdfgroup.org
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.hdfgroup.org/pipermail/hdf-forum_hdfgroup.org/attachments/20120724/ccb27dfb/attachment-0001.html>------------------------------
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.orgEnd of Hdf-forum Digest, Vol 37, Issue 22
*****************************************
--
Steve Sullivan steves@ucar.edu 303-497-2823
FL/2, Research Applications Laboratory
National Center for Atmospheric Research
PO Box 3000
Boulder CO 80307
USA