HDF GZ compression increases file size ??

Hdf-forum archives
1

Hi i'm using hdf 1.8.2
I've been playing with the standard GZ filter to compress my files and
when treating some files i found out it actually increased the file size ! Has anyone else seen this ? even level 1 comrpession or level 6
compression yield the same file size and an increase of almost 50% in file
size. Is it possible that chunk size is responsible of this ? Is there an
intelligent way to set this parameter then ? thanks a lot for any help you might provide
Pablo

2

hi

Checking with h5dump yielded these results. So the compression is indeed active and the dataspace size is the same but not the storage layout size.I get a lot larger storage size with the compression.
Do you have any clue as to how to set dynamically the chunk size to avoid this issue ? ( i'm supposing it comes from our chunk size of course .. maybe i'm wrong).

Gabriel

  DATASET "bnd\_tri\->elem" \{
     DATATYPE  H5T\_STD\_I32LE
     DATASPACE  SIMPLE \{ \( 1706886 \) / \( 1706886 \) \}
     STORAGE\_LAYOUT \{
        CHUNKED \( 682754 \)
        SIZE 8193048 \(0\.833:1 COMPRESSION\)
      \}
     FILTERS \{
        COMPRESSION DEFLATE \{ LEVEL 6 \}
     \}
     FILLVALUE \{
        FILL\_TIME H5D\_FILL\_TIME\_IFSET
        VALUE  0
     \}
     ALLOCATION\_TIME \{
        H5D\_ALLOC\_TIME\_INCR
     \}
  \}

  DATASET "bnd\_tri\->elem" \{
     DATATYPE  H5T\_STD\_I32LE
     DATASPACE  SIMPLE \{ \( 1706886 \) / \( 1706886 \) \}
     STORAGE\_LAYOUT \{
        CONTIGUOUS
        SIZE 6827544
        OFFSET 122165496
     \}
     FILTERS \{
        NONE
     \}
     FILLVALUE \{
        FILL\_TIME H5D\_FILL\_TIME\_IFSET
        VALUE  0
     \}
     ALLOCATION\_TIME \{
        H5D\_ALLOC\_TIME\_LATE
     \}
···

________________________________
De : Alligator <crocopablo@yahoo.fr>
À : "hdf-forum@hdfgroup.org" <hdf-forum@hdfgroup.org>
Envoyé le : Mercredi 6 juin 2012 22h25
Objet : [Hdf-forum] HDF GZ compression increases file size ??

Hi i'm using hdf 1.8.2
I've been playing with the standard GZ filter to compress my files and
when treating some files i found out it actually increased the file size ! Has anyone else seen this ? even level 1 comrpession or level 6
compression yield the same file size and an increase of almost 50% in file
size. Is it possible that chunk size is responsible of this ? Is there an
intelligent way to set this parameter then ? thanks a lot for any help you might provide
Pablo
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org