Hi,
I have run into a problem where scale-offset compressed data on a big endian
machine is uncompressed perfectly on the BE machine, but is messed up when
read on a LE machine (including output of stuff like h5dump).
Here is the (correct) output on the BE machine (64 bit IBM Power6) which
wrote the data:
bp-login2:/u/home/ac/orf/work15/run/so-only.00480.cdir% uname -a
AIX bp-login2 1 6 00C06B924C00
bp-login2:/u/home/ac/orf/work15/run/so-only.00480.cdir% h5dump -d
/3d/vinterp so-only.00480_0009.cm1hdf5|head
HDF5 "so-only.00480_0009.cm1hdf5" {
DATASET "/3d/vinterp" {
DATATYPE H5T_IEEE_F32BE
DATASPACE SIMPLE { ( 40, 30, 30 ) / ( 40, 30, 30 ) }
DATA {
(0,0,0): -0.0837158, -0.0837158, -0.0837158, -0.0837158, -0.0827158,
(0,0,5): -0.0827158, -0.0827158, -0.0817158, -0.0807158, -0.0787158,
(0,0,10): -0.0767158, -0.0727158, -0.0627158, -0.0407158, -0.00571576,
(0,0,15): 0.0372842, 0.0732842, 0.0942842, 0.0982842, 0.0902842,
(0,0,20): 0.0722842, 0.0492842, 0.0282842, 0.0122842, 0.00928424,
(0,0,25): 0.0242842, 0.0562842, 0.0892842, 0.112284, 0.129284,
(0,1,0): -0.0847158, -0.0847158, -0.0847158, -0.0847158, -0.0847158,
The same file, moved to a LE machine (32 bit intel):
seagrape:/users/orf/work15/compare% uname -a
Linux seagrape 2.6.18-194.26.1.el5PAE #1 SMP Tue Nov 9 13:34:42 EST 2010
i686 i686 i386 GNU/Linux
seagrape:/users/orf/work15/compare% h5dump -d /3d/vinterp
so-only.00480_0009.cm1hdf5|head
HDF5 "so-only.00480_0009.cm1hdf5" {
DATASET "/3d/vinterp" {
DATATYPE H5T_IEEE_F32BE
DATASPACE SIMPLE { ( 40, 30, 30 ) / ( 40, 30, 30 ) }
DATA {
(0,0,0): 1.012, 1.012, 1.012, 1.012, 1.013, 1.013, 1.013, 1.014, 1.015,
(0,0,9): 1.017, 1.019, 1.023, 1.033, 1.055, 1.09, 1.133, 1.169, 1.19,
(0,0,18): 1.194, 1.186, 1.168, 1.145, 1.124, 1.108, 1.105, 1.12, 1.152,
(0,0,27): 1.185, 1.208, 1.225,
(0,1,0): 1.011, 1.011, 1.011, 1.011, 1.011, 1.011, 1.011, 1.012, 1.012,
It is worth noting that no negative values are ever read on the LE machine
for any variable which really is negative.
The BE machine has Version 1.8.4-patch1 while the LE machine has Version
1.8.5-patch1. I am currently unable to build 1.8.5-patch1 on the BE machine
(internal compiler error, fun fun, I love AIX).
Data was written as such:
call h5screate_simple_f(rank,dims,dspace_id,ierror)
call h5pcreate_f(H5P_DATASET_CREATE_F,chunk_id,ierror)
call h5pset_chunk_f(chunk_id,rank,chunkdims,ierror)
call h5pset_scaleoffset_f (chunk_id,0,scale_factor,ierror)
call
h5dcreate_f(file_id,trim(varname),H5T_NATIVE_REAL,dspace_id,dset_id,ierror,chunk_id)
call
h5dwrite_f(dset_id,H5T_NATIVE_REAL,fl_array_3d,dims,ierror,H5S_ALL_F,H5S_ALL_F,H5P_DEFAULT_F)
I do not have these problems when using gzip compression or no compression.
Leigh
···
--
Leigh Orf
Associate Professor of Atmospheric Science
Department of Geology and Meteorology
Central Michigan University
Currently on sabbatical at the National Center for Atmospheric Research
in Boulder, CO
NCAR office phone: (303) 497-8200