R hdf5 package error

Hdf-forum archives
1

I was wondering if anyone has experience with the hdf5 package for the R
statistical software. Right now we can get the code to compile and while
it usually reads existing hdf5 files it doesn't write hdf5 files that can
be read by itself or other software.

We have tried installing this on various Linux machines (both Red Hat and
Ubuntu) by using both the package managers and compiling the code from
scratch. We've compiled using both HDF5 1.8 and 1.6 and neither works.

Specifically, this simple test in R produces the following error

Step 1: create file (run from within R)

library(hdf5)
data = 1:10
hdf5save("test.h5","dat")

Step 2: exit and then restart R

hdf5load("test.h5")

HDF5-DIAG: Error detected in HDF5 library version: 1.6.6 thread
47700932800864. Back trace follows.
#000: H5F.c line 2049 in H5Fopen(): unable to open file
  major(04): File interface
  minor(17): Unable to open file
#001: H5F.c line 1829 in H5F_open(): unable to read superblock
  major(04): File interface
  minor(24): Read failed
#002: H5Fsuper.c line 114 in H5F_read_superblock(): unable to find file
signature
  major(04): File interface
  minor(19): Not an HDF5 file
#003: H5F.c line 1281 in H5F_locate_signature(): unable to find a valid
file signature
  major(05): Low-level I/O layer
  minor(29): Unable to initialize object
Error in hdf5load("test.h5") : unable to open HDF file: test.h5

···

________________________________________________________________________
Michael Dietze mdietze@illinois.edu
Assistant Professor www.life.uiuc.edu/dietze
Department of Plant Biology (217) 265-8020
University of Illinois 1405 IGB / 183 Morrill

2

Silly question here. Is the second arg in hdf5save supposed to be the
name of the array you create above it? If so, one is spelled 'data' and
the arg is spelled "dat". Don't know if that matters.

Mark

···

On Mon, 2010-03-29 at 17:04, mdietze@life.illinois.edu wrote:

I was wondering if anyone has experience with the hdf5 package for the R
statistical software. Right now we can get the code to compile and while
it usually reads existing hdf5 files it doesn't write hdf5 files that can
be read by itself or other software.

We have tried installing this on various Linux machines (both Red Hat and
Ubuntu) by using both the package managers and compiling the code from
scratch. We've compiled using both HDF5 1.8 and 1.6 and neither works.

Specifically, this simple test in R produces the following error

Step 1: create file (run from within R)

library(hdf5)
data = 1:10
hdf5save("test.h5","dat")

Step 2: exit and then restart R

hdf5load("test.h5")

HDF5-DIAG: Error detected in HDF5 library version: 1.6.6 thread
47700932800864. Back trace follows.
#000: H5F.c line 2049 in H5Fopen(): unable to open file
  major(04): File interface
  minor(17): Unable to open file
#001: H5F.c line 1829 in H5F_open(): unable to read superblock
  major(04): File interface
  minor(24): Read failed
#002: H5Fsuper.c line 114 in H5F_read_superblock(): unable to find file
signature
  major(04): File interface
  minor(19): Not an HDF5 file
#003: H5F.c line 1281 in H5F_locate_signature(): unable to find a valid
file signature
  major(05): Low-level I/O layer
  minor(29): Unable to initialize object
Error in hdf5load("test.h5") : unable to open HDF file: test.h5

________________________________________________________________________
Michael Dietze mdietze@illinois.edu
Assistant Professor www.*life.uiuc.edu/dietze
Department of Plant Biology (217) 265-8020
University of Illinois 1405 IGB / 183 Morrill

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://*mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

--
Mark C. Miller, Lawrence Livermore National Laboratory
================!!LLNL BUSINESS ONLY!!================
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-851

3

I was wondering if anyone has experience with the hdf5 package for the R
statistical software. Right now we can get the code to compile and while
it usually reads existing hdf5 files it doesn't write hdf5 files that can
be read by itself or other software.

We have tried installing this on various Linux machines (both Red Hat and
Ubuntu) by using both the package managers and compiling the code from
scratch. We've compiled using both HDF5 1.8 and 1.6 and neither works.

We use R routinely with files converted using h4toh5, but 99% of the time we
use hdf5load. We try to use the platform's hdf5 library, which in our case is
1.6 on Ubuntu and 1.8 via macports on MacOSX.

There have been versions of the R hdf5 library for Ubuntu, but some versions
abort when they encounter elements in the hdf5 that aren't supported. Our
files have such elements but the information is redundant, so we patch the
library to omit the unsupported elements.

Have you run "example(hdf5save)" successfully?

Specifically, this simple test in R produces the following error

Step 1: create file (run from within R)

library(hdf5)
data = 1:10
hdf5save("test.h5","dat")

did you mean: hdf5save("test.h5","data")

Step 2: exit and then restart R

hdf5load("test.h5")

HDF5-DIAG: Error detected in HDF5 library version: 1.6.6 thread
47700932800864. Back trace follows.
#000: H5F.c line 2049 in H5Fopen(): unable to open file
  major(04): File interface
  minor(17): Unable to open file

This looks more like a permission problem. If you change the filename to
"/tmp/test.h5" do you get the same error?

···

On 3/29/10, mdietze@life.illinois.edu <mdietze@life.illinois.edu> wrote:

#001: H5F.c line 1829 in H5F_open(): unable to read superblock
  major(04): File interface
  minor(24): Read failed
#002: H5Fsuper.c line 114 in H5F_read_superblock(): unable to find file
signature
  major(04): File interface
  minor(19): Not an HDF5 file
#003: H5F.c line 1281 in H5F_locate_signature(): unable to find a valid
file signature
  major(05): Low-level I/O layer
  minor(29): Unable to initialize object
Error in hdf5load("test.h5") : unable to open HDF file: test.h5

--
George N. White III <aa056@chebucto.ns.ca>
Head of St. Margarets Bay, Nova Scotia

4

Michael,

Did you run HDF5 tests when you built HDF5 library? Did they pass?

Elena

···

On Mar 29, 2010, at 7:04 PM, mdietze@life.illinois.edu wrote:

I was wondering if anyone has experience with the hdf5 package for the R
statistical software. Right now we can get the code to compile and while
it usually reads existing hdf5 files it doesn't write hdf5 files that can
be read by itself or other software.

We have tried installing this on various Linux machines (both Red Hat and
Ubuntu) by using both the package managers and compiling the code from
scratch. We've compiled using both HDF5 1.8 and 1.6 and neither works.

Specifically, this simple test in R produces the following error

Step 1: create file (run from within R)

library(hdf5)
data = 1:10
hdf5save("test.h5","dat")

Step 2: exit and then restart R

hdf5load("test.h5")

HDF5-DIAG: Error detected in HDF5 library version: 1.6.6 thread
47700932800864. Back trace follows.
#000: H5F.c line 2049 in H5Fopen(): unable to open file
major(04): File interface
minor(17): Unable to open file
#001: H5F.c line 1829 in H5F_open(): unable to read superblock
major(04): File interface
minor(24): Read failed
#002: H5Fsuper.c line 114 in H5F_read_superblock(): unable to find file
signature
major(04): File interface
minor(19): Not an HDF5 file
#003: H5F.c line 1281 in H5F_locate_signature(): unable to find a valid
file signature
major(05): Low-level I/O layer
minor(29): Unable to initialize object
Error in hdf5load("test.h5") : unable to open HDF file: test.h5

________________________________________________________________________
Michael Dietze mdietze@illinois.edu
Assistant Professor www.life.uiuc.edu/dietze
Department of Plant Biology (217) 265-8020
University of Illinois 1405 IGB / 183 Morrill

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

5

Hello,

I have a integer dataset, say 500 million by 1.

Each row in that dataset represents a discrete point.

I read in a file which contains both points and values for *some* of those points in the dataset.

I want to write, for each point in my file, a value in the dataset.

my current code is similar to what follows,

while (fgets (line, LINE_MAX, baseOut) != NULL)
      {
        tc = strtok(line, sep);
        if (strcmp(tc, "chr") == 0)
        {
          continue;
        }
        tl = strtok(NULL, sep);
        start [0] = atoi(tl);
        start [1] = 0;
        tl = strtok(NULL, sep);
        group = H5Gopen(file, tc, H5P_DEFAULT);
        dataset = H5Dopen2(group, "positions", H5P_DEFAULT);
        count [0] = 1;
        count [1] = 1;
        dims[0] = 1;
        dims[1] = 1;
        tl = strtok(NULL, sep);
        data[0][0][0] = atof(tl);
        memspace = H5Screate_simple (3, dims, NULL);
        dataspace = H5Dget_space (dataset);
        status = H5Sselect_hyperslab (dataspace, H5S_SELECT_SET, start, NULL,
                          count, NULL);
        status = H5Dwrite (dataset, H5T_NATIVE_FLOAT, memspace,
                     dataspace, H5P_DEFAULT, data);
        status = H5Sclose(memspace);
        status = H5Sclose(dataspace);
        status = H5Dclose(dataset);
        status = H5Gclose(group);
      }
        }
    status = H5Fclose(file);
        fclose(baseOut);

it basically loops through the file, opens the correct group, opens the correct dataset, reads a point/value, writes the point/value to the data, closes the group, closes the dataset, etc.
the problem is the write speed -- i recently tested out reading a 4.4G file and writing to a dataset, and it took about 55 minutes using 1 core on a lustre filesystem (i'm not sure these are in par with the expectations..?).
i think my implementation is unideal, since it writes one point at a time; but the datapoints are always non-contiguous.

any advice for speedup?

Thanks,
-Paul

6

You should take a lot things out of the while loop for better performance, such as
H5Gopen(), H5Dopen(), H5Screate_simple.

Thanks
--pc

Paul Zumbo wrote:

···

Hello,

I have a integer dataset, say 500 million by 1.

Each row in that dataset represents a discrete point.

I read in a file which contains both points and values for *some* of those points in the dataset.

I want to write, for each point in my file, a value in the dataset.

my current code is similar to what follows,

while (fgets (line, LINE_MAX, baseOut) != NULL)
      {
        tc = strtok(line, sep);
        if (strcmp(tc, "chr") == 0)
        {
          continue;
        }
        tl = strtok(NULL, sep);
        start [0] = atoi(tl);
        start [1] = 0;
        tl = strtok(NULL, sep);
        group = H5Gopen(file, tc, H5P_DEFAULT);
        dataset = H5Dopen2(group, "positions", H5P_DEFAULT);
        count [0] = 1;
        count [1] = 1;
        dims[0] = 1;
        dims[1] = 1;
        tl = strtok(NULL, sep);
        data[0][0][0] = atof(tl);
        memspace = H5Screate_simple (3, dims, NULL); dataspace = H5Dget_space (dataset);
        status = H5Sselect_hyperslab (dataspace, H5S_SELECT_SET, start, NULL,
                          count, NULL);
        status = H5Dwrite (dataset, H5T_NATIVE_FLOAT, memspace,
                     dataspace, H5P_DEFAULT, data);
        status = H5Sclose(memspace);
        status = H5Sclose(dataspace);
        status = H5Dclose(dataset);
        status = H5Gclose(group);
      }
        }
    status = H5Fclose(file);
        fclose(baseOut);

it basically loops through the file, opens the correct group, opens the correct dataset, reads a point/value, writes the point/value to the data, closes the group, closes the dataset, etc.
the problem is the write speed -- i recently tested out reading a 4.4G file and writing to a dataset, and it took about 55 minutes using 1 core on a lustre filesystem (i'm not sure these are in par with the expectations..?).
i think my implementation is unideal, since it writes one point at a time; but the datapoints are always non-contiguous.

any advice for speedup?

Thanks,
-Paul

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

7

Message: 5
Date: Mon, 29 Mar 2010 19:12:51 -0600
From: "George N. White III" <gnwiii@gmail.com>
To: HDF Users Discussion List <hdf-forum@hdfgroup.org>
Subject: Re: [Hdf-forum] R hdf5 package error
Message-ID:
  <22af238a1003291812r7b0ccde2yc84857e8f645171f@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

>
> I was wondering if anyone has experience with the hdf5 package for the R
> statistical software. Right now we can get the code to compile and while
> it usually reads existing hdf5 files it doesn't write hdf5 files that can
> be read by itself or other software.
>
> We have tried installing this on various Linux machines (both Red Hat and
> Ubuntu) by using both the package managers and compiling the code from
> scratch. We've compiled using both HDF5 1.8 and 1.6 and neither works.

We use R routinely with files converted using h4toh5, but 99% of the time we
use hdf5load. We try to use the platform's hdf5 library, which in our case is
1.6 on Ubuntu and 1.8 via macports on MacOSX.

There have been versions of the R hdf5 library for Ubuntu, but some versions
abort when they encounter elements in the hdf5 that aren't supported. Our
files have such elements but the information is redundant, so we patch the
library to omit the unsupported elements.

Have you run "example(hdf5save)" successfully?

Yes, that runs fine. In fact, as long as we don't quit out of R we can
both read and write files, but if we quit and restart R then we can no
longer read the files we just wrote and were able to read a second ago.

> Specifically, this simple test in R produces the following error
>
> Step 1: create file (run from within R)
>
> library(hdf5)
> data = 1:10
> hdf5save("test.h5","dat")

did you mean: hdf5save("test.h5","data")

That's a typo...just in the email, not in what I've been running -- I
actually meant for both to be "dat"

>
> Step 2: exit and then restart R
>
> hdf5load("test.h5")
>
> HDF5-DIAG: Error detected in HDF5 library version: 1.6.6 thread
> 47700932800864. Back trace follows.
> #000: H5F.c line 2049 in H5Fopen(): unable to open file
> major(04): File interface
> minor(17): Unable to open file

This looks more like a permission problem. If you change the filename to
"/tmp/test.h5" do you get the same error?

yes I get the same error

I also checked that I do have read/write permissions for the file. In
addition from outside R I ran

h5dump test.h5

which returned

h5dump error: unable to open file "test.h5"

An additional odd bit of info is that we do have one computer that runs
an older version of Ubuntu where the library has worked for a long time
and we only discovered this error when our sysadmin tried putting this
on our Redhat cluster. Even when we download the same exact version of
hdf5 and the R hdf5 library that is on the machine that does work we
still can't get things to work. We also found that on our newer Ubuntu
machines that the version that comes from the Synaptic package manager
doesn't work either.

We are just storing vectors of floats so the data format is nothing odd
that should be hard to read or write.

Also, for what it's worth, I tried contacting the author of the R hdf5
package about a week ago and got no response.

···

On 3/29/10, mdietze@life.illinois.edu <mdietze@life.illinois.edu> wrote:

> #001: H5F.c line 1829 in H5F_open(): unable to read superblock
> major(04): File interface
> minor(24): Read failed
> #002: H5Fsuper.c line 114 in H5F_read_superblock(): unable to find file
> signature
> major(04): File interface
> minor(19): Not an HDF5 file
> #003: H5F.c line 1281 in H5F_locate_signature(): unable to find a valid
> file signature
> major(05): Low-level I/O layer
> minor(29): Unable to initialize object
> Error in hdf5load("test.h5") : unable to open HDF file: test.h5

--
George N. White III <aa056@chebucto.ns.ca>
Head of St. Margarets Bay, Nova Scotia

------------------------------

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

End of Hdf-forum Digest, Vol 9, Issue 55
****************************************

--
________________________________________________________________________
Michael Dietze mdietze@illinois.edu
Assistant Professor www.life.uiuc.edu/dietze
Department of Plant Biology (217) 265-8020
University of Illinois 1405 IGB / 183 Morrill

8

Michael,

Could you please try h5check tool to see if file is corrupted?

We are in the process of releasing a new version of h5check, so there is no an officially released tar ball yet.
Please grab the code from ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/epourmal/h5check.tar

Elena

···

On Mar 30, 2010, at 11:35 AM, Michael Dietze wrote:

Message: 5
Date: Mon, 29 Mar 2010 19:12:51 -0600
From: "George N. White III" <gnwiii@gmail.com>
To: HDF Users Discussion List <hdf-forum@hdfgroup.org>
Subject: Re: [Hdf-forum] R hdf5 package error
Message-ID:
  <22af238a1003291812r7b0ccde2yc84857e8f645171f@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

On 3/29/10, mdietze@life.illinois.edu <mdietze@life.illinois.edu> wrote:

I was wondering if anyone has experience with the hdf5 package for the R
statistical software. Right now we can get the code to compile and while
it usually reads existing hdf5 files it doesn't write hdf5 files that can
be read by itself or other software.

We have tried installing this on various Linux machines (both Red Hat and
Ubuntu) by using both the package managers and compiling the code from
scratch. We've compiled using both HDF5 1.8 and 1.6 and neither works.

We use R routinely with files converted using h4toh5, but 99% of the time we
use hdf5load. We try to use the platform's hdf5 library, which in our case is
1.6 on Ubuntu and 1.8 via macports on MacOSX.

There have been versions of the R hdf5 library for Ubuntu, but some versions
abort when they encounter elements in the hdf5 that aren't supported. Our
files have such elements but the information is redundant, so we patch the
library to omit the unsupported elements.

Have you run "example(hdf5save)" successfully?

Yes, that runs fine. In fact, as long as we don't quit out of R we can
both read and write files, but if we quit and restart R then we can no
longer read the files we just wrote and were able to read a second ago.

Specifically, this simple test in R produces the following error

Step 1: create file (run from within R)

library(hdf5)
data = 1:10
hdf5save("test.h5","dat")

did you mean: hdf5save("test.h5","data")

That's a typo...just in the email, not in what I've been running -- I
actually meant for both to be "dat"

Step 2: exit and then restart R

hdf5load("test.h5")

HDF5-DIAG: Error detected in HDF5 library version: 1.6.6 thread
47700932800864. Back trace follows.
#000: H5F.c line 2049 in H5Fopen(): unable to open file
major(04): File interface
minor(17): Unable to open file

This looks more like a permission problem. If you change the filename to
"/tmp/test.h5" do you get the same error?

yes I get the same error

I also checked that I do have read/write permissions for the file. In
addition from outside R I ran

h5dump test.h5

which returned

h5dump error: unable to open file "test.h5"

An additional odd bit of info is that we do have one computer that runs
an older version of Ubuntu where the library has worked for a long time
and we only discovered this error when our sysadmin tried putting this
on our Redhat cluster. Even when we download the same exact version of
hdf5 and the R hdf5 library that is on the machine that does work we
still can't get things to work. We also found that on our newer Ubuntu
machines that the version that comes from the Synaptic package manager
doesn't work either.

We are just storing vectors of floats so the data format is nothing odd
that should be hard to read or write.

Also, for what it's worth, I tried contacting the author of the R hdf5
package about a week ago and got no response.

#001: H5F.c line 1829 in H5F_open(): unable to read superblock
major(04): File interface
minor(24): Read failed
#002: H5Fsuper.c line 114 in H5F_read_superblock(): unable to find file
signature
major(04): File interface
minor(19): Not an HDF5 file
#003: H5F.c line 1281 in H5F_locate_signature(): unable to find a valid
file signature
major(05): Low-level I/O layer
minor(29): Unable to initialize object
Error in hdf5load("test.h5") : unable to open HDF file: test.h5

--
George N. White III <aa056@chebucto.ns.ca>
Head of St. Margarets Bay, Nova Scotia

------------------------------

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

End of Hdf-forum Digest, Vol 9, Issue 55
****************************************

--
________________________________________________________________________
Michael Dietze mdietze@illinois.edu
Assistant Professor www.life.uiuc.edu/dietze
Department of Plant Biology (217) 265-8020
University of Illinois 1405 IGB / 183 Morrill

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

9

Michael,

Could you please try h5check tool to see if file is corrupted?

I've been travelling (with a netbook -- no access to compilers), but I
can confirm the problem on
MacOSX. I'm using the hdf5-18 libraries from macports:

$ port info hdf5-18
hdf5-18 @1.8.4-patch1 (science)
Variants: threadsafe, (+)universal

Description: HDF5 general purpose library and file format for storing
                      scientific data
Homepage: http://www.hdfgroup.org/HDF5/

Library Dependencies: zlib, szip
Conflicts with: hdf5
Platforms: darwin
License: unknown
Maintainers: mmoll@macports.org, openmaintainer@macports.org

$ port installed hdf5-18
The following ports are currently installed:
  hdf5-18 @1.8.4_0+universal
  hdf5-18 @1.8.4-patch1_0+universal (active)

hdf5load("test.h5")

HDF5-DIAG: Error detected in HDF5 (1.8.4-patch1) thread 0:
  #000: H5F.c line 1514 in H5Fopen(): unable to open file
    major: File accessability
    minor: Unable to open file
  #001: H5F.c line 1309 in H5F_open(): unable to read superblock
    major: File accessability
    minor: Read failed
  #002: H5Fsuper.c line 305 in H5F_super_read(): unable to find file signature
    major: File accessability
    minor: Not an HDF5 file
  #003: H5Fsuper.c line 153 in H5F_locate_signature(): unable to find
a valid file signature
    major: Low-level I/O
    minor: Unable to initialize object
Error in hdf5load("test.h5") : unable to open HDF file: test.h5

system("/opt/hdfgroup.org/bin/h5check test.h5")

VALIDATING test.h5 according to library version 1.8.0

***Error***
Superblock:Unable to find super block signature at addr 0
***End of Error messages***
ASSUMING super block at physical address 0.
***Error***
Errors found when checking superblock. Validation stopped.
Superblock v.0/1:Undefined object header address in root group symbol
table entry at addr 24
Superblock v.0/1:Invalid End of File Address at addr 24
Superblock v.0/1:Invalid base address at addr 24
Superblock v.0/1:Invalid value for Group Internal Node K at addr 18
Superblock v.0/1:Invalid value for Group Leaf Node K at addr 16
Superblock v.0/1:Invalid Size of Lengths at addr 14
Superblock v.0/1:Invalid Size of Offsets at addr 13
***End of Error messages***
Non-compliance errors found

···

On Tue, Mar 30, 2010 at 2:18 PM, Elena Pourmal <epourmal@hdfgroup.org> wrote:

We are in the process of releasing a new version of h5check, so there is no an officially released tar ball yet.
Please grab the code from ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/epourmal/h5check.tar

Elena
On Mar 30, 2010, at 11:35 AM, Michael Dietze wrote:

Message: 5
Date: Mon, 29 Mar 2010 19:12:51 -0600
From: "George N. White III" <gnwiii@gmail.com>
To: HDF Users Discussion List <hdf-forum@hdfgroup.org>
Subject: Re: [Hdf-forum] R hdf5 package error
Message-ID:
<22af238a1003291812r7b0ccde2yc84857e8f645171f@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

On 3/29/10, mdietze@life.illinois.edu <mdietze@life.illinois.edu> wrote:

I was wondering if anyone has experience with the hdf5 package for the R
statistical software. Right now we can get the code to compile and while
it usually reads existing hdf5 files it doesn't write hdf5 files that can
be read by itself or other software.

We have tried installing this on various Linux machines (both Red Hat and
Ubuntu) by using both the package managers and compiling the code from
scratch. We've compiled using both HDF5 1.8 and 1.6 and neither works.

We use R routinely with files converted using h4toh5, but 99% of the time we
use hdf5load. We try to use the platform's hdf5 library, which in our case is
1.6 on Ubuntu and 1.8 via macports on MacOSX.

There have been versions of the R hdf5 library for Ubuntu, but some versions
abort when they encounter elements in the hdf5 that aren't supported. Our
files have such elements but the information is redundant, so we patch the
library to omit the unsupported elements.

Have you run "example(hdf5save)" successfully?

Yes, that runs fine. In fact, as long as we don't quit out of R we can
both read and write files, but if we quit and restart R then we can no
longer read the files we just wrote and were able to read a second ago.

Specifically, this simple test in R produces the following error

Step 1: create file (run from within R)

library(hdf5)
data = 1:10
hdf5save("test.h5","dat")

did you mean: hdf5save("test.h5","data")

That's a typo...just in the email, not in what I've been running -- I
actually meant for both to be "dat"

Step 2: exit and then restart R

hdf5load("test.h5")

HDF5-DIAG: Error detected in HDF5 library version: 1.6.6 thread
47700932800864. Back trace follows.
#000: H5F.c line 2049 in H5Fopen(): unable to open file
major(04): File interface
minor(17): Unable to open file

This looks more like a permission problem. If you change the filename to
"/tmp/test.h5" do you get the same error?

yes I get the same error

I also checked that I do have read/write permissions for the file. In
addition from outside R I ran

h5dump test.h5

which returned

h5dump error: unable to open file "test.h5"

An additional odd bit of info is that we do have one computer that runs
an older version of Ubuntu where the library has worked for a long time
and we only discovered this error when our sysadmin tried putting this
on our Redhat cluster. Even when we download the same exact version of
hdf5 and the R hdf5 library that is on the machine that does work we
still can't get things to work. We also found that on our newer Ubuntu
machines that the version that comes from the Synaptic package manager
doesn't work either.

We are just storing vectors of floats so the data format is nothing odd
that should be hard to read or write.

Also, for what it's worth, I tried contacting the author of the R hdf5
package about a week ago and got no response.

#001: H5F.c line 1829 in H5F_open(): unable to read superblock
major(04): File interface
minor(24): Read failed
#002: H5Fsuper.c line 114 in H5F_read_superblock(): unable to find file
signature
major(04): File interface
minor(19): Not an HDF5 file
#003: H5F.c line 1281 in H5F_locate_signature(): unable to find a valid
file signature
major(05): Low-level I/O layer
minor(29): Unable to initialize object
Error in hdf5load("test.h5") : unable to open HDF file: test.h5

--
George N. White III <aa056@chebucto.ns.ca>
Head of St. Margarets Bay, Nova Scotia

------------------------------

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

End of Hdf-forum Digest, Vol 9, Issue 55
****************************************

--
________________________________________________________________________
Michael Dietze mdietze@illinois.edu
Assistant Professor www.life.uiuc.edu/dietze
Department of Plant Biology (217) 265-8020
University of Illinois 1405 IGB / 183 Morrill

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

--
George N. White III <aa056@chebucto.ns.ca>
Head of St. Margarets Bay, Nova Scotia