HDF.close doesn't seem to close - subsequent open throws an exception

Hi,

This is my first HDF5 experience, so forgive newbie errors. I've got a
simple HDF5 save facility working for .NET program which needs to export
data for import into Matlab. But I've got a strange problem. The first file
save works fine when I press the save button. But trying to save again
causes an error:

cid:image001.png@01CDFED0.A8BEED70

If I quit the program and start it again, I can once again save
successfully. It's as if the H5F.close function isn't really closing the
file. Here's some of the code:

      public void SaveResultsHDF5() {

        H5FileId fileId = H5F.create("tests.h5", H5F.CreateMode.ACC_TRUNC);

        // Rank is the number of dimensions of the data array.

        const int RANK = 2;

        long[] dims = new long[RANK];

        dims[0] = NRfPnts;

        dims[1] = NFreqPnts;

        H5DataSpaceId spaceId = H5S.create_simple(RANK, dims);

        H5DataTypeId typeId = H5T.copy(H5T.H5Type.NATIVE_INT);

        // Create the data set.

        H5DataSetId dataSetId = H5D.create(fileId, "/data", typeId,
spaceId);

        // Write the integer data to the data set.

        H5D.write(dataSetId, typeId, new H5Array<int>(Results));

       // Close all the open resources.

        H5D.close(dataSetId);

        H5T.close(typeId);

        H5F.flush(fileId, H5F.Scope.GLOBAL);

        H5F.close(fileId);

      }

    }

Look's pretty simple, right? What could be I be doing wrong?

Note also that I had to add H5F.flush to the code to get the file to flush.
Otherwise I can't import the file into Matlab until I quit the .NET program.
That seems like further evidence that H5F.close isn't really closing the
file in the way that I would expect a file close operation to do.

Any help on this would be greatly appreciated!

- Andy Voelkel

Have you tried adding H5S.close(spaceId) ? G.

Voelkel

···

From: Hdf-forum [mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of Andy
Sent: Wednesday, January 30, 2013 12:38 PM
To: hdf-forum@hdfgroup.org
Subject: [Hdf-forum] HDF.close doesn't seem to close - subsequent open
throws an exception

Hi,

This is my first HDF5 experience, so forgive newbie errors. I've got a
simple HDF5 save facility working for .NET program which needs to export
data for import into Matlab. But I've got a strange problem. The first file
save works fine when I press the save button. But trying to save again
causes an error:

cid:image001.png@01CDFED0.A8BEED70

If I quit the program and start it again, I can once again save
successfully. It's as if the H5F.close function isn't really closing the
file. Here's some of the code:

      public void SaveResultsHDF5() {

        H5FileId fileId = H5F.create("tests.h5", H5F.CreateMode.ACC_TRUNC);

        // Rank is the number of dimensions of the data array.

        const int RANK = 2;

        long[] dims = new long[RANK];

        dims[0] = NRfPnts;

        dims[1] = NFreqPnts;

        H5DataSpaceId spaceId = H5S.create_simple(RANK, dims);

        H5DataTypeId typeId = H5T.copy(H5T.H5Type.NATIVE_INT);

        // Create the data set.

        H5DataSetId dataSetId = H5D.create(fileId, "/data", typeId,
spaceId);

        // Write the integer data to the data set.

        H5D.write(dataSetId, typeId, new H5Array<int>(Results));

       // Close all the open resources.

        H5D.close(dataSetId);

        H5T.close(typeId);

        H5F.flush(fileId, H5F.Scope.GLOBAL);

        H5F.close(fileId);

      }

    }

Look's pretty simple, right? What could be I be doing wrong?

Note also that I had to add H5F.flush to the code to get the file to flush.
Otherwise I can't import the file into Matlab until I quit the .NET program.
That seems like further evidence that H5F.close isn't really closing the
file in the way that I would expect a file close operation to do.

Any help on this would be greatly appreciated!

- Andy Voelkel

Hi Gerd,

On your suggestion, I tried this, but it didn't help.

- Andy

Heber

···

From: Hdf-forum [mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of Gerd
Sent: Wednesday, January 30, 2013 5:48 PM
To: hdf-forum@hdfgroup.org
Subject: Re: [Hdf-forum] HDF.close doesn't seem to close - subsequent open
throws an exception

Have you tried adding H5S.close(spaceId) ? G.

From: Hdf-forum [mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of Andy
Voelkel
Sent: Wednesday, January 30, 2013 12:38 PM
To: hdf-forum@hdfgroup.org
Subject: [Hdf-forum] HDF.close doesn't seem to close - subsequent open
throws an exception

Hi,

This is my first HDF5 experience, so forgive newbie errors. I've got a
simple HDF5 save facility working for .NET program which needs to export
data for import into Matlab. But I've got a strange problem. The first file
save works fine when I press the save button. But trying to save again
causes an error:

cid:image001.png@01CDFED0.A8BEED70

If I quit the program and start it again, I can once again save
successfully. It's as if the H5F.close function isn't really closing the
file. Here's some of the code:

      public void SaveResultsHDF5() {

        H5FileId fileId = H5F.create("tests.h5", H5F.CreateMode.ACC_TRUNC);

        // Rank is the number of dimensions of the data array.

        const int RANK = 2;

        long[] dims = new long[RANK];

        dims[0] = NRfPnts;

        dims[1] = NFreqPnts;

        H5DataSpaceId spaceId = H5S.create_simple(RANK, dims);

        H5DataTypeId typeId = H5T.copy(H5T.H5Type.NATIVE_INT);

        // Create the data set.

        H5DataSetId dataSetId = H5D.create(fileId, "/data", typeId,
spaceId);

        // Write the integer data to the data set.

        H5D.write(dataSetId, typeId, new H5Array<int>(Results));

       // Close all the open resources.

        H5D.close(dataSetId);

        H5T.close(typeId);

        H5F.flush(fileId, H5F.Scope.GLOBAL);

        H5F.close(fileId);

      }

    }

Look's pretty simple, right? What could be I be doing wrong?

Note also that I had to add H5F.flush to the code to get the file to flush.
Otherwise I can't import the file into Matlab until I quit the .NET program.
That seems like further evidence that H5F.close isn't really closing the
file in the way that I would expect a file close operation to do.

Any help on this would be greatly appreciated!

- Andy Voelkel

Andy, I think it's a problem in a part of your code that we can't see.

Have a look at the attached sample, which creates the same file

10 times in a loop without causing the problem you're seeing.

Best, G.

Voelkel

chunkedDataset.py (1.16 KB)

···

From: Hdf-forum [mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of Andy
Sent: Wednesday, January 30, 2013 4:17 PM
To: 'HDF Users Discussion List'
Subject: Re: [Hdf-forum] HDF.close doesn't seem to close - subsequent open
throws an exception

Hi Gerd,

On your suggestion, I tried this, but it didn't help.

- Andy

From: Hdf-forum [mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of Gerd
Heber
Sent: Wednesday, January 30, 2013 5:48 PM
To: hdf-forum@hdfgroup.org
Subject: Re: [Hdf-forum] HDF.close doesn't seem to close - subsequent open
throws an exception

Have you tried adding H5S.close(spaceId) ? G.

From: Hdf-forum [mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of Andy
Voelkel
Sent: Wednesday, January 30, 2013 12:38 PM
To: hdf-forum@hdfgroup.org
Subject: [Hdf-forum] HDF.close doesn't seem to close - subsequent open
throws an exception

Hi,

This is my first HDF5 experience, so forgive newbie errors. I've got a
simple HDF5 save facility working for .NET program which needs to export
data for import into Matlab. But I've got a strange problem. The first file
save works fine when I press the save button. But trying to save again
causes an error:

cid:image001.png@01CDFED0.A8BEED70

If I quit the program and start it again, I can once again save
successfully. It's as if the H5F.close function isn't really closing the
file. Here's some of the code:

      public void SaveResultsHDF5() {

        H5FileId fileId = H5F.create("tests.h5", H5F.CreateMode.ACC_TRUNC);

        // Rank is the number of dimensions of the data array.

        const int RANK = 2;

        long[] dims = new long[RANK];

        dims[0] = NRfPnts;

        dims[1] = NFreqPnts;

        H5DataSpaceId spaceId = H5S.create_simple(RANK, dims);

        H5DataTypeId typeId = H5T.copy(H5T.H5Type.NATIVE_INT);

        // Create the data set.

        H5DataSetId dataSetId = H5D.create(fileId, "/data", typeId,
spaceId);

        // Write the integer data to the data set.

        H5D.write(dataSetId, typeId, new H5Array<int>(Results));

       // Close all the open resources.

        H5D.close(dataSetId);

        H5T.close(typeId);

        H5F.flush(fileId, H5F.Scope.GLOBAL);

        H5F.close(fileId);

      }

    }

Look's pretty simple, right? What could be I be doing wrong?

Note also that I had to add H5F.flush to the code to get the file to flush.
Otherwise I can't import the file into Matlab until I quit the .NET program.
That seems like further evidence that H5F.close isn't really closing the
file in the way that I would expect a file close operation to do.

Any help on this would be greatly appreciated!

- Andy Voelkel

Thanks Gerd. I'm sure that I can go back and forth between your example and
my code and figure out where the problem is. It may be slightly time
consuming, of course. I'll report back once I find the problem, or at least
narrow it down.

- Andy

Heber

···

From: Hdf-forum [mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of Gerd
Sent: Wednesday, January 30, 2013 7:32 PM
To: 'HDF Users Discussion List'
Subject: Re: [Hdf-forum] HDF.close doesn't seem to close - subsequent open
throws an exception

Andy, I think it's a problem in a part of your code that we can't see.

Have a look at the attached sample, which creates the same file

10 times in a loop without causing the problem you're seeing.

Best, G.

From: Hdf-forum [mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of Andy
Voelkel
Sent: Wednesday, January 30, 2013 4:17 PM
To: 'HDF Users Discussion List'
Subject: Re: [Hdf-forum] HDF.close doesn't seem to close - subsequent open
throws an exception

Hi Gerd,

On your suggestion, I tried this, but it didn't help.

- Andy

From: Hdf-forum [mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of Gerd
Heber
Sent: Wednesday, January 30, 2013 5:48 PM
To: hdf-forum@hdfgroup.org
Subject: Re: [Hdf-forum] HDF.close doesn't seem to close - subsequent open
throws an exception

Have you tried adding H5S.close(spaceId) ? G.

From: Hdf-forum [mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of Andy
Voelkel
Sent: Wednesday, January 30, 2013 12:38 PM
To: hdf-forum@hdfgroup.org
Subject: [Hdf-forum] HDF.close doesn't seem to close - subsequent open
throws an exception

Hi,

This is my first HDF5 experience, so forgive newbie errors. I've got a
simple HDF5 save facility working for .NET program which needs to export
data for import into Matlab. But I've got a strange problem. The first file
save works fine when I press the save button. But trying to save again
causes an error:

cid:image001.png@01CDFED0.A8BEED70

If I quit the program and start it again, I can once again save
successfully. It's as if the H5F.close function isn't really closing the
file. Here's some of the code:

      public void SaveResultsHDF5() {

        H5FileId fileId = H5F.create("tests.h5", H5F.CreateMode.ACC_TRUNC);

        // Rank is the number of dimensions of the data array.

        const int RANK = 2;

        long[] dims = new long[RANK];

        dims[0] = NRfPnts;

        dims[1] = NFreqPnts;

        H5DataSpaceId spaceId = H5S.create_simple(RANK, dims);

        H5DataTypeId typeId = H5T.copy(H5T.H5Type.NATIVE_INT);

        // Create the data set.

        H5DataSetId dataSetId = H5D.create(fileId, "/data", typeId,
spaceId);

        // Write the integer data to the data set.

        H5D.write(dataSetId, typeId, new H5Array<int>(Results));

       // Close all the open resources.

        H5D.close(dataSetId);

        H5T.close(typeId);

        H5F.flush(fileId, H5F.Scope.GLOBAL);

        H5F.close(fileId);

      }

    }

Look's pretty simple, right? What could be I be doing wrong?

Note also that I had to add H5F.flush to the code to get the file to flush.
Otherwise I can't import the file into Matlab until I quit the .NET program.
That seems like further evidence that H5F.close isn't really closing the
file in the way that I would expect a file close operation to do.

Any help on this would be greatly appreciated!

- Andy Voelkel

Hi Gerd,

I found the problem. It was an attribute resource that had not been closed.
You have to very careful with this!

Thanks much - I'm off and running.

- Andy

Heber

···

From: Hdf-forum [mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of Gerd
Sent: Wednesday, January 30, 2013 7:32 PM
To: 'HDF Users Discussion List'
Subject: Re: [Hdf-forum] HDF.close doesn't seem to close - subsequent open
throws an exception

Andy, I think it's a problem in a part of your code that we can't see.

Have a look at the attached sample, which creates the same file

10 times in a loop without causing the problem you're seeing.

Best, G.

I agree it is difficult to know when HDF5 has indeed closed a file due
to the possibility of having left resources associated with a file open.

There are some things you can do to make it easier.

First, when you call H5Fopen, build a non-default prop. list that uses
H5Pset_fclose_degree(fapl, H5F_CLOSE_SEMI) property.

That way, H5Fclose will return error if it was unable to indeed close
the file.

Also, just prior to calling H5Fclose, can use H5Fget_obj_count to
confirm whether or not you have left any open objects in the file.
However, I don't think thats foolproof as I think may neglect to include
uncommitted datatypes (maybe thats just a memory resource though and
won't prevent closing of the file).

Mark

···

On Thu, 2013-01-31 at 15:42 -0300, Andy Voelkel wrote:

Hi Gerd,

I found the problem. It was an attribute resource that had not been
closed. You have to very careful with this!

Thanks much – I’m off and running.

- Andy

From: Hdf-forum [mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of
Gerd Heber
Sent: Wednesday, January 30, 2013 7:32 PM
To: 'HDF Users Discussion List'
Subject: Re: [Hdf-forum] HDF.close doesn't seem to close - subsequent
open throws an exception

Andy, I think it’s a problem in a part of your code that we can’t see.

Have a look at the attached sample, which creates the same file

10 times in a loop without causing the problem you’re seeing.

Best, G.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

--
Mark C. Miller, Lawrence Livermore National Laboratory
================!!LLNL BUSINESS ONLY!!================
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511