Infinite loop closing library

andrew.collette · November 28, 2010, 11:17pm

Hi,

Increasingly often, I get the following message when my application
exits, having used HDF5:

HDF5: infinite loop closing library
R,D,G,A,S,T,F,FD,P,FD,P,FD,P,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E

Does anyone know where this message originates, or what the letters
mean? Is it benign or could it be associated with data loss? For
some reason, the message appears on Linux but not when the same
program is run on Windows.

Unfortunately I have not been able to narrow down the problem enough
to provide a C test script.

Thanks,
Andrew Collette

epourmal · November 29, 2010, 12:11am

Hi Andrew,

For some reason library cannot shutdown different interfaces (R - references, E -error, G - group, etc.).

Which version of HDF5 do you use?

Hi,

Increasingly often, I get the following message when my application
exits, having used HDF5:

HDF5: infinite loop closing library
R,D,G,A,S,T,F,FD,P,FD,P,FD,P,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E

Does anyone know where this message originates, or what the letters
mean? Is it benign or could it be associated with data loss?

This error may indicate a bug in the HDF5 library or in the application
Having an example (in any language) that reproduces the problem will definitely help.

Thank you!

Elena

···

On Nov 28, 2010, at 5:17 PM, Andrew Collette wrote:

For
some reason, the message appears on Linux but not when the same
program is run on Windows.

Unfortunately I have not been able to narrow down the problem enough
to provide a C test script.

Thanks,
Andrew Collette

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

andrew.collette · November 29, 2010, 2:01am

Hi Elena,

For some reason library cannot shutdown different interfaces (R - references, E -error, G - group, etc.).

Which version of HDF5 do you use?

I've tested it with HDF5 1.8.4 and 1.8.5 so far.

This error may indicate a bug in the HDF5 library or in the application
Having an example (in any language) that reproduces the problem will definitely help.

Yes. Unfortunately what happens is that I run the entire test suite
for my package (~200 tests, each of which makes dozens or hundreds of
calls to HDF5), and then when I exit this crops up.

What application-level things could be responsible for this? I am
registering callbacks with the H5E and H5T interfaces, but these don't
seem to be the problem... if I register them and then exit without
doing anything else, it doesn't print the message.

Thanks,
Andrew

Quincey_Koziol · November 29, 2010, 4:12pm

Hi Andrew,

For some reason library cannot shutdown different interfaces (R - references, E -error, G - group, etc.).

Usually, that means that there's a resource leak of some sort.

Quincey

···

On Nov 28, 2010, at 6:11 PM, Elena Pourmal wrote:

Which version of HDF5 do you use?
On Nov 28, 2010, at 5:17 PM, Andrew Collette wrote:

Hi,

Increasingly often, I get the following message when my application
exits, having used HDF5:

HDF5: infinite loop closing library
R,D,G,A,S,T,F,FD,P,FD,P,FD,P,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E,E

Does anyone know where this message originates, or what the letters
mean? Is it benign or could it be associated with data loss?

This error may indicate a bug in the HDF5 library or in the application
Having an example (in any language) that reproduces the problem will definitely help.

Thank you!

Elena

For
some reason, the message appears on Linux but not when the same
program is run on Windows.

Unfortunately I have not been able to narrow down the problem enough
to provide a C test script.

Thanks,
Andrew Collette

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

werner · November 29, 2010, 3:03am

Hi Andrew,

I got these problems when some HDF5 calls were issued after the HDF5 library
was closed. That was mainly a problem with C++, dynamic libraries and global
destructors. The HDF5 library uses an atexit() call to register a cleanup
routine at termination of the main() routine, and after that there must be no
call to HDF5 functions any more.
It may also be an indication of memory corruption, but that is even harder to
trace down. A difference among Linux and Windows appears to be initialization
of newly allocated memories by zero's or not (from malloc() ), so might be
related to uninitialized RAM.

Werner

···

On Sun, 28 Nov 2010 20:01:05 -0600, Andrew Collette <andrew.collette@gmail.com> wrote:

Hi Elena,

For some reason library cannot shutdown different interfaces (R - references, E -error, G - group, etc.).

Which version of HDF5 do you use?

I've tested it with HDF5 1.8.4 and 1.8.5 so far.

This error may indicate a bug in the HDF5 library or in the application
Having an example (in any language) that reproduces the problem will definitely help.

Yes. Unfortunately what happens is that I run the entire test suite
for my package (~200 tests, each of which makes dozens or hundreds of
calls to HDF5), and then when I exit this crops up.

What application-level things could be responsible for this? I am
registering callbacks with the H5E and H5T interfaces, but these don't
seem to be the problem... if I register them and then exit without
doing anything else, it doesn't print the message.

Thanks,
Andrew

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

epourmal · November 29, 2010, 3:06am

Hi Andrew,

Hi Elena,

For some reason library cannot shutdown different interfaces (R - references, E -error, G - group, etc.).

Which version of HDF5 do you use?

I've tested it with HDF5 1.8.4 and 1.8.5 so far.

Could you please test it with 1.8.5-patch1? It would be even better to test with 1.8.6-pre2. Quincey and team did a lot of bug fixing and removed memory leaks in HDF5.

This error may indicate a bug in the HDF5 library or in the application
Having an example (in any language) that reproduces the problem will definitely help.

Yes. Unfortunately what happens is that I run the entire test suite
for my package (~200 tests, each of which makes dozens or hundreds of
calls to HDF5), and then when I exit this crops up.

What application-level things could be responsible for this?

Some kind of memory corruption?

The HDF folks,
Could someone, please, elaborate?

Thank you!

Elena

···

On Nov 28, 2010, at 8:01 PM, Andrew Collette wrote:

I am
registering callbacks with the H5E and H5T interfaces, but these don't
seem to be the problem... if I register them and then exit without
doing anything else, it doesn't print the message.

Thanks,
Andrew

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Quincey_Koziol · November 29, 2010, 4:14pm

Hi Andrew,

Hi Elena,

For some reason library cannot shutdown different interfaces (R - references, E -error, G - group, etc.).

Which version of HDF5 do you use?

I've tested it with HDF5 1.8.4 and 1.8.5 so far.

Could you please test it with 1.8.5-patch1? It would be even better to test with 1.8.6-pre2. Quincey and team did a lot of bug fixing and removed memory leaks in HDF5.

This error may indicate a bug in the HDF5 library or in the application
Having an example (in any language) that reproduces the problem will definitely help.

Yes. Unfortunately what happens is that I run the entire test suite
for my package (~200 tests, each of which makes dozens or hundreds of
calls to HDF5), and then when I exit this crops up.

What application-level things could be responsible for this?

Some kind of memory corruption?

The HDF folks,
Could someone, please, elaborate?

I can look into it further with a reproducible failure, but it would be best if it was as simple as possible.

Quincey

···

On Nov 28, 2010, at 9:06 PM, Elena Pourmal wrote:

On Nov 28, 2010, at 8:01 PM, Andrew Collette wrote:

Thank you!

Elena

I am
registering callbacks with the H5E and H5T interfaces, but these don't
seem to be the problem... if I register them and then exit without
doing anything else, it doesn't print the message.

Thanks,
Andrew

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

joe · January 23, 2019, 5:56pm

I’ve recently come across this same issue, though, with parallel HDF5 in Fortran. It only occurs for select number of MPI ranks when writing hyperslabs to file.

Had there been any resolution on this ?
Is a minimal reproducer still valuable here ?
Alternatively, if someone from HDF5 would like access to a platform where I can reproduce the issue in a larger code base, I’m more than happy to accommodate.