We have an application that produces an HDF5 file using library 1.8.8. It
has been working fine for some time on Windows XP SP 3.
Running the SAME BINARIES on Windows 7 64 bit machine in 32 bit environment
produces some files that are corrupt (but others are fine).
The corruption is reported by h5debug as "unable to find a valid file
signature".
Detailed examination and comparison with a hex editor shows that ALL of the
metadata appears to be missing: the first part of the file is just zeroes,
whereas for the "good" file (produced on the win32 xp box, using same
binaries) contains the hdf5 signature string at the start of the file.
Subsequent differences in the files all seem to point to missing metadata
(tree structure, attributes etc).
Anyone else come across this problem?
Just to be clear: the HDF character sequence that forms the start of the
"\211HDF\r\n\032\n" signature is found in the good file at the beginning,
but is not found ANYWHERE in the bad file.
** UPDATE **
Problem has been solved by calling flush(H5F_SCOPE_GLOBAL) prior to calling
the close() method on the H5File object.
Which raises the following questions ...
the fortran api states:
H5Fclose terminates access to an HDF5 file by flushing all data to storage
and terminating access to the file through file_id.
So does this mean the C++ "close" api behaves differently? If so, why?
Why did the previous code always work ok for XP SP3 but only work for some
files on Windows 7 (32 bit or 32 bit on 64 bit OS) - some difference in the
way Windows handles memory/file flushing between the 2 OS types?
Has there been a change in the behaviour on file close between 1.8.5 and
1.8.8 (I did my original dev work at the time of 1.8.5)?
···
--
View this message in context: http://hdf-forum.184993.n3.nabble.com/HDF5-in-32-bit-environment-on-Windows-7-64-bit-machine-produces-corrupt-files-tp4025341p4025349.html
Sent from the hdf-forum mailing list archive at Nabble.com.
Hi Quincey, thanks for the feedback. A superficial scan of our source doesn't suggest any obvious hid_t leaks; we use RAII patterns wherever we can to avoid such issues.
Needs further investigation at our end, but the urgency has dropped off on this one now we have a "fix" (i.e. all our current test cases pass for our product), so that probably won't happen until we're less busy...
Thanks for the tip on H5Pset_fclose_degree, looks like it could be useful!
Steve
···
-----Original Message-----
From: hdf-forum-bounces@hdfgroup.org [mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of Quincey Koziol
Sent: 30 August 2012 20:05
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] HDF5 in 32 bit environment on Windows 7 64 bit machine produces corrupt files.
Hi Steve,
On Aug 28, 2012, at 9:33 AM, Steve Bissell wrote:
We have an application that produces an HDF5 file using library 1.8.8. It
has been working fine for some time on Windows XP SP 3.
Running the SAME BINARIES on Windows 7 64 bit machine in 32 bit environment
produces some files that are corrupt (but others are fine).
The corruption is reported by h5debug as "unable to find a valid file
signature".
Detailed examination and comparison with a hex editor shows that ALL of the
metadata appears to be missing: the first part of the file is just zeroes,
whereas for the "good" file (produced on the win32 xp box, using same
binaries) contains the hdf5 signature string at the start of the file.
Subsequent differences in the files all seem to point to missing metadata
(tree structure, attributes etc).
Anyone else come across this problem?
Just to be clear: the HDF character sequence that forms the start of the
"\211HDF\r\n\032\n" signature is found in the good file at the beginning,
but is not found ANYWHERE in the bad file.
** UPDATE **
Problem has been solved by calling flush(H5F_SCOPE_GLOBAL) prior to calling
the close() method on the H5File object.
Which raises the following questions ...
the fortran api states:
H5Fclose terminates access to an HDF5 file by flushing all data to storage
and terminating access to the file through file_id.
So does this mean the C++ "close" api behaves differently? If so, why?
Why did the previous code always work ok for XP SP3 but only work for some
files on Windows 7 (32 bit or 32 bit on 64 bit OS) - some difference in the
way Windows handles memory/file flushing between the 2 OS types?
Has there been a change in the behaviour on file close between 1.8.5 and
1.8.8 (I did my original dev work at the time of 1.8.5)?
I'm guessing that you have an HDF5 ID leak in your application somewhere, which is holding the file ID open. You could use the H5Pset_fclose_degree() property to help debug this.
Regards,
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
This mail has originated outside your organization, either from an external partner or the Global Internet.
Keep this in mind if you answer this message.
The information in this e-mail is confidential. The contents may not be disclosed or used by anyone other than the addressee. Access to this e-mail by anyone else is unauthorised.
If you are not the intended recipient, please notify Airbus immediately and delete this e-mail.
Airbus cannot accept any responsibility for the accuracy or completeness of this e-mail as it has been sent over public networks. If you have any concerns over the content of this message or its Accuracy or Integrity, please contact Airbus immediately.
All outgoing e-mails from Airbus are checked using regularly updated virus scanning software but you should take whatever measures you deem to be appropriate to ensure that this message and any attachments are virus free.
