Hello!
Many (all?) contemporary OSes have notion of "Temporary file" -- a file that is automatically deleted when the last handle to it is closed (see http://man7.org/linux/man-pages/man3/tmpfile.3.html, FILE_FLAG_DELETE_ON_CLOSE in http://msdn.microsoft.com/en-us/library/windows/desktop/aa363858(v=vs.85).aspx).
Currently it is not possible to specify that FH5Fcreate should open the file as temporary, nor it is possible to pass existing file handle. However I believe that temp HDF5 files would be valuable feature:
1) Robust: file is deleted even if program crashes/ends unexpectedly.
2) Easy-to-use: file is deleted when but not earlier than the last reference to a file object is closed.
Are there any plans to support temp files in HDF5 library, or maybe there is already some way to create them in released version?
Best wishes,
Andrey Paramonov
···
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
Hello Andrey,
This sounds like a good idea. Something like this already exists, see http://www.hdfgroup.org/HDF5/doc1.6/UG/08_TheFile.html and look for "8.8 Buffered temporary files in memory -- CORE driver"
If 'in memory' files are too restrictive for you then you could do as Elena suggested and modify a file driver.
The old technique for doing this under unix-like systems was to *open* or *fopen* a file and then *unlink* the file. The open file handle could still be used and when the program exited for what ever reason the disk space was reclaimed. This would be slightly less robust then the *tmpfile(3)* you reference, but probably good enough for HDF5-type applications.
Unfortunately the HDF5 documentation is ambiguous about whether or not the underlying Unix file handle associated with a HDF5 open file object is always maintained by the library or if it ever gets closed and then reopened by the library during the course of normal processing.
If you need this feature immediately I recommend you test this out as it is really simple to implement.
Cheers,
--dan
···
On 04/16/14 04:21, ??? ??? wrote:
Hello!
Many (all?) contemporary OSes have notion of "Temporary file" -- a file that is automatically deleted when the last handle to it is closed (see http://man7.org/linux/man-pages/man3/tmpfile.3.html, FILE_FLAG_DELETE_ON_CLOSE in http://msdn.microsoft.com/en-us/library/windows/desktop/aa363858(v=vs.85).aspx).
Currently it is not possible to specify that FH5Fcreate should open the file as temporary, nor it is possible to pass existing file handle. However I believe that temp HDF5 files would be valuable feature:
1) Robust: file is deleted even if program crashes/ends unexpectedly.
2) Easy-to-use: file is deleted when but not earlier than the last reference to a file object is closed.
Are there any plans to support temp files in HDF5 library, or maybe there is already some way to create them in released version?
Best wishes,
Andrey Paramonov
--
Daniel Kahn
Science Systems and Applications Inc.
301-867-2162
18.04.2014 18:19, Daniel Kahn пишет:
Hello Andrey,
This sounds like a good idea. Something like this already exists, see
http://www.hdfgroup.org/HDF5/doc1.6/UG/08_TheFile.html and look for "8.8
Buffered temporary files in memory -- CORE driver"
It looked like a good idea at the first glance, but the CORE driver requires data to fill into memory, which is not the case for my applications, unfortunately.
The old technique for doing this under unix-like systems was to *open*
or *fopen* a file and then *unlink* the file. The open file handle could
still be used and when the program exited for what ever reason the disk
space was reclaimed. This would be slightly less robust then the
*tmpfile(3)* you reference, but probably good enough for HDF5-type
applications.
Yes, it would be the way-to-go, but my code must run on Windows 
Unfortunately the HDF5 documentation is ambiguous about whether or not
the underlying Unix file handle associated with a HDF5 open file object
is always maintained by the library or if it ever gets closed and then
reopened by the library during the course of normal processing.
Hmm, I didn't think about it (I assumed H5Fopen does create low-level FILE handle immediately). But isn't it driver-specific then?
I believe the easiest way to implement the desired feature would be to introduce new flag e.g. H5_ACC_TEMP to be passed to H5Fcreate. It would require very little changes to HDF5 code. Another way I see is to create special TEMPFILE driver. What do you think?
Best wishes,
Andrey Paramonov
···
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
