Hi all.
I need to perform file locking on hdf5 file, because it is in use by several applications. Right now I implement my own driver and set it via the H5::FileAccPropList::setDriver(). This is not very convenient way to implement locking, because I become dependent on some internal hdf5 structures and not portable between hdf5 versions.
Is there more simple way to perform file locking when HDF5 file is opened?
If there is no such a way, may be you can add some callback which will be called just after file is opened by hdf5 library?
···
--
Best regards, Sergey Spiridonov
Hi Sergey
I have implemented a file locking mechanism which is also platform
independent. The only way to make it work is that you take the file handle
form the hdf driver and put the lock on the first byte. This means that
locking is coordinated between processes that respect this convention. It
is not risk-free though you need to make sure that between opening the file
and locking the file nobody else has written anything. I do this (although
I have to check my cod for details) by waiting for a shared lock on a
different handle, once I have this I go ahead and open the real hdf5
handle, release the first lock and make a new lock. This also implies that
there is only one writer allowed...
HTH
-- dimitris
···
2013/2/19 Sergey Spiridonov <sena@hurd.homeunix.org>
Hi all.
I need to perform file locking on hdf5 file, because it is in use by
several applications. Right now I implement my own driver and set it via
the H5::FileAccPropList::**setDriver(). This is not very convenient way
to implement locking, because I become dependent on some internal hdf5
structures and not portable between hdf5 versions.
Is there more simple way to perform file locking when HDF5 file is opened?
If there is no such a way, may be you can add some callback which will be
called just after file is opened by hdf5 library?
--
Best regards, Sergey Spiridonov
______________________________**_________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/**mailman/listinfo/hdf-forum_**hdfgroup.org<http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org>
Hi Dimitris
I have implemented a file locking mechanism which is also platform
independent. The only way to make it work is that you take the file
handle form the hdf driver and put the lock on the first byte. This
means that locking is coordinated between processes that respect this
convention. It is not risk-free though you need to make sure that
between opening the file and locking the file nobody else has written
anything. I do this (although I have to check my cod for details) by
waiting for a shared lock on a different handle, once I have this I go
ahead and open the real hdf5 handle, release the first lock and make a
new lock. This also implies that there is only one writer allowed...
Why do you do internal lock then? If you do already external lock, this must be sufficient...
External lock is a way to go, only problem can be on read-only filesystems. Thank you for an idea.
I do not think that implementing some callback for locking is a complicated task, but before doing it I want hear opinion of HDF5 developers.
···
On 02/19/2013 07:00 PM, Dimitris Servis wrote:
--
Best regards, Sergey Spiridonov
Hi Sergei
Hi Dimitris
I am not sure what you mean with internal/external lock
One way to implement locking would be the following:
before opening hdf5 file /somepath/somefile.hdf create an external file
/somepath/.somefile.hdf.lock$$**$ and lock it. After closing hdf5 file,
delete and unlock this external file.
I understood you as you did it this way, because you wrote about "waiting
for a shared lock on a different handle".
OK now it's clear. No I did not do it this way. The reason is that you may
not have the rights to create an external lock file. If you can guarantee
you do it is indeed the best solution. That was my first implementation
until I discovered I cannot guarantee the proper user rights all the time.
When I wrote "waiting for a shared lock on a different handle" I meant
another file handle to the same file but not the file handle that HDF5 uses
to write. Unfortunately you cannot lock one handle and write with another
from the same process, at least on Windows if I recall correctly.
Kind Regards
Dimitris Servis
···
2013/2/22 Sergey Spiridonov <sena@hurd.homeunix.org>
On 02/22/2013 11:03 AM, Dimitris Servis wrote:
for the feature. This is understandable if the paying customers do not need
it. Let me put it another way: if I did not have requirements to make it
possible to have the files at the disposal (and mercy) of users, I would
have a process hiding HDF5 from processes and coordinating access.
HTH
-- ds
···
2013/2/22 Sergey Spiridonov <sena@hurd.homeunix.org>
Hi Dimitris
On 02/22/2013 06:11 PM, Dimitris Servis wrote:
When I wrote "waiting for a shared lock on a different handle" I meant
another file handle to the same file but not the file handle that HDF5
uses to write. Unfortunately you cannot lock one handle and write with
another from the same process, at least on Windows if I recall correctly.
Got it, thanks! I will give it a try! Did you also ask HDF5 developers?
Why there is no better way to implement it? I did not find anything in
archive of hdf5 mail list.
AFAIK there is no interest in the matter unless someone is willing to pay
