Hi Dimitris and everyone,
after having implemented a nice file locking mechanism that locks on lock
files instead of the HDF5 file and manages to correctly coordinate access
between computers on shared drives, I realized that this mechanism is
actually not much good... the reason being that I expect the use case where
the user wants to read a file but does not have write permissions to the
directory cannot be ruled out. Obviously without write permissions to the
directory one cannot create a lock and read from that file which is quite
controversial in its own right.
Could you place the lock file into some other place that is writable by
everyone, such as /tmp? You'd just have to encode the position of the HDF5
file into the file name of the lock file in a common temp directory.
In such a scheme, a process desiring access to /dir1/dir2/datafile.h5
would first encode that file name e.g. to datafile.h5_fse243t23tegf and
check if a file by that name exists in the common temp directory.
I left the file name, datafile.h5, unencoded for human-readability, whereas
the directory path would be encoded to a string like "fse243t23tegf" produced
by some hash-encoding algorithm, so that every target file path would
correspond to a unique lock file name in the common temp directory.
Best Regards,
Vesa Paatero
···
Dimitris wrote on 2009-11-12:
------------------------------
Date: Thu, 12 Nov 2009 08:29:57 +0100
From: Dimitris Servis <servisster@gmail.com>
To: HDF forum <hdf-forum@hdfgroup.org>
Subject: [Hdf-forum] File locking revisited
Message-ID:
<65b8aaff0911112329y3506ff6fu1984e6b7d4c3d169@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi all,
after having implemented a nice file locking mechanism that locks on lock
files instead of the HDF5 file and manages to correctly coordinate access
between computers on shared drives, I realized that this mechanism is
actually not much good... the reason being that I expect the use case where
the user wants to read a file but does not have write permissions to the
directory cannot be ruled out. Obviously without write permissions to the
directory one cannot create a lock and read from that file which is quite
controversial in its own right. I therefore plan to turn away from the
separate lock file. I am reluctant on locking the file itself, as especially
on Linux any lock on a handle to a file opened by HDF5 library using fcntl
will be released as soon as the library releases one of its own locks. I do
not know to which extent this is probable. One question is also if it would
be fine to lock the file from a new file handle opened on the file using
system functions for exactly this purpose while the file is already opened
by HDF5. Any thoughts on that would be highly appreciated.
Another idea that came up to me while over the scratchpad again is this:
what I want is multiple processes being able to read from the file reliably
but not what another process is writing. So for example processes want to
read /foo/bar/one and /foo/bar/two datasets while another process is writing
in /foo/bar/three. In my case datasets one and two are completed and there's
no chance of a process wanting to write anything there. So these can safely
be read while a process is writing something else I guess. The only danger
lies with the case where reading processes realize that all the three
datasets exist based on the metadata but "three" is still being written. I
could place an attribute on "three" that would indicate that this is still
being written. This has the drawback that the attribute has to be removed
and all reading has to start by checking this attribute. Also I cannot
exclude the case that while a process is reading dataset "three" which did
not have the special attribute in the beginning, another process would like
to continue writing there. Maybe this is safe?
A final approach is this: start writing the new collection of data somewhere
else, like /tmp/hostname/process/thread/timestamp/three. A process can work
there for as much as it wants while other processes open and close the file
or even keep it open and read in other areas. When the writer finishes, it
links /foo/bar/three to /tmp/hostname/process/thread/timestamp/three in one
atomic (hopefully) operation. Any processes opening or reopening the file,
only then will they realize there's a new dataset and read it safely while
the writer is on to his next task. If this works, it seems to me as the
safest and most portable idea. The risk here is that if the writing process
goes down, you're left with some garbage in /tmp...
Any other ideas/suggestions are mostly welcome.
Thanks a lot!
-- dimitris