I was wondering where I could find some more technical details about
concurrent reading/writing.
The FAQ discusses it briefly
(http://www.hdfgroup.org/hdf5-quest.html#grdwt):
<excerpt>
It is possible for multiple processes to read an HDF5 file when it is
being written to, and still read correct data. (The following steps
should be followed, EVEN IF the dataset that is being written to is
different than the datasets that are read.)
Here's what needs to be done:
* Call H5Fflush() from the writing process.
* The writing process _must_ wait until either a copy of the file is
made for the reading process, or the reading process is done accessing
the file (so that more data isn't written to the file, giving the reader
an inconsistent view of the file's state).
* The reading process _must_ open the file (it cannot have the file
open before the writing process flushes its information, or it runs the
risk of having its data cached in memory being incorrect with respect to
the state of the file) and read whatever information it wants.
* The reading process must close the file.
* The writing process may now proceed to write more data to the
file.
There must also be some mechanism for the writing process to signal the
reading process that the file is ready for reading and some way for the
reading process to signal the writing process that the file may be
written to again.
</excerpt>
Could someone elaborate in a more technical manner? e.g. SWMR
(single-writer multiple-reader) can occur if the following is true (not
sure if I have this correct; I use "process" rather than "threads" here
& am not sure if HDF5 in-memory caches have thread affinity):
1. At all times the file is in one of the following states:
(a) unmodified
(b) modified (written to, but not flushed)
2. In the unmodified state, zero or more processes may have the file
open. No process may write to the data.
3. In the modified state, exactly one process may have the file open.
This is the process that can write to it.
4. A successful transition from the unmodified state -> modified state
takes place when exactly one process has the file open and begins
writing to it.
5. A successful transition from the modified state -> unmodified state
takes place when the process that has written to the file completes a
successful call to H5Fflush().
The facilities to ensure that only one process has the file open for (4)
above are not provided by the HDF5 library and must be provided by
OS-specific facilities e.g. mutexes/semaphores/messaging/etc.
···
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.