In the "Single Write Multiple Read" (SWMR)" document it says that your file system needs to be POSIX compliant. Our software runs on Windows 7, 8 and 10. I am finding confusing information about if these O/S's are POSIX compliant.
Anybody know if the SWMR will work on these O/S's?
Hi Andrew,
We do not currently test SWMR on Windows but we would expect it to work on NTFS. Basically we need for write ordering to be preserved, which is usually true for a straightforward local file system. Parallel file systems like GPFS are also generally supported.
Most network file systems (NFS, SMB/Windows file shares, etc.), however, do not guarantee write ordering and are not supported under SWMR.
Dana Robinson
Software Engineer
The HDF Group
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Ryland, Andrew
Sent: Wednesday, March 16, 2016 11:47 AM
To: hdf-forum@lists.hdfgroup.org
Subject: [Hdf-forum] SWMR requirement to be POSIX compliant
In the "Single Write Multiple Read" (SWMR)" document it says that your file system needs to be POSIX compliant. Our software runs on Windows 7, 8 and 10. I am finding confusing information about if these O/S's are POSIX compliant.
Anybody know if the SWMR will work on these O/S's?
We are updating the document for the release to make it more clear. "Posix compliant" will be replaced with “the file system that complies with the POSIX write() semantics”.
As Dana said, the most important issue is write ordering.
Elena
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Elena Pourmal The HDF Group http://hdfgroup.org
1800 So. Oak St., Suite 203, Champaign IL 61820
217.531.6112
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
On Mar 16, 2016, at 11:28 AM, Dana Robinson <derobins@hdfgroup.org<mailto:derobins@hdfgroup.org>> wrote:
Hi Andrew,
We do not currently test SWMR on Windows but we would expect it to work on NTFS. Basically we need for write ordering to be preserved, which is usually true for a straightforward local file system. Parallel file systems like GPFS are also generally supported.
Most network file systems (NFS, SMB/Windows file shares, etc.), however, do not guarantee write ordering and are not supported under SWMR.
Dana Robinson
Software Engineer
The HDF Group
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Ryland, Andrew
Sent: Wednesday, March 16, 2016 11:47 AM
To: hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>
Subject: [Hdf-forum] SWMR requirement to be POSIX compliant
In the “Single Write Multiple Read" (SWMR)” document it says that your file system needs to be POSIX compliant. Our software runs on Windows 7, 8 and 10. I am finding confusing information about if these O/S’s are POSIX compliant.
Anybody know if the SWMR will work on these O/S’s?
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
On the topic of parallel file systems, it is unclear if the PVFS (aka OrangeFS) file system is POSIX-enough to support SWMR. PVFS does not implement full POSIX semantics, for performance reasons, but neither is it as sloppy as NFS is.
==rob
On 03/16/2016 11:28 AM, Dana Robinson wrote:
Hi Andrew,
We do not currently test SWMR on Windows but we would expect it to work
on NTFS. Basically we need for write ordering to be preserved, which is
usually true for a straightforward local file system. Parallel file
systems like GPFS are also generally supported.
We provide a program in the “test" directory of the HDF5 1.10 distribution - twriteorder.c<https://svn.hdfgroup.org/hdf5/branches/hdf5_1_10_0/test/twriteorder.c>. See description<https://svn.hdfgroup.org/hdf5/branches/hdf5_1_10_0/test/SWMR_POSIX_Order_UG.txt>.
On NFS it fails pretty quickly. We couldn’t make it to fail on GPFS. Please see the report<https://svn.hdfgroup.org/hdf5/branches/hdf5_1_10_0/test/POSIX_Order_Write_Test_Report.pdf>.
It would be interesting to run it on PVFS.
Elena
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Elena Pourmal The HDF Group http://hdfgroup.org
1800 So. Oak St., Suite 203, Champaign IL 61820
217.531.6112
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
On Mar 29, 2016, at 11:22 AM, Rob Latham <robl@mcs.anl.gov<mailto:robl@mcs.anl.gov>> wrote:
On 03/16/2016 11:28 AM, Dana Robinson wrote:
Hi Andrew,
We do not currently test SWMR on Windows but we would expect it to work
on NTFS. Basically we need for write ordering to be preserved, which is
usually true for a straightforward local file system. Parallel file
systems like GPFS are also generally supported.
On the topic of parallel file systems, it is unclear if the PVFS (aka OrangeFS) file system is POSIX-enough to support SWMR. PVFS does not implement full POSIX semantics, for performance reasons, but neither is it as sloppy as NFS is.
==rob
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
