File locking of parallel HDF5 on lustre without file locking support

Hi everyone!
I'm trying to run simulations at a nehalem-cluster which is using a lustre
system for I/O. My code uses parallel HDF5 output writing timestep-groups
with 3D data in one file, thus leading to one big outputfile when the
simulation is done.
The problem is, that parallel HDF5 needs file locking which must be provided
by the lustre via some daemon or something else. This daemon will lead to
massive performance losses of the lustre up to 50%. This is why the
cluster-admins refuse to enable file locking at their lustre system.
Consequently I will not be able to write anything. When I try it stops with:

File locking failed in ADIOI_Set_lock(fd 18,cmd F_SETLKW/7,type
F_WRLCK/1,whence 0) with return value FFFFFFFF and errno 26.
If the file system is NFS, you need to use NFS version 3, ensure that the
lockd daemon is running on all the machines, and mount the directory with
the 'noac' option (no attribute caching).
ADIOI_Set_lock:: Function not implemented
ADIOI_Set_lock:offset 6488, length 96

and so on.
Is there any workaround for this or is HDF5 reliant on file locking
respectively? Otherwise I will be not able to use this cluster.
Thanks and best regards
Sebastian

···

--
View this message in context: http://hdf-forum.184993.n3.nabble.com/File-locking-of-parallel-HDF5-on-lustre-without-file-locking-support-tp2553896p2553896.html
Sent from the hdf-forum mailing list archive at Nabble.com.

The file locking comes from the MPI-IO layer.

You can disable data sieving at the MPI-IO layer and get around this
problem.

through an HDF5 property list you can set the MPI-IO hints
"romio_ds_read" to "disable" and "romio_ds_write" to "disable".

==rob

···

On Tue, Feb 22, 2011 at 09:24:10AM -0800, Meisenmann wrote:

and so on.
Is there any workaround for this or is HDF5 reliant on file locking
respectively? Otherwise I will be not able to use this cluster.
Thanks and best regards

--
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA

Oh, and in addition to turning off data sieving in MPI-IO, be sure to
turn on collective I/O in HDF5.

If you are on a Cray, be sure to use MPT-3.2 or newer.

If you are on a linux cluster, use MPICH2-1.3.1 or newer.

==rob

···

On Tue, Feb 22, 2011 at 09:24:10AM -0800, Meisenmann wrote:

and so on.
Is there any workaround for this or is HDF5 reliant on file locking
respectively? Otherwise I will be not able to use this cluster.

--
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA