'Morning,
While recently running some hastily-written benchmarks, we noticed
H5Fcreate was slow when supplying H5F_ACC_TRUNK and overwriting an
existing GPFS-stored file on Mira up at ALCF. So far as we could
tell, the slowness came from needing to truncate the existing file and
it exhibited a delay that scaled linearly with existing file size.
We didn't expressly hint for GPFS in any way or use
H5Pset_libver_bounds, so apologies if this behavior is known and/or
fixed on the latest releases.
A fix on our end would be to, when overwriting is desired, unlink(2)
the path before using H5Fcreate. This should improve the wall time
greatly but will not look like an overwrite from the filesystem
perspective.
Is there some savvy way to speed up the truncation in a manner that
doesn't use unlink(2)? I'm having trouble deciphering H5F_open within
the library source's H5F.c to see if we're hitting some sort of edge
case in the truncation logic.
Thanks,
Rhys