Hi all,
I'm wondering what happens internally in parallel HDF5 when two or
more procs have overlapping hyperslab selections, e.g. when each proc
has a layer of ghost zones for a stencil calculation and it tries to
write all of the data including the ghost zones.
Does HDF5 issue redundant read/write requests for the elements in the
overlapping region? Or does it try to shrink the regions until they
are disjoint? In the MPI-IO VFD, will the behavior depend on
collective vs. independent mode?
Thanks
Mark
Thanks Quincey, this is good to know. Mark
···
On Thu, Jul 8, 2010 at 9:43 AM, Quincey Koziol <koziol@hdfgroup.org> wrote:
Hi Mark,
On Jul 8, 2010, at 11:07 AM, Mark Howison wrote:
Hi all,
I'm wondering what happens internally in parallel HDF5 when two or
more procs have overlapping hyperslab selections, e.g. when each proc
has a layer of ghost zones for a stencil calculation and it tries to
write all of the data including the ghost zones.
Does HDF5 issue redundant read/write requests for the elements in the
overlapping region? Or does it try to shrink the regions until they
are disjoint? In the MPI-IO VFD, will the behavior depend on
collective vs. independent mode?
The HDF5 library doesn't check for overlapping selections and the behavior is dependent on the timing of the I/O operations \(for serial and parallel independent I/O\) and whatever the MPI library does \(for parallel collective I/O\), but the MPI standard says that this operation's outcome is undefined \(and thus will likely be subject to timing or process ordering issues\)\.
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
In similar situations, I've wondered the same thing. However, my
experience in 'ghost zoning' in typical FEM simulations is that the
applications tend to go out of there way to ensure that the numerical
values in ghost zones (e.g. zones shared by more than one processor) are
bit-for-bit identical.
In that case, the outcome should be the same, regardless of ultimate
order and timing of operations, right? I mean even if for an 8 byte
double, some of the bytes get written by one MPI task and some by
another but both MPI tasks agree on what those bytes should be, the
result is going to be the same regardless, right?
Mark
···
On Thu, 2010-07-08 at 09:43, Quincey Koziol wrote:
Hi Mark,
On Jul 8, 2010, at 11:07 AM, Mark Howison wrote:
> Hi all,
>
> I'm wondering what happens internally in parallel HDF5 when two or
> more procs have overlapping hyperslab selections, e.g. when each proc
> has a layer of ghost zones for a stencil calculation and it tries to
> write all of the data including the ghost zones.
>
> Does HDF5 issue redundant read/write requests for the elements in the
> overlapping region? Or does it try to shrink the regions until they
> are disjoint? In the MPI-IO VFD, will the behavior depend on
> collective vs. independent mode?
The HDF5 library doesn't check for overlapping selections and the behavior is dependent on the timing of the I/O operations (for serial and parallel independent I/O) and whatever the MPI library does (for parallel collective I/O), but the MPI standard says that this operation's outcome is undefined (and thus will likely be subject to timing or process ordering issues).
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://*mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
Mark C. Miller, Lawrence Livermore National Laboratory
================!!LLNL BUSINESS ONLY!!================
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511
In similar situations, I've wondered the same thing. However, my
experience in 'ghost zoning' in typical FEM simulations is that the
applications tend to go out of there way to ensure that the numerical
values in ghost zones (e.g. zones shared by more than one processor) are
bit-for-bit identical.
In that case, the outcome should be the same, regardless of ultimate
order and timing of operations, right? I mean even if for an 8 byte
double, some of the bytes get written by one MPI task and some by
another but both MPI tasks agree on what those bytes should be, the
result is going to be the same regardless, right?
Probably, but the MPI standard doesn't guarantee it. (And I can't think of a realistic scenario where an MPI implementation would end up with different bytes)
Quincey
···
On Jul 8, 2010, at 12:06 PM, Mark Miller wrote:
Mark
On Thu, 2010-07-08 at 09:43, Quincey Koziol wrote:
Hi Mark,
On Jul 8, 2010, at 11:07 AM, Mark Howison wrote:
Hi all,
I'm wondering what happens internally in parallel HDF5 when two or
more procs have overlapping hyperslab selections, e.g. when each proc
has a layer of ghost zones for a stencil calculation and it tries to
write all of the data including the ghost zones.
Does HDF5 issue redundant read/write requests for the elements in the
overlapping region? Or does it try to shrink the regions until they
are disjoint? In the MPI-IO VFD, will the behavior depend on
collective vs. independent mode?
The HDF5 library doesn't check for overlapping selections and the behavior is dependent on the timing of the I/O operations (for serial and parallel independent I/O) and whatever the MPI library does (for parallel collective I/O), but the MPI standard says that this operation's outcome is undefined (and thus will likely be subject to timing or process ordering issues).
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://*mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
Mark C. Miller, Lawrence Livermore National Laboratory
================!!LLNL BUSINESS ONLY!!================
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
