Parallel H5Read no collective

Hdf-forum archives
1

Hi all,

Im trying to read large amount of data via a H5Read collective operation from one dataset.

Even though I passed the collective flag I see several MPI_File_read_at operation below (could see after an instrumentation with scorep)

So I guess H5Read does not use collective IO.

What I do:

I access one large dataset with all processes.

Each process reads a subset specified via hyperslabs (no overlaps).

The dataset is a compound dataset containing: 1x Int and 5x float values.

In memory I have a vector of struct objects:

struct {

b: int <---- *

c: int

d float[5] <---- *

}

* only theses values should be assigned in memory.

So I use H5Tcreate to create a compound memtype with the needed offsets.

I use HDF5 1.8.15

I would welcome any hint. :slight_smile:

Thanks a lot!

Till

···

a: int

2

Hi Till,

There are a couple of reasons to explain the independent reads:

  1. they could be metadata reads issued by the HDF5 library
  2. The HDF5 library is breaking collective access because it is not supported in your scenario.

To check if 2 is the case and why the library is breaking collective, you could use the following function:
https://www.hdfgroup.org/HDF5/doc/RM/RM_H5P.html#Property-GetMpioNoCollectiveCause

Thanks,
Mohamad

···

From: Hdf-forum <hdf-forum-bounces@lists.hdfgroup.org<mailto:hdf-forum-bounces@lists.hdfgroup.org>> on behalf of Schumann Till <till.schumann@epfl.ch<mailto:till.schumann@epfl.ch>>
Reply-To: hdf-forum <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Date: Tuesday, February 2, 2016 at 2:57 PM
To: hdf-forum <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Cc: Kumbhar Pramod Shivaji <pramod.kumbhar@epfl.ch<mailto:pramod.kumbhar@epfl.ch>>
Subject: [Hdf-forum] Parallel H5Read no collective

Hi all,

Im trying to read large amount of data via a H5Read collective operation from one dataset.

Even though I passed the collective flag I see several MPI_File_read_at operation below (could see after an instrumentation with scorep)

So I guess H5Read does not use collective IO.

What I do:

I access one large dataset with all processes.

Each process reads a subset specified via hyperslabs (no overlaps).

The dataset is a compound dataset containing: 1x Int and 5x float values.

In memory I have a vector of struct objects:

struct {

a: int

b: int <---- *

c: int

d float[5] <---- *

}

* only theses values should be assigned in memory.

So I use H5Tcreate to create a compound memtype with the needed offsets.

I use HDF5 1.8.15

I would welcome any hint. :slight_smile:

Thanks a lot!

Till