Asynchronous writes with HDF5?

Hdf-forum archives
1

Hello,

I have an application that uses HDF5 and collective I/O to write data, I
wonder if it is possible to write asynchronously (still collectively) and
without to much change in my code? I searched in the H5P part of the
documentation, looking for any flag which would have turned I/O into
asynchronous, but found nothing...
Thank you,

Matthieu Dorier

···

--
Matthieu Dorier
ENS Cachan, Brittany (Computer Science dpt.)
IRISA Rennes, Office E324
http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/

2

Hi Matthieu,

···

On Aug 15, 2011, at 11:25 AM, Matthieu Dorier wrote:

Hello,

I have an application that uses HDF5 and collective I/O to write data, I wonder if it is possible to write asynchronously (still collectively) and without to much change in my code? I searched in the H5P part of the documentation, looking for any flag which would have turned I/O into asynchronous, but found nothing...

  Unfortunately, MPI doesn't (yet) support nonblocking collective I/O operations. We (The HDF Group) are participating in the MPI Forum standard committee, and are proposing this feature be a part of the upcoming MPI 3.0 standard, but that's still going to be about a year away.

  Quincey

3

Thanks for the answer, I didn't know non-blocking collective I/O didn't
exist yet. What about non-blocking independent I/O in HDF5 ?

Matthieu

···

2011/8/16 Quincey Koziol <koziol@hdfgroup.org>

Hi Matthieu,

On Aug 15, 2011, at 11:25 AM, Matthieu Dorier wrote:

> Hello,
>
> I have an application that uses HDF5 and collective I/O to write data, I
wonder if it is possible to write asynchronously (still collectively) and
without to much change in my code? I searched in the H5P part of the
documentation, looking for any flag which would have turned I/O into
asynchronous, but found nothing...

        Unfortunately, MPI doesn't (yet) support nonblocking collective I/O
operations. We (The HDF Group) are participating in the MPI Forum standard
committee, and are proposing this feature be a part of the upcoming MPI 3.0
standard, but that's still going to be about a year away.

       Quincey

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

--
Matthieu Dorier
ENS Cachan, Brittany (Computer Science dpt.)
IRISA Rennes, Office E324
http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/

4

Hi Matthieu,

Thanks for the answer, I didn't know non-blocking collective I/O didn't exist yet. What about non-blocking independent I/O in HDF5?

  We are in the process of adding support for async I/O to HDF5, but it's early yet and we haven't specifically worked on parallel features. Probably in the next year or so…

  Quincey

···

On Aug 18, 2011, at 9:34 AM, Matthieu Dorier wrote:

Matthieu

2011/8/16 Quincey Koziol <koziol@hdfgroup.org>
Hi Matthieu,

On Aug 15, 2011, at 11:25 AM, Matthieu Dorier wrote:

> Hello,
>
> I have an application that uses HDF5 and collective I/O to write data, I wonder if it is possible to write asynchronously (still collectively) and without to much change in my code? I searched in the H5P part of the documentation, looking for any flag which would have turned I/O into asynchronous, but found nothing...

       Unfortunately, MPI doesn't (yet) support nonblocking collective I/O operations. We (The HDF Group) are participating in the MPI Forum standard committee, and are proposing this feature be a part of the upcoming MPI 3.0 standard, but that's still going to be about a year away.

       Quincey

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

--
Matthieu Dorier
ENS Cachan, Brittany (Computer Science dpt.)
IRISA Rennes, Office E324
http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org