HDF lib incompatible with HDF file spec?

HDF5 Library
21

herr_t H5Sselect_none(hid_t space_id);
https://support.hdfgroup.org/HDF5/hdf5-quest.html#par-nodata

Jarom

···

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Rafal Lichwala
Sent: Thursday, September 28, 2017 6:52 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] high level API for parallel version of HDF5 library

Hi,

Thank you for an answer and example codes.
Creating metadata (groups, datasets) is clear now and works fine, but I've got the last doubt: what in case I'm running 4 MPI processes but only 3 of them have some data to be written to the given dataset.
Since the H5Dwrite() call is in collective mode, my program hangs...
How to solve this?

Regards,
Rafal

W dniu 2017-09-27 o 22:50, Nelson, Jarom pisze:

Calls that affect the metadata need to be collective so that each
process has a consistent view of what the file metadata should be.

Collective HDF5 Calls in Parallel

Something like this (or the attached):

plist_id = H5Pcreate(H5P_FILE_ACCESS);

H5Pset_fapl_mpio(plist_id, comm, info);

H5Pset_all_coll_metadata_ops( plist_id, true );

file_id = H5Fcreate(H5FILE_NAME, H5F_ACC_TRUNC, H5P_DEFAULT,
plist_id);

H5Pclose(plist_id);

for(int procid = 0; i < mpi_size; ++i) {

hid_t gr_id = H5Gcreate(file_id, std::to_string(procid).c_str(),
H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);

H5Gclose(gr_id);

}

H5Fclose(file_id);

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On
Behalf Of Rafal Lichwala
Sent: Wednesday, September 27, 2017 12:32 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] high level API for parallel version of HDF5
library

Hi Barbara, Hi All,

Thank you for your answer. That's clear now about H5TBmake_table()
call, but...

H5Gcreate() in not a high level API, isn't it?

So why I cannot use it in parallel processes?

Maybe I'm just doing something wrong, so could you please provide me a
short example how to create a set of groups (each one is the process

number) running 4 parallel MPI processes? You can limit the example
code to the sequence of HDF5 calls only...

My current code works fine for just one process, but when I try it for
2 (or more) parallel processes the result file is corrupted:

plist_id = H5Pcreate(H5P_FILE_ACCESS);

H5Pset_fapl_mpio(plist_id, comm, info);

H5Pset_all_coll_metadata_ops( plist_id, true ); file_id =
H5Fcreate(H5FILE_NAME, H5F_ACC_TRUNC, H5P_DEFAULT, plist_id);
H5Pclose(plist_id); hid_t gr_id = H5Gcreate(file_id,
std::to_string(procid).c_str(), H5P_DEFAULT, H5P_DEFAULT,
H5P_DEFAULT); H5Gclose(gr_id); H5Fclose(file_id);

Best regards,

Rafal

W dniu 2017-09-25 o 22:20, Barbara Jones pisze:

> Hi Rafal,

>

> No, the HDF5 High Level APIs are not supported in the parallel
version of HDF5.

>

> -Barbara

> help@hdfgroup.org <mailto:help@hdfgroup.org>

>

> -----Original Message-----

> From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On
Behalf Of Rafal Lichwala

> Sent: Monday, September 18, 2017 8:53 AM

> To: hdf-forum@lists.hdfgroup.org
<mailto:hdf-forum@lists.hdfgroup.org>

> Subject: [Hdf-forum] high level API for parallel version of HDF5
library

>

> Hi,

>

> Can I use high level API function calls (H5TBmake_table(...)) in
parallel version of the HDF5 library?

> There are no property list parameters for that function calls...

>

> Regards,

> Rafal

>

>

> _______________________________________________

> Hdf-forum is for HDF software users discussion.

> Hdf-forum@lists.hdfgroup.org <mailto:Hdf-forum@lists.hdfgroup.org>

>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.or
g

> Twitter: https://twitter.com/hdf5

>

> _______________________________________________

> Hdf-forum is for HDF software users discussion.

> Hdf-forum@lists.hdfgroup.org <mailto:Hdf-forum@lists.hdfgroup.org>

>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.or
g

> Twitter: https://twitter.com/hdf5

>

_______________________________________________

Hdf-forum is for HDF software users discussion.

Hdf-forum@lists.hdfgroup.org <mailto:Hdf-forum@lists.hdfgroup.org>

http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.or
g

Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.or
g
Twitter: https://twitter.com/hdf5

--

***
Rafał Lichwała
Poznańskie Centrum Superkomputerowo-Sieciowe ul. Jana Pawła II nr 10
61-139 Poznań
e-mail: syriusz@man.poznan.pl
***

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

22

Hi Jarom, Hi All,

Thank you very much for the concrete answer! But...
What in case the given process (which has no data to write) does not know the structure of the given dataset (no data => no info about structure), so it cannot produce a proper datatype for collective H5Dwrite() call - even if "space" is properly marked by H5Sselect_none() ?...
It seems H5Dwrite() requires datatype also to be identical in all processes during collective call...
I cannot use H5Tcommit() to share the datatype between processes in file, because this function is also collective...
Can you see any solution for this problem?

Best regards,
Rafal

W dniu 2017-09-28 o 16:10, Nelson, Jarom pisze:

···

herr_t H5Sselect_none(hid_t space_id);
https://support.hdfgroup.org/HDF5/hdf5-quest.html#par-nodata
https://www.hdfgroup.org/2015/08/parallel-io-with-hdf5/
https://support.hdfgroup.org/HDF5/PHDF5/

Jarom

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Rafal Lichwala
Sent: Thursday, September 28, 2017 6:52 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] high level API for parallel version of HDF5 library

Hi,

Thank you for an answer and example codes.
Creating metadata (groups, datasets) is clear now and works fine, but I've got the last doubt: what in case I'm running 4 MPI processes but only 3 of them have some data to be written to the given dataset.
Since the H5Dwrite() call is in collective mode, my program hangs...
How to solve this?

Regards,
Rafal

W dniu 2017-09-27 o 22:50, Nelson, Jarom pisze:

Calls that affect the metadata need to be collective so that each
process has a consistent view of what the file metadata should be.

https://support.hdfgroup.org/HDF5/doc/RM/CollectiveCalls.html

Something like this (or the attached):

plist_id = H5Pcreate(H5P_FILE_ACCESS);

H5Pset_fapl_mpio(plist_id, comm, info);

H5Pset_all_coll_metadata_ops( plist_id, true );

file_id = H5Fcreate(H5FILE_NAME, H5F_ACC_TRUNC, H5P_DEFAULT,
plist_id);

H5Pclose(plist_id);

for(int procid = 0; i < mpi_size; ++i) {

   hid_t gr_id = H5Gcreate(file_id, std::to_string(procid).c_str(),
H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);

   H5Gclose(gr_id);

}

H5Fclose(file_id);

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On
Behalf Of Rafal Lichwala
Sent: Wednesday, September 27, 2017 12:32 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] high level API for parallel version of HDF5
library

Hi Barbara, Hi All,

Thank you for your answer. That's clear now about H5TBmake_table()
call, but...

H5Gcreate() in not a high level API, isn't it?

So why I cannot use it in parallel processes?

Maybe I'm just doing something wrong, so could you please provide me a
short example how to create a set of groups (each one is the process

number) running 4 parallel MPI processes? You can limit the example
code to the sequence of HDF5 calls only...

My current code works fine for just one process, but when I try it for
2 (or more) parallel processes the result file is corrupted:

plist_id = H5Pcreate(H5P_FILE_ACCESS);

H5Pset_fapl_mpio(plist_id, comm, info);

H5Pset_all_coll_metadata_ops( plist_id, true ); file_id =
H5Fcreate(H5FILE_NAME, H5F_ACC_TRUNC, H5P_DEFAULT, plist_id);
H5Pclose(plist_id); hid_t gr_id = H5Gcreate(file_id,
std::to_string(procid).c_str(), H5P_DEFAULT, H5P_DEFAULT,
H5P_DEFAULT); H5Gclose(gr_id); H5Fclose(file_id);

Best regards,

Rafal

W dniu 2017-09-25 o 22:20, Barbara Jones pisze:

  > Hi Rafal,

  >

  > No, the HDF5 High Level APIs are not supported in the parallel
version of HDF5.

  >

  > -Barbara

  > help@hdfgroup.org <mailto:help@hdfgroup.org>

  >

  > -----Original Message-----

  > From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On
Behalf Of Rafal Lichwala

  > Sent: Monday, September 18, 2017 8:53 AM

  > To: hdf-forum@lists.hdfgroup.org
<mailto:hdf-forum@lists.hdfgroup.org>

  > Subject: [Hdf-forum] high level API for parallel version of HDF5
library

  >

  > Hi,

  >

  > Can I use high level API function calls (H5TBmake_table(...)) in
parallel version of the HDF5 library?

  > There are no property list parameters for that function calls...

  >

  > Regards,

  > Rafal

  >

  >

  > _______________________________________________

  > Hdf-forum is for HDF software users discussion.

  > Hdf-forum@lists.hdfgroup.org <mailto:Hdf-forum@lists.hdfgroup.org>

  >
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.or
g

  > Twitter: https://twitter.com/hdf5

  >

  > _______________________________________________

  > Hdf-forum is for HDF software users discussion.

  > Hdf-forum@lists.hdfgroup.org <mailto:Hdf-forum@lists.hdfgroup.org>

  >
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.or
g

  > Twitter: https://twitter.com/hdf5

  >

_______________________________________________

Hdf-forum is for HDF software users discussion.

Hdf-forum@lists.hdfgroup.org <mailto:Hdf-forum@lists.hdfgroup.org>

http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.or
g

Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.or
g
Twitter: https://twitter.com/hdf5

23

Hi Barbara,

Of course! :slight_smile: Sorry for the false alarm and thank you for an answer!

Best regards,
Rafal

W dniu 2017-10-02 o 20:17, Barbara Jones pisze:

···

Hi Rafal,

It looks like there may be a small typo in the name of the API that you are specifying ("mpio" vs "mpi").
I found H5Fset_mpi_atomicity (but not H5Fset_mpio_atomicity). Is that what you are looking for?

-Barbara
help@hdfgroup.org

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Rafal Lichwala
Sent: Wednesday, September 27, 2017 7:57 AM
To: hdf-forum@lists.hdfgroup.org
Subject: [Hdf-forum] H5Fset_mpio_atomicity missing in 1.10 release

Hi,

The library documentation says that H5Fset_mpio_atomicity() is available since 1.8.9 release, but I cannot find this in the latest 1.10 (built with parallel mode).
How to solve this?

Regards,
Rafal

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

24

Hi again,

I solved the issue with data structure - this is now known for each process, so I can properly create groups and datasets collectively in all parallel processes.
Now I've got another problem:
When I create all datasets collectively, initially they have "0" size and "unlimited" max-size (so dataset is chunked), as I don't know if there will be any data to write by the given process.
Then, when the given process knows what data should write to the given dataset, I try to call H5Dset_extent(), but it seems this function must be also called collectively (!)...
That means each process should know what is the size of data to write in all other processes. Am I correct?
How to solve that issue?

I found very similar issue described a long time ago (December 2012) here:
https://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2012-December/006322.html

and my current problem is properly described here in this thread:
https://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2012-December/006337.html

Further in this thread someone said: "This is a limitation currently in the HDF5 standard that we plan to relax in the future."
Is this already solved now (after 5 years of development)?

Best regards,
Rafal

W dniu 2017-09-29 o 13:07, Rafal Lichwala pisze:

···

Hi Jarom, Hi All,

Thank you very much for the concrete answer! But...
What in case the given process (which has no data to write) does not know the structure of the given dataset (no data => no info about structure), so it cannot produce a proper datatype for collective H5Dwrite() call - even if "space" is properly marked by H5Sselect_none() ?...
It seems H5Dwrite() requires datatype also to be identical in all processes during collective call...
I cannot use H5Tcommit() to share the datatype between processes in file, because this function is also collective...
Can you see any solution for this problem?

Best regards,
Rafal

W dniu 2017-09-28 o 16:10, Nelson, Jarom pisze:

herr_t H5Sselect_none(hid_t space_id);
https://support.hdfgroup.org/HDF5/hdf5-quest.html#par-nodata
https://www.hdfgroup.org/2015/08/parallel-io-with-hdf5/
https://support.hdfgroup.org/HDF5/PHDF5/

Jarom

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Rafal Lichwala
Sent: Thursday, September 28, 2017 6:52 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] high level API for parallel version of HDF5 library

Hi,

Thank you for an answer and example codes.
Creating metadata (groups, datasets) is clear now and works fine, but I've got the last doubt: what in case I'm running 4 MPI processes but only 3 of them have some data to be written to the given dataset.
Since the H5Dwrite() call is in collective mode, my program hangs...
How to solve this?

Regards,
Rafal

W dniu 2017-09-27 o 22:50, Nelson, Jarom pisze:

Calls that affect the metadata need to be collective so that each
process has a consistent view of what the file metadata should be.

https://support.hdfgroup.org/HDF5/doc/RM/CollectiveCalls.html

Something like this (or the attached):

plist_id = H5Pcreate(H5P_FILE_ACCESS);

H5Pset_fapl_mpio(plist_id, comm, info);

H5Pset_all_coll_metadata_ops( plist_id, true );

file_id = H5Fcreate(H5FILE_NAME, H5F_ACC_TRUNC, H5P_DEFAULT,
plist_id);

H5Pclose(plist_id);

for(int procid = 0; i < mpi_size; ++i) {

hid\_t gr\_id = H5Gcreate\(file\_id, std::to\_string\(procid\)\.c\_str\(\),

H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);

H5Gclose\(gr\_id\);

}

H5Fclose(file_id);

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On
Behalf Of Rafal Lichwala
Sent: Wednesday, September 27, 2017 12:32 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] high level API for parallel version of HDF5
library

Hi Barbara, Hi All,

Thank you for your answer. That's clear now about H5TBmake_table()
call, but...

H5Gcreate() in not a high level API, isn't it?

So why I cannot use it in parallel processes?

Maybe I'm just doing something wrong, so could you please provide me a
short example how to create a set of groups (each one is the process

number) running 4 parallel MPI processes? You can limit the example
code to the sequence of HDF5 calls only...

My current code works fine for just one process, but when I try it for
2 (or more) parallel processes the result file is corrupted:

plist_id = H5Pcreate(H5P_FILE_ACCESS);

H5Pset_fapl_mpio(plist_id, comm, info);

H5Pset_all_coll_metadata_ops( plist_id, true ); file_id =
H5Fcreate(H5FILE_NAME, H5F_ACC_TRUNC, H5P_DEFAULT, plist_id);
H5Pclose(plist_id); hid_t gr_id = H5Gcreate(file_id,
std::to_string(procid).c_str(), H5P_DEFAULT, H5P_DEFAULT,
H5P_DEFAULT); H5Gclose(gr_id); H5Fclose(file_id);

Best regards,

Rafal

W dniu 2017-09-25 o 22:20, Barbara Jones pisze:

> Hi Rafal,

>

> No, the HDF5 High Level APIs are not supported in the parallel
version of HDF5.

>

> -Barbara

> help@hdfgroup.org <mailto:help@hdfgroup.org>

>

> -----Original Message-----

> From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On
Behalf Of Rafal Lichwala

> Sent: Monday, September 18, 2017 8:53 AM

> To: hdf-forum@lists.hdfgroup.org
<mailto:hdf-forum@lists.hdfgroup.org>

> Subject: [Hdf-forum] high level API for parallel version of HDF5
library

>

> Hi,

>

> Can I use high level API function calls (H5TBmake_table(...)) in
parallel version of the HDF5 library?

> There are no property list parameters for that function calls...

>

> Regards,

> Rafal

>

>

> _______________________________________________

> Hdf-forum is for HDF software users discussion.

> Hdf-forum@lists.hdfgroup.org <mailto:Hdf-forum@lists.hdfgroup.org>

>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.or
g

> Twitter: https://twitter.com/hdf5

>

> _______________________________________________

> Hdf-forum is for HDF software users discussion.

> Hdf-forum@lists.hdfgroup.org <mailto:Hdf-forum@lists.hdfgroup.org>

>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.or
g

> Twitter: https://twitter.com/hdf5

>

_______________________________________________

Hdf-forum is for HDF software users discussion.

Hdf-forum@lists.hdfgroup.org <mailto:Hdf-forum@lists.hdfgroup.org>

http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.or
g

Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.or
g
Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

25

Since the dataset extent is in the metadata, it will need to be a collective call, and you'll have to find some way to gather that information in order to make the call collectively. You can MPI_Gather an array of dataset sizes, each rank filling in the size of data to write for their rank, and then setting the extent of each dataset (collectively) from that array. Or you could MPI_Reduce a maximum extent, and set the extent of each dataset to the maximum.

I don't think you'll be able to get around setting the extent collectively in the current versions of the library. It doesn't look like the plan to relax this constraint was pushed forward.

Jarom

···

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Rafal Lichwala
Sent: Wednesday, October 4, 2017 3:40 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] high level API for parallel version of HDF5 library

Hi again,

I solved the issue with data structure - this is now known for each process, so I can properly create groups and datasets collectively in all parallel processes.
Now I've got another problem:
When I create all datasets collectively, initially they have "0" size and "unlimited" max-size (so dataset is chunked), as I don't know if there will be any data to write by the given process.
Then, when the given process knows what data should write to the given dataset, I try to call H5Dset_extent(), but it seems this function must be also called collectively (!)...
That means each process should know what is the size of data to write in all other processes. Am I correct?
How to solve that issue?

I found very similar issue described a long time ago (December 2012) here:
https://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2012-December/006322.html

and my current problem is properly described here in this thread:
https://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2012-December/006337.html

Further in this thread someone said: "This is a limitation currently in the HDF5 standard that we plan to relax in the future."
Is this already solved now (after 5 years of development)?

Best regards,
Rafal

W dniu 2017-09-29 o 13:07, Rafal Lichwala pisze:

Hi Jarom, Hi All,

Thank you very much for the concrete answer! But...
What in case the given process (which has no data to write) does not
know the structure of the given dataset (no data => no info about
structure), so it cannot produce a proper datatype for collective
H5Dwrite() call - even if "space" is properly marked by
H5Sselect_none() ?...
It seems H5Dwrite() requires datatype also to be identical in all
processes during collective call...
I cannot use H5Tcommit() to share the datatype between processes in
file, because this function is also collective...
Can you see any solution for this problem?

Best regards,
Rafal

W dniu 2017-09-28 o 16:10, Nelson, Jarom pisze:

herr_t H5Sselect_none(hid_t space_id);
https://support.hdfgroup.org/HDF5/hdf5-quest.html#par-nodata
https://www.hdfgroup.org/2015/08/parallel-io-with-hdf5/
https://support.hdfgroup.org/HDF5/PHDF5/

Jarom

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On
Behalf Of Rafal Lichwala
Sent: Thursday, September 28, 2017 6:52 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] high level API for parallel version of HDF5
library

Hi,

Thank you for an answer and example codes.
Creating metadata (groups, datasets) is clear now and works fine, but
I've got the last doubt: what in case I'm running 4 MPI processes but
only 3 of them have some data to be written to the given dataset.
Since the H5Dwrite() call is in collective mode, my program hangs...
How to solve this?

Regards,
Rafal

W dniu 2017-09-27 o 22:50, Nelson, Jarom pisze:

Calls that affect the metadata need to be collective so that each
process has a consistent view of what the file metadata should be.

https://support.hdfgroup.org/HDF5/doc/RM/CollectiveCalls.html

Something like this (or the attached):

plist_id = H5Pcreate(H5P_FILE_ACCESS);

H5Pset_fapl_mpio(plist_id, comm, info);

H5Pset_all_coll_metadata_ops( plist_id, true );

file_id = H5Fcreate(H5FILE_NAME, H5F_ACC_TRUNC, H5P_DEFAULT,
plist_id);

H5Pclose(plist_id);

for(int procid = 0; i < mpi_size; ++i) {

hid\_t gr\_id = H5Gcreate\(file\_id, std::to\_string\(procid\)\.c\_str\(\),

H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);

H5Gclose\(gr\_id\);

}

H5Fclose(file_id);

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On
Behalf Of Rafal Lichwala
Sent: Wednesday, September 27, 2017 12:32 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] high level API for parallel version of HDF5
library

Hi Barbara, Hi All,

Thank you for your answer. That's clear now about H5TBmake_table()
call, but...

H5Gcreate() in not a high level API, isn't it?

So why I cannot use it in parallel processes?

Maybe I'm just doing something wrong, so could you please provide me
a short example how to create a set of groups (each one is the
process

number) running 4 parallel MPI processes? You can limit the example
code to the sequence of HDF5 calls only...

My current code works fine for just one process, but when I try it
for
2 (or more) parallel processes the result file is corrupted:

plist_id = H5Pcreate(H5P_FILE_ACCESS);

H5Pset_fapl_mpio(plist_id, comm, info);

H5Pset_all_coll_metadata_ops( plist_id, true ); file_id =
H5Fcreate(H5FILE_NAME, H5F_ACC_TRUNC, H5P_DEFAULT, plist_id);
H5Pclose(plist_id); hid_t gr_id = H5Gcreate(file_id,
std::to_string(procid).c_str(), H5P_DEFAULT, H5P_DEFAULT,
H5P_DEFAULT); H5Gclose(gr_id); H5Fclose(file_id);

Best regards,

Rafal

W dniu 2017-09-25 o 22:20, Barbara Jones pisze:

> Hi Rafal,

>

> No, the HDF5 High Level APIs are not supported in the parallel
version of HDF5.

>

> -Barbara

> help@hdfgroup.org <mailto:help@hdfgroup.org>

>

> -----Original Message-----

> From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On
Behalf Of Rafal Lichwala

> Sent: Monday, September 18, 2017 8:53 AM

> To: hdf-forum@lists.hdfgroup.org
<mailto:hdf-forum@lists.hdfgroup.org>

> Subject: [Hdf-forum] high level API for parallel version of HDF5
library

>

> Hi,

>

> Can I use high level API function calls (H5TBmake_table(...)) in
parallel version of the HDF5 library?

> There are no property list parameters for that function calls...

>

> Regards,

> Rafal

>

>

> _______________________________________________

> Hdf-forum is for HDF software users discussion.

> Hdf-forum@lists.hdfgroup.org
<mailto:Hdf-forum@lists.hdfgroup.org>

>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup\.
or
g

> Twitter: https://twitter.com/hdf5

>

> _______________________________________________

> Hdf-forum is for HDF software users discussion.

> Hdf-forum@lists.hdfgroup.org
<mailto:Hdf-forum@lists.hdfgroup.org>

>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup\.
or
g

> Twitter: https://twitter.com/hdf5

>

_______________________________________________

Hdf-forum is for HDF software users discussion.

Hdf-forum@lists.hdfgroup.org <mailto:Hdf-forum@lists.hdfgroup.org>

http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup\.
or
g

Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup\.
or
g
Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.or
g
Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

26

Yep, it seems I have to use MPI_Allgather for that purposes...
Thank you for all comments, suggestions and help on this forum!

Regards,
Rafal

W dniu 2017-10-06 o 00:34, Nelson, Jarom pisze:

···

Since the dataset extent is in the metadata, it will need to be a collective call, and you'll have to find some way to gather that information in order to make the call collectively. You can MPI_Gather an array of dataset sizes, each rank filling in the size of data to write for their rank, and then setting the extent of each dataset (collectively) from that array. Or you could MPI_Reduce a maximum extent, and set the extent of each dataset to the maximum.

I don't think you'll be able to get around setting the extent collectively in the current versions of the library. It doesn't look like the plan to relax this constraint was pushed forward.

Jarom

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Rafal Lichwala
Sent: Wednesday, October 4, 2017 3:40 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] high level API for parallel version of HDF5 library

Hi again,

I solved the issue with data structure - this is now known for each process, so I can properly create groups and datasets collectively in all parallel processes.
Now I've got another problem:
When I create all datasets collectively, initially they have "0" size and "unlimited" max-size (so dataset is chunked), as I don't know if there will be any data to write by the given process.
Then, when the given process knows what data should write to the given dataset, I try to call H5Dset_extent(), but it seems this function must be also called collectively (!)...
That means each process should know what is the size of data to write in all other processes. Am I correct?
How to solve that issue?

I found very similar issue described a long time ago (December 2012) here:
https://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2012-December/006322.html

and my current problem is properly described here in this thread:
https://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2012-December/006337.html

Further in this thread someone said: "This is a limitation currently in the HDF5 standard that we plan to relax in the future."
Is this already solved now (after 5 years of development)?

Best regards,
Rafal

W dniu 2017-09-29 o 13:07, Rafal Lichwala pisze:

Hi Jarom, Hi All,

Thank you very much for the concrete answer! But...
What in case the given process (which has no data to write) does not
know the structure of the given dataset (no data => no info about
structure), so it cannot produce a proper datatype for collective
H5Dwrite() call - even if "space" is properly marked by
H5Sselect_none() ?...
It seems H5Dwrite() requires datatype also to be identical in all
processes during collective call...
I cannot use H5Tcommit() to share the datatype between processes in
file, because this function is also collective...
Can you see any solution for this problem?

Best regards,
Rafal

W dniu 2017-09-28 o 16:10, Nelson, Jarom pisze:

herr_t H5Sselect_none(hid_t space_id);
https://support.hdfgroup.org/HDF5/hdf5-quest.html#par-nodata
https://www.hdfgroup.org/2015/08/parallel-io-with-hdf5/
https://support.hdfgroup.org/HDF5/PHDF5/

Jarom

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On
Behalf Of Rafal Lichwala
Sent: Thursday, September 28, 2017 6:52 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] high level API for parallel version of HDF5
library

Hi,

Thank you for an answer and example codes.
Creating metadata (groups, datasets) is clear now and works fine, but
I've got the last doubt: what in case I'm running 4 MPI processes but
only 3 of them have some data to be written to the given dataset.
Since the H5Dwrite() call is in collective mode, my program hangs...
How to solve this?

Regards,
Rafal

W dniu 2017-09-27 o 22:50, Nelson, Jarom pisze:

Calls that affect the metadata need to be collective so that each
process has a consistent view of what the file metadata should be.

https://support.hdfgroup.org/HDF5/doc/RM/CollectiveCalls.html

Something like this (or the attached):

plist_id = H5Pcreate(H5P_FILE_ACCESS);

H5Pset_fapl_mpio(plist_id, comm, info);

H5Pset_all_coll_metadata_ops( plist_id, true );

file_id = H5Fcreate(H5FILE_NAME, H5F_ACC_TRUNC, H5P_DEFAULT,
plist_id);

H5Pclose(plist_id);

for(int procid = 0; i < mpi_size; ++i) {

 hid\_t gr\_id = H5Gcreate\(file\_id, std::to\_string\(procid\)\.c\_str\(\),

H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);

 H5Gclose\(gr\_id\);

}

H5Fclose(file_id);

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On
Behalf Of Rafal Lichwala
Sent: Wednesday, September 27, 2017 12:32 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] high level API for parallel version of HDF5
library

Hi Barbara, Hi All,

Thank you for your answer. That's clear now about H5TBmake_table()
call, but...

H5Gcreate() in not a high level API, isn't it?

So why I cannot use it in parallel processes?

Maybe I'm just doing something wrong, so could you please provide me
a short example how to create a set of groups (each one is the
process

number) running 4 parallel MPI processes? You can limit the example
code to the sequence of HDF5 calls only...

My current code works fine for just one process, but when I try it
for
2 (or more) parallel processes the result file is corrupted:

plist_id = H5Pcreate(H5P_FILE_ACCESS);

H5Pset_fapl_mpio(plist_id, comm, info);

H5Pset_all_coll_metadata_ops( plist_id, true ); file_id =
H5Fcreate(H5FILE_NAME, H5F_ACC_TRUNC, H5P_DEFAULT, plist_id);
H5Pclose(plist_id); hid_t gr_id = H5Gcreate(file_id,
std::to_string(procid).c_str(), H5P_DEFAULT, H5P_DEFAULT,
H5P_DEFAULT); H5Gclose(gr_id); H5Fclose(file_id);

Best regards,

Rafal

W dniu 2017-09-25 o 22:20, Barbara Jones pisze:

> Hi Rafal,

>

> No, the HDF5 High Level APIs are not supported in the parallel
version of HDF5.

>

> -Barbara

> help@hdfgroup.org <mailto:help@hdfgroup.org>

>

> -----Original Message-----

> From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On
Behalf Of Rafal Lichwala

> Sent: Monday, September 18, 2017 8:53 AM

> To: hdf-forum@lists.hdfgroup.org
<mailto:hdf-forum@lists.hdfgroup.org>

> Subject: [Hdf-forum] high level API for parallel version of HDF5
library

>

> Hi,

>

> Can I use high level API function calls (H5TBmake_table(...)) in
parallel version of the HDF5 library?

> There are no property list parameters for that function calls...

>

> Regards,

> Rafal

>

>

> _______________________________________________

> Hdf-forum is for HDF software users discussion.

> Hdf-forum@lists.hdfgroup.org
<mailto:Hdf-forum@lists.hdfgroup.org>

>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.
or
g

> Twitter: https://twitter.com/hdf5

>

> _______________________________________________

> Hdf-forum is for HDF software users discussion.

> Hdf-forum@lists.hdfgroup.org
<mailto:Hdf-forum@lists.hdfgroup.org>

>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.
or
g

> Twitter: https://twitter.com/hdf5

>

_______________________________________________

Hdf-forum is for HDF software users discussion.

Hdf-forum@lists.hdfgroup.org <mailto:Hdf-forum@lists.hdfgroup.org>

http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.
or
g

Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.
or
g
Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.or
g
Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

27

Hi All,

I've just discovered the problem with type conversion between C++ native "long double" and H5T_NATIVE_LDOUBLE.
On Windows x86_64 machine (Cygwin + GCC) it seems that long double has 128 bits, while H5T_NATIVE_LDOUBLE is treated as "64-bit floating point" (as described in HDFView). That means the value stored in HDF file may be silently truncated to 64 bits...

In C++ on such machine long double macro LDBL_MAX value is properly reported as 1.18973e+4932, but in HDF file (HDFView) it is already "Infinity".

Is it a limitation in the HDF library?
Is there any workaround for that issue?

Best regards,
Rafal

28

Hi Rafal,

I entered bug HDFFV-10305 for this issue. We will investigate it.
Feel free to send a message to the helpdesk (help@hdfgroup.org) if you want to check on the status of the problem.

Thank you!
-Barbara
help@hdfgroup.org

···

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Rafal Lichwala
Sent: Wednesday, October 11, 2017 6:39 AM
To: hdf-forum@lists.hdfgroup.org
Subject: [Hdf-forum] long double type problem

Hi All,

I've just discovered the problem with type conversion between C++ native "long double" and H5T_NATIVE_LDOUBLE.
On Windows x86_64 machine (Cygwin + GCC) it seems that long double has
128 bits, while H5T_NATIVE_LDOUBLE is treated as "64-bit floating point"
(as described in HDFView). That means the value stored in HDF file may be silently truncated to 64 bits...

In C++ on such machine long double macro LDBL_MAX value is properly reported as 1.18973e+4932, but in HDF file (HDFView) it is already "Infinity".

Is it a limitation in the HDF library?
Is there any workaround for that issue?

Best regards,
Rafal

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

29

(I’m here replying to the original post by Markus (“HDF lib incompatible with HDF file spec?”). The threading seems to have been messed up in the transition to Discourse (or possibly it was tangled up before). So that other conversations are visible on this post.)

Markus, do you know if anything ever happened with the HDFFV-10300 issue that was filed? Was it a problem with the HDF5 lib and/or the spec, and was any action taken?