Toward multi-threaded HDF5 library

Dear HDF5 users,

I am soliciting information on the “typical and maximum” number of property lists your application(s) uses. (You may list the H5Pcalls or the number of H5P identifiers in use).

Our company “Lifeboat, LLC” received DOE’s Phase I grant to explore paths toward multi-threaded HDF5. We identified required HDF5 packages that have to be modified to allow multi-threaded HDF5 VOL connectors. H5P is one of those and its current implementation presents the biggest challenge. Information on how HDF5 property lists are used will help us to finalize the changes to H5P to make it multi-threaded.

All design documents will be published on Lifeboat’s GitHub. (Currently, you will find a brief description of our approach and initial draft designs. Design of the changes is an iterative process; the final versions will be posted by February 28, 2023).

We plan to implement and contribute the changes to the open source HDF5 if we receive Phase II grant. If not, the community will have design documents to work with if someone is interested in following this path :smile:
If you have questions about my request, please reply to this post or contact me directly at elena.pourmal@lifeboat.llc

Thank you and Happy New Year to all HDF5 users and to The HDF Group!

Elena

4 Likes

I think this depends on which side of the application we are discussing. For writing a file, it’s typically just one FAPL for H5Pset_meta_block_size and H5Pset_libver_bounds and a DCPL for H5Pset_chunk.

1 Like