Hello and good afternoon!
We are happy to announce release candidate 1 for HDF5 1.12.0. You may find the release candidate tar ball at https://gamma.hdfgroup.org/ftp/pub/outgoing/hdf5_1.12/
File format and all APIs should now be stable. The final HDF5 1.12.0 release is planned for the end of this week.
Please note that applications built with earlier versions of HDF5 are very likely to fail to compile with the HDF5 1.12.0 library due to the change from haddr_t to “object tokens” described below. There are three options that will allow such applications to compile with HDF5 1.12.0:
- build HDF5 1.12.0 with default version 1.10 APIs using the --enable-default-api-version=v110 configure flag for autotools or set DEFAULT_API_VERSION=v110 for CMake.
- use -DH5_USE_110_API when compiling the application with h5cc
- refactor the application code to use new APIs provided for object tokens.
Applications built with HDF5 1.8 APIs can use the same options with v18.
Further information about this release can be found on the Support Portal:
• HDF5 documentation
• New Features in HDF5 Release 1.12
• Software Changes from Release to Release
Major changes from previous releases of HDF5:
Public exposure of haddr_t type was refactored in favor of “object tokens”.
To better accommodate HDF5 VOL connectors where “object addresses in a file” may not be meaningful, a new H5O_token_t “object token” type has been introduced. The new “object tokens” represent unique and permanent identifiers for referencing HDF5 objects within a container and are designed to replace object addresses.
A new H5Oopen_by_token API call was introduced to open an object by a token, similar to how object addresses were previously used with H5Oopen_by_addr. New versions of certain H5L, and H5O functions were also added, along with compatibility macros to use previous or new versions of the functions according to configure options or compile flags to specify default api versions, and a “token” class with its own utility functions was added to a VOL class structure for interacting with object tokens using a VOL connector. See https://gamma.hdfgroup.org/ftp/pub/outgoing/hdf5_1.12/hdf5-1.12.0-rc1-RELEASE.txt for more information about the release.
The required CMake version has been increased to 3.14 for Windows platforms and 3.12 for all other platforms.
Added support for new references to h5dump and h5ls tools.
Improved performance when creating a large number of small datasets.
Added new public functions H5Sselect_adjust, H5Sselect_project_intersection, and H5VLget_file_type,which may be useful when developing VOL connectors.
Updated DDL documentation with the new types of references, VDS and opaque types:
In this release, we introduced several new features and along with performance improvement for hyperslab selection. A short description of the features is below.
Virtual Object Layer (VOL)
In this major HDF5 release we introduce HDF5 Virtual Object Layer (VOL). VOL is an abstraction layer within the HDF5 library that enables different methods for accessing data and objects that conform to the HDF5 data model. The VOL layer intercepts all HDF5 API calls that potentially modify data on disk and forwards those calls to a plugin “object driver”. The data on disk can be a different format than the HDF5 format. For more information about VOL we refer the reader to the following documents (under review):
VOL HDF5 APIs
For available VOL connectors please see
https://bitbucket.hdfgroup.org/projects/HDF5VOL
We are asking VOL connector developers in particular to test this release and let us know about any problems ASAP. Please let us know if you want to contribute your code to the repository above to be more visible to HDF5 community.
Enhancements to HDF5 References
HDF5 references were extended to support attributes, and object and dataset selections that reside in another HDF5 file. For more information including a list of new APIs, see
This feature requires HDF5 File Format extension. HDF5 Libraries prior to 1.12.0 will not be able to read them.
New S3 and HDFS Virtual File Drivers (VFDs)
This release has two new VFDs. The S3 VFD allows accessing HDF5 files on AWS S3 buckets. HDFS VFD allows accessing HDF5 file stored on Apache HDFS. See
for information on enabling those drivers and using those APIs.
Hyperslab selection performance improvements
In 1.12.0 we optimized hyperslab selection code to achieve better performance. In general, performance improved by an order of magnitude. In the case of reading a regular selection from 20GB dataset into a one dimensional array performance was improved by a factor of 6000. If you are interested in the benchmark we ran, please login to JIRA and see
https://jira.hdfgroup.org/browse/HDFFV-10930.
If you have time to test this release candidate, we will greatly appreciate it. We test HDF5 on a variety of platforms and with multiple compilers, but there is always a system that we couldn’t test on. Your feedback is critical.
Your prompt reports of any issues found will be invaluable and very much appreciated. Again, the final 1.12.0 release is planned for the end of this week. Thank you and happy testing!
Larry Knox