Release of HDF5-1.10.2 - Newsletter #160

The HDF5-1.10.2 release is now available for download from: https://www.hdfgroup.org/downloads/hdf5/
The source code (only) can also be obtained from: https://portal.hdfgroup.org/display/support/HDF5+1.10.2
User documentation for HDF5-1.10 can be accessed here: https://portal.hdfgroup.org/display/HDF5/

This release contains several major new features and changes, including the following:

  • Parallel HDF5 includes several improvements:
    • Compressed datasets can now be written using the MPI I/O driver.
    • Optimizations were added to improve performance of the HDF5 parallel open/close/flush operations.
    • MPI I/O transfers now allow reading or writing of more than 2GB of data in a single I/O operation.
  • The following vulnerabilities were addressed:
  • The following changes were added to the HDF5 library:
    • The H5F_LIBVER_V18 parameter can now be specified in the H5Pset_libver_bounds() function to provide control over object creation and ensure that files are fully compatible with HDF5-1.8.In addition, H5_LIBVER_LATEST is now set to H5F_LIBVER_V110. Please note that any applications that call H5Pset_libver_bounds to specify the latest file format will use HDF5-1.10 as the latest file format as of this release. (However, the h5format_convert utility can be used to change the file format version, if need be.)
    • The issue has been addressed where the path to VDS source files was interpreted (incorrectly) as relative to the directory where the executable ran (HDFFV-9724). The location of the source files can now be set with a function call or with an environment variable. For details, see H5Pset_virtual, and the new APIs H5Pset_virtual_prefix and H5Pget_virtual_prefix.
    • The H5DOread_chunk and H5Dget_chunk_storage_size functions were added to enable and support reading a raw data chunk directly from a dataset in a file into a buffer.
    • The H5Fget_eoa and H5Fincrement_filesize functions were added for retrieving and setting a file's EOA/EOF. New options were also added to h5clear to do the same:

      – filesize Print the file’s EOA and EOF
      – increment=C Set the file’s EOA to the maximum of (EOA, EOF) + C for the file

    • The H5FDdriver_query function was added to query a VFL driver for its feature flags when a file is not available.

  • Several changes to CMake were added:
    • The minimum CMake version required was changed to 3.10.
    • pkg-config file generation was added.
    • The creation of h5cc and similar compile scripts in builds was added for Linux.
  • C++ API wrappers and classes were added for link operations.
  • Several Java wrappers were added.

There are many other changes not listed here. Please see the release notes for details: https://portal.hdfgroup.org/display/support/HDF5+1.10.2

Also refer to the HDF5 Software Changes from Release to Release for HDF5-1.10: https://portal.hdfgroup.org/display/HDF5/Software+Changes+from+Release+to+Release+for+HDF5-1.10

Future Changes to Supported Compilers and Platforms
Please be aware that after this release, we will be dropping support for VS 2013 and adding support for VS 2017.

If you build your HDF5 library with the configure option --with-default-api-version=v18, does the format of HDF5 files created with this library default to H5F_LIBVER_V110? So if you want your output files to default to H5F_LIBVER_V18, you also must call H5Pset_libver_bounds with the low/high values set to H5F_LIBVER_V18 to guarantee that other user’s of your files that use the HDF5 1.8.x library can still read your files?

Ann,

The configure option --with-default-api-version=v18 will default two functions H5Fget_info and H5Rderefernce functions to their version 1. API versioning for those two functions do not change file format.

Usually, a file becomes unreadable by HDF5 maintenance releases (e.g., 1.8 series) when new features of the major HDF5 release (e.g., 1.10.0) that require file format changes are used to create the file. [For HDF5 1.10.* features that require file format changes (e.g., VDS, SWMR, cache image, etc.) see https://portal.hdfgroup.org/display/HDF5/Software+Changes+from+Release+to+Release+for+HDF5-1.10.]

If an application written for 1.8.* is linked with HDF5 1.10.(i.e., it doesn’t use any new features of 1.10.0), a file created by the application will be readable by HDF5 1.8. (or even earlier versions). The exception to this rule would be an application that uses H5Pset_libver_bounds with H5F_LIBVER_LATEST value for its “low” and “high” parameters.

In this case both parameters should be set to H5F_LIBVER_V18 to create 1.8 compatible file.

File control versioning became available in the HDF5 1.10.3 release. As a safeguard feature, we recommend using H5Pset_libver_bounds function to control file versioning.

I should admit that it is not easy to find in our documentation answers about API versioning and file format changes, and backward/forward compatibility topic is not the easiest one :slight_smile: We will try to improve our documentation.

Thank you!

Elena