HDF5 1.8.22-rc2 source available for testing


#1

Hello everyone,

A second pre-release version of HDF5 1.8.22 source (hdf5-1.8.22-rc2) is available for testing and can be downloaded at the following link:
https://gamma.hdfgroup.org/ftp/pub/outgoing/hdf5. The first pre-release version had signature changes for 2 unused functions that caused binary incompatibility. Those changes have been reverted in rc2, which is the only code change from rc1 to rc2.

HDF5 1.8.22 will be followed later in 2021 by HDF5 1.8.23, the final release of HDF5 1.8. HDF5 1.10 and HDF5 1.12 are currently available. Detailed information about these releases can be found at: https://portal.hdfgroup.org/display/HDF5/Release+Specific+Information.

This release contains new features and changes from active HDF5 development that are compatible with hdf5_1_8:

• HDF5 now supports building with the AEC library as a replacement library for SZip.
• CVE-2018-13870 , CVE-2018-13869 and CVE-2018-17435 were fixed in this release.
• Improvements to CMake support were brought to HDF5 1.8.22 to bring it to the same support as other HDF5 versions.
• A CMake option was added to link generated Fortran MOD files into the include directory.
• Toolchain and cross-compile support were added in CMake.
• Support for building only shared libraries was added to CMake.
• Support for VS2019 was added for CMake version 3.15 and later.
• S3 and HDFS VFDs have been added to HDF5 1.8.
• Symbolic links were added for HL Fortran lib files to match the general pattern of hl lib file names.

For details about these and other changes or more information about the release see https://gamma.hdfgroup.org/ftp/pub/outgoing/hdf5/hdf5-1.8.22-rc2-RELEASE.txt.

If you have time to test this release candidate, we would greatly appreciate it. We test HDF5 on a variety of platforms and with multiple compilers, but there is always a system that we couldn’t test on. Your feedback is critical.

For those who use autoconf to build HDF5, please notice that the default configuration mode for this distribution is “–enable-build-mode=production". Use the “–enable-build-mode=debug” configure flag if you want to build HDF5 in the debug mode. Testing in both modes is recommended when possible.

The shared library version numbers for lib.so.* files in HDF5-1.8.22-rc1 have been updated according to libtool’s versioning system and the changes in symbols from HDF5-1.8.21. The API compatibility report is at this link: https://gamma.hdfgroup.org/ftp/pub/outgoing/hdf5/hdf5-1.8.22-rc2-vs-hdf5-1.8.21--preliminary-interface_compatibility_report.html.

We expect the official release of HDF5 1.8.22 around January 25. Reports of issues in the next week will be
invaluable and very much appreciated.

Thank you!
Larry Knox
The HDF Group


#2

Apologies for the late report. Testing 1.8.22-rc2 on Mac 10.14.5 with GCC 8.3. Make check gets a lot of errors like this for h5repack:

H5tools-DIAG: Error detected in HDF5:tools (1.8.22) thread 0:
  #000: h5diff_array.c line 748 in diff_datum(): H5T_STD_REF_OBJ reference type
    major: Failure in tools library
    minor: function info

There are also several thousand GCC compiler warnings during make. Those look mostly harmless, but I can’t be sure. See log files:
logfiles.hdf5-1.8.22rc2.mac-10.14.5.2020.0125b.tar (91.2 KB)

(Correction: Fixed typo in Mac OS version number.)


#3

Hi Dave,

Thank you for testing and sending the report! Those errors in the h5repack test are error output from a test that checks to see that h5repack appropriately fails. Perhaps a line stating that the test caused errors as expected could be added in the next release.

Looking at your log file there are 2022 unique warnings. We see similar numbers in our daily tests on Mac. Some work has been done to reduce warnings, focused on Linux GCC compilers. We will need to also examine more closely the warnings and the flags that produce them in HDF5 builds on macOS with clang.

Larry


#4

Larry, thanks for explaining these valid results. To expand on your suggestion, I suggest adding test summary lines to include a status of something like XFAIL or EXPECTED FAIL. I have seen the former used in other software.

I trust that the pedantic warnings are nothing new and that the release is working properly.


#5

I just tested binary compatibility with programs compiled with the previous version 1.8.21, on Mac 10.14.5. This failed with the well-known message:

***HDF5 library version mismatched error***

However, your published compatibility report says “binary compatible”. This is not sufficient for widespread public distribution. Drop-in binary compatibility without recompiling applications is needed, to the extent that the ABI is 100% backward compatible.

I suggest a simple change to the version check. Modify to check for previous version numbers that are known to be 100% backward binary compatible, and do not fail or issue warnings in those cases.

Please incorporate this easy change into 1.8.22 before release. Thank you.


#6

Thank you for the request. We looked at making a change for this release, but it was decided that test changes were too complicated at this time. An issue has been created to change the behavior for all upcoming releases, 1.12.1, 1.13.0, 1.10.8, and later in the year, 1.8.23. There is an environment variable available to skip the version check, and this workaround has been documented as follows on the download page, in RELEASE.txt, and in the newsletter announcing the release:

Binary compatibility issue and workaround

HDF5-1.8.22 binaries can replace the previous HDF5 version to run with applications compiled and linked with HDF5-1.8.21 and possibly HDF5 versions as early as 1.8.15. However, HDF5 checks versions in library files against versions in header files used to compile an application and will cause the application to abort when they do not match. An environment variable, HDF5_DISABLE_VERSION_CHECK, governs this behavior and can be set as follows:

2 skips the check
1 warns but does not abort
0 is the default behavior, aborting when the HDF5 version in the library files does not match the version in the header files.