Bug: Library compatibility version differs between CMake and autotools build on macOS


#1

Building 1.10.4 using autotools, the encoded library compatibility version is 104.0.0, while building the same version with CMake, the encoded library compatibility version is 103.0.0. This means a program linked against an autotools-built HDF5 cannot run with a CMake-built HDF5, even if the HDF5 version is the same.

For example, this is the HDF5 1.10.4 that is included with the h5py 2.9.0 macOS wheel, built using autotools:

(env) dirac:hdf5-1.10.4 insight$ otool -l ~/env/lib/python3.7/site-packages/h5py/.dylibs/libhdf5.103.dylib | grep -A 5 LC_ID_DYLIB
          cmd LC_ID_DYLIB
      cmdsize 56
         name /DLC/h5py/libhdf5.103.dylib (offset 24)
   time stamp 1 Thu Jan  1 01:00:01 1970
      current version 104.0.0
compatibility version 104.0.0

While this is the same version (1.10.4) that I’ve built with CMake:

(env) dirac:hdf5-1.10.4 insight$ otool -l ~/Insight/HDF5-1.10.4-Darwin/HDF_Group/HDF5/1.10.4/lib/libhdf5.103.dylib | grep -A 5 LC_ID_DYLIB
          cmd LC_ID_DYLIB
      cmdsize 56
         name @rpath/libhdf5.103.dylib (offset 24)
   time stamp 1 Thu Jan  1 01:00:01 1970
      current version 103.0.0
compatibility version 103.0.0
(env) dirac:hdf5-1.10.4 insight$

I believe this discrepancy is due to how the HDF5 CMake build calculates the SOVERSION, which is what CMake uses as -compatibility_version to the linker.

From CMakeLists.txt:

#-----------------------------------------------------------------------------
# parse the full soversion number from config/lt_vers.am and include in H5_SOVERS_INFO
#-----------------------------------------------------------------------------
file (READ ${HDF5_SOURCE_DIR}/config/lt_vers.am _lt_vers_am_contents)
string (REGEX REPLACE ".*LT_VERS_INTERFACE[ \t]+=[ \t]+([0-9]*).*$"
    "\\1" H5_LIB_SOVERS_INTERFACE ${_lt_vers_am_contents})
string (REGEX REPLACE ".*LT_VERS_REVISION[ \t]+=[ \t]+([0-9]*).*$"
    "\\1" H5_LIB_SOVERS_MINOR ${_lt_vers_am_contents})
string (REGEX REPLACE ".*LT_VERS_AGE[ \t]+=[ \t]+([0-9]*).*$"
    "\\1" H5_LIB_SOVERS_RELEASE ${_lt_vers_am_contents})
math (EXPR H5_LIB_SOVERS_MAJOR ${H5_LIB_SOVERS_INTERFACE}-${H5_LIB_SOVERS_RELEASE})
message (STATUS "SOVERSION: ${H5_LIB_SOVERS_MAJOR}.${H5_LIB_SOVERS_RELEASE}.${H5_LIB_SOVERS_MINOR}")

so LT_VERS_INTERFACE = 103 and LT_VERS_AGE = 0 from config/lt_vers.am are parsed into H5_LIB_SOVERS_INTERFACE and H5_LIB_SOVERS_RELEASE respectively, and H5_LIB_SOVERS_MAJOR is set to H5_LIB_SOVERS_INTERFACE - H5_LIB_SOVERS_RELEASE = 103.

Later in CMakeLists.txt, this H5_LIB_SOVERS_MAJOR is assigned to HDF5_LIB_PACKAGE_SOVERSION_MAJOR and then we have this snippet in config/cmake/HDF5Macros.cmake:

  if (${libtype} MATCHES "SHARED")
    set (PACKAGE_SOVERSION ${HDF5_${libpackage}_PACKAGE_SOVERSION})
    if (WIN32)
      set (LIBHDF_VERSION ${HDF5_PACKAGE_VERSION_MAJOR})
    else ()
      set (LIBHDF_VERSION ${HDF5_${libpackage}_PACKAGE_SOVERSION_MAJOR})
    endif ()
    set_target_properties (${libtarget} PROPERTIES VERSION ${PACKAGE_SOVERSION})
    if (WIN32)
        set (${LIB_OUT_NAME} "${LIB_OUT_NAME}-${LIBHDF_VERSION}")
    else ()
        set_target_properties (${libtarget} PROPERTIES SOVERSION ${LIBHDF_VERSION})
    endif ()
  endif ()

Here the SOVERSION is set to 103, and from https://cmake.org/cmake/help/v3.12/prop_tgt/SOVERSION.html :

For shared libraries and executables on Mach-O systems (e.g. OS X, iOS), the SOVERSION property corresponds to compatibility version and VERSION to current version.

I’m not sure what the real bug is actually. Should the LT_VERS_INTERFACE in config/lt_vers.am really be 103 for the 1.10.4 release? I haven’t looked at the autotools side of things, so I’m not sure if/how that value is used there.


Community Ticket Access now available
#2

I had a look at the autotools stuff, and I believe this snippet from bin/ltmain.sh explains it:

darwin)
  # Like Linux, but with the current version available in
  # verstring for coding it into the library header
  func_arith $current - $age
  major=.$func_arith_result
  versuffix=$major.$age.$revision
  # Darwin ld doesn't like 0 for these options...
  func_arith $current + 1
  minor_current=$func_arith_result
  xlcverstring="$wl-compatibility_version $wl$minor_current $wl-current_version $wl$minor_current.$revision"
  verstring="-compatibility_version $minor_current -current_version $minor_current.$revision"
  # On Darwin other compilers
  case $CC in
      nagfor*)
          verstring="$wl-compatibility_version $wl$minor_current $wl-current_version $wl$minor_current.$revision"
          ;;
      *)
          verstring="-compatibility_version $minor_current -current_version $minor_current.$revision"
          ;;
  esac
  ;;

Libtool doing + 1 here probably explains why you’ve chosen to set LT_VERS_INTERFACE to 103 on HDF5 1.10.4?

Then maybe the CMake logic should be updated to match the libtool logic, so add 1?

As a side note, I found this snipped just below in config/cmake/HDF5Macros.cmake:

  #-- Apple Specific install_name for libraries
  if (APPLE)
    option (HDF5_BUILD_WITH_INSTALL_NAME "Build with library install_name set to the installation path" OFF)
    if (HDF5_BUILD_WITH_INSTALL_NAME)
      set_property(TARGET ${libtarget} APPEND PROPERTY
          LINK_FLAGS "-current_version ${HDF5_PACKAGE_VERSION} -compatibility_version ${HDF5_PACKAGE_VERSION}"
      )
      set_target_properties (${libtarget} PROPERTIES
          INSTALL_NAME_DIR "${CMAKE_INSTALL_PREFIX}/lib"
          BUILD_WITH_INSTALL_RPATH ${HDF5_BUILD_WITH_INSTALL_NAME}
      )
    endif ()
    ...

so it seems that currently, one way of working around this discrepancy issue is to make do the CMake build with the HDF5_BUILD_WITH_INSTALL_NAME option set, because then the CMake scripts will manually pass -compatibility_version ${HDF5_PACKAGE_VERSION}. Haven’t tested this though, just an observation.


Community Ticket Access now available