Problems building HDF5 with plugin support

HDF5 Library
1

Some preliminaries:

  • Compiling from source for hdf5-1.14.6 (also tried hdf5-1.14.5)
  • Compiling on Linux with Intel compilers 2023.2.1 (icx and ifort)
  • Using CMake, as Autotools option doesn’t seem to support hdf5_plugins
  • Using “Unix Makefiles”

Between INSTALL_CMake.txt and discussion in Compiling HDF5 with Plugins (LZF), it appears HDF5 and hdf5_filters can be build simultaneously using the HDF5 cmake invocation. However, cmake fails in the first step. My invocation:

cmake -G “Unix Makefiles”
-DCMAKE_BUILD_TYPE:STRING=Release
-DBUILD_SHARED_LIBS:BOOL=OFF
-DBUILD_TESTING:BOOL=ON
-DHDF5_BUILD_FORTRAN:BOOL=ON
-DHDF5_INSTALL_MOD_FORTRAN:BOOL=ON
-DHDF5_BUILD_TOOLS:BOOL=ON
-DCMAKE_C_FLAGS:STRING=“-fpic”
-DHDF5_ENABLE_PLUGIN_SUPPORT:BOOL=ON
-DHDF5_ALLOW_EXTERNAL_SUPPORT:STRING=“TGZ”
-DPLUGIN_TGZ_NAME:STRING=“hdf5_plugins-1.14.tar.gz”
-DTGZPATH:PATH=“/discover/nobackup/projects/usaf_lis/emkemp/ZSTD/intel-2023.2.1/tarballs”
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF
…/hdf5-1.14.6 >& config.log

The error message is:
[ 22%] Performing download step (DIR copy) for ‘hdf5_zlib-populate’
Error copying directory from “/” to “/discover/nobackup/projects/usaf_lis/emkemp
/ZSTD/intel-2023.2.1/myhdfstuff/build/_deps/hdf5_zlib-src”.
gmake[2]: *** [CMakeFiles/hdf5_zlib-populate.dir/build.make:100: hdf5_zlib-popul
ate-prefix/src/hdf5_zlib-populate-stamp/hdf5_zlib-populate-download] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/hdf5_zlib-populate.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2

Modifying the -DTGZPATH flag to replace :PATH with :STRING does not change this behavior.

I assume an additional flag is required to specify the source directory (instead of "/'), but I don’t know what this is.

Any suggestions?

2

Hi, @eric.kemp !

You may try something like these:
hdf5_plugins/.github/workflows/cmake-ctest.yml at master · HDFGroup/hdf5_plugins
hdf5_plugins/.github/workflows/cmake-script.yml at master · HDFGroup/hdf5_plugins

However, I cannot guarantee that it will work based on the current CI results:
Workflow runs · HDFGroup/hdf5_plugins

3

This is the library compression zlib, not a plugin. Because you are using TGZ option the zlib tgz file must also be at that location.

4

Thanks @byrn. I made several attempts to modify the cmake command line and got different errors. For simplicity, the below examples don’t try to build the hdf5_plugins.

Attempt 1, mimicking INSTALL_CMake.txt IV-2-C:

cmake -G “Unix Makefiles”
-DCMAKE_BUILD_TYPE:STRING=Release
-DBUILD_SHARED_LIBS:BOOL=OFF
-DBUILD_TESTING:BOOL=ON
-DHDF5_BUILD_FORTRAN:BOOL=ON
-DHDF5_INSTALL_MOD_FORTRAN:BOOL=ON
-DHDF5_BUILD_TOOLS:BOOL=ON
-DCMAKE_C_FLAGS:STRING=“-fpic”
-DHDF5_ALLOW_EXTERNAL_SUPPORT:STRING=“TGZ”
-DZLIB_TGZ_NAME:STRING=“zlib-1.3.1.tar.gz”
-DTGZPATH:STRING=“/discover/nobackup/projects/usaf_lis/emkemp/ZSTD/intel-2023.2.1/tarballs/”
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF
…/hdf5-1.14.6 >& config.log

Error message:
[ 11%] Creating directories for ‘hdf5_zlib-populate’
[ 22%] Performing download step (verify and extract) for ‘hdf5_zlib-populate’
CMake Error at hdf5_zlib-subbuild/hdf5_zlib-populate-prefix/src/hdf5_zlib-populate-stamp/verify-hdf5_zlib-populate.cmake:11 (message):
File not found: /zlib-1.3.1.tar.gz

gmake[2]: *** [CMakeFiles/hdf5_zlib-populate.dir/build.make:99: hdf5_zlib-populate-prefix/src/hdf5_zlib-populate-stamp/hdf5_zlib-populate-download] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/hdf5_zlib-populate.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2

CMake Error at /usr/share/cmake/Modules/FetchContent.cmake:1012 (message):
Build step for hdf5_zlib failed: 2
Call Stack (most recent call first):
/usr/share/cmake/Modules/FetchContent.cmake:1141:EVAL:2 (__FetchContent_directPopulate)
/usr/share/cmake/Modules/FetchContent.cmake:1141 (cmake_language)
/usr/share/cmake/Modules/FetchContent.cmake:1184 (FetchContent_Populate)
config/cmake/HDFLibMacros.cmake:38 (FetchContent_MakeAvailable)
CMakeFilters.cmake:140 (EXTERNAL_ZLIB_LIBRARY)
CMakeLists.txt:921 (include)

Attempt 2, mimicking INSTALL_CMake.txt IV-2-D:
cmake -G “Unix Makefiles”
-DCMAKE_BUILD_TYPE:STRING=Release
-DBUILD_SHARED_LIBS:BOOL=OFF
-DBUILD_TESTING:BOOL=ON
-DHDF5_BUILD_FORTRAN:BOOL=ON
-DHDF5_INSTALL_MOD_FORTRAN:BOOL=ON
-DHDF5_BUILD_TOOLS:BOOL=ON
-DCMAKE_C_FLAGS:STRING=“-fpic”
-DZLIB_TGZ_NAME:STRING=“zlib-1.3.1.tar.gz”
-DZLIB_TGZ_ORIGPATH:STRING=“/discover/nobackup/projects/usaf_lis/emkemp/ZSTD/intel-2023.2.1/tarballs/”
-DHDF5_ALLOW_EXTERNAL_SUPPORT:STRING=“TGZ”
-DZLIB_USE_LOCALCONTENT:BOOL=OFF
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF
…/hdf5-1.14.6 >& config.log

Error message:
– Renaming
– /discover/nobackup/projects/usaf_lis/emkemp/ZSTD/intel-2023.2.1/myhdfstuff/build/_deps/hdf5_zlib-src/zconf.h
– to ‘zconf.h.included’ because this file is included with zlib
– but CMake generates it automatically in the build directory.
CMake Error at config/cmake/HDFLibMacros.cmake:40 (add_library):
add_library cannot create ALIAS target “zlib-static” because another target
with the same name already exists.
Call Stack (most recent call first):
CMakeFilters.cmake:140 (EXTERNAL_ZLIB_LIBRARY)
CMakeLists.txt:921 (include)

CMake Warning at CMakeFilters.cmake:163 (message):
ZLib support in HDF5 was enabled but not found
Call Stack (most recent call first):
CMakeLists.txt:921 (include)

[…snip…]

– HDF5 H5_LIBVER_DIR: 114 HDF5_VERSION_MAJOR: 1.14
– Configuring incomplete, errors occurred!

Attempt 3 follows the very end of mimicking INSTALL_CMake.txt IV-2-D, from “The files can also be retrieved from a local path if necessary”

cmake -G “Unix Makefiles”
-DCMAKE_BUILD_TYPE:STRING=Release
-DBUILD_SHARED_LIBS:BOOL=OFF
-DBUILD_TESTING:BOOL=ON
-DHDF5_BUILD_FORTRAN:BOOL=ON
-DHDF5_INSTALL_MOD_FORTRAN:BOOL=ON
-DHDF5_BUILD_TOOLS:BOOL=ON
-DCMAKE_C_FLAGS:STRING=“-fpic”
-DZLIB_TGZ_NAME:STRING=“zlib-1.3.1.tar.gz”
-DTGZPATH:STRING=“/discover/nobackup/projects/usaf_lis/emkemp/ZSTD/intel-2
023.2.1/tarballs/”
-DHDF5_ALLOW_EXTERNAL_SUPPORT:STRING=“TGZ”
-DZLIB_USE_LOCALCONTENT:BOOL=ON
-DHDF5_ENABLE_SZIP_SUPPORT:BOOL=OFF
…/hdf5-1.14.6 >& config.log

But I get the same error.