Installation of old version of h5py (2.10.0) fails


As a dependency for another project I would like to install the old version of h5py==2.10.0 on a rolling-release linux system (manjaro) with pip 20.2.2 from /usr/lib/python3.9/site-packages/pip (python 3.9).

Running pip install h5py==2.10.0 leads to following error message:

Collecting h5py==2.10.0
  Using cached h5py-2.10.0.tar.gz (301 kB)
Requirement already satisfied: numpy>=1.7 in /usr/lib/python3.9/site-packages (from h5py==2.10.0) (1.19.4)
Requirement already satisfied: six in /usr/lib/python3.9/site-packages (from h5py==2.10.0) (1.15.0)
Installing collected packages: h5py
    Running install for h5py ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-mfjc2p9w/h5py/'"'"'; __file__='"'"'/tmp/pip-install-mfjc2p9w/h5py/'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);'"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-yoa5iho8/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/user/.local/include/python3.9/h5py
         cwd: /tmp/pip-install-mfjc2p9w/h5py/
    In file included from /usr/lib/python3.9/site-packages/numpy/core/include/numpy/ndarrayobject.h:21,
                     from /usr/lib/python3.9/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                     from /tmp/pip-install-mfjc2p9w/h5py/h5py/api_compat.h:26,
                     from /tmp/pip-install-mfjc2p9w/h5py/h5py/h5i.c:654:
    At top level:
    /usr/lib/python3.9/site-packages/numpy/core/include/numpy/__multiarray_api.h:1463:1: warning: ‘_import_array’ defined but not used [-Wunused-function]
     1463 | _import_array(void)
          | ^~~~~~~~~~~~~
    error: command '/usr/bin/gcc' failed with exit code 1
ERROR: Command errored out with exit status 1: /usr/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-mfjc2p9w/h5py/'"'"'; __file__='"'"'/tmp/pip-install-mfjc2p9w/h5py/'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);'"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-yoa5iho8/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/user/.local/include/python3.9/h5py Check the logs for full command output.

This happens both when running with and without administrator rights. Any suggestion?

I suspect you’ll need to play with different versions of cython to get h5py 2.10 to build, I don’t think we released python 3.9 wheels for 2.10.

1 Like

Thanks for your answer. Indeed installing with python 3.7.7 worked well.


I have the need to use hdf5 v1.8.22 and so am trying to install a similar aged h5py==2.10.0 onto a python 3.9 distribution. First I install cython (it doesn’t seem to matter what version, but here I use a similar vintage version):

pip install --no-cache-dir -I --no-binary=cython cython==0.29.22

then I install h5py (having set HDF5_DIR and HDF5_VERSION to point to my hdf5 build):

pip install --no-cache-dir -I --no-binary=h5py h5py==2.10.0

and this fails because it can’t find Cython:

        File "/tmp/jay/pip-build-env-xx4797jf/overlay/lib/python3.9/site-packages/setuptools/", line 1233, in run_command
        File "/tmp/jay/pip-build-env-xx4797jf/overlay/lib/python3.9/site-packages/setuptools/_distutils/", line 988, in run_command

        File "/tmp/jay/pip-install-lsdm4si0/h5py_605d1be1fdac465892ceac0d0878f29e/", line 161, in run
          from Cython.Build import cythonize
      ModuleNotFoundError: No module named 'Cython'
      [end of output]

It turns out that I can install h5py==3.8.0 and h5py==3.9.0 successfully (although their symbol tables don’t match my hdf5 1.8.22 library, which is why I want to build an old h5py), but I can’t build an older version.

My question is: is there a known break in support for older h5py (and hdf5) in python 3.9? And is there any chance that this can be made to work? Thanks.

Hello @salmonson1,

Have you tried installing h5py 3.9.0 with your libhdf5 version? I do not understand what you meant by “It turns out that I can install h5py==3.8.0 and h5py==3.9.0 successfully”. Did you try something like:

% HDF5_DIR=... pip install --no-cache-dir  --no-binary=h5py h5py

Are you using a virtual environment for this?


Thank you for the rapid response. Yes, I did exactly as you say to install h5py==3.9.0; setting HDF5_DIR and HDF5_VERSION to point to my hdf5 library (v1.8.22). And yes, this is in a virtual environment. It installs ok (both 3.9.0 or 3.8.0), but I when I start python and “import h5py” I get “undefined symbols” errors. My hypothesis is that this is because I’m trying to link a modern h5py (3.9.0) to an old hdf5 library (1.8.22), thus I am trying to build an older h5py.

Thanks, Jay

Good. It is much better to re-try from a clean slate by deleting the old virtual environment.

Can you share the undefined symbols errors on h5py import?

As for trying the old h5py version, I suggest only running the pip install command for h5py and let it figure out which cython version to install as dependency.


The error it reports is: undefined symbol: H5Rdereference. I will follow up with more undefined symbol names (as per running ‘nm’).


Ah yes… That function is available from libhdf5-1.10.0 but in the h5py source is not marked as such.

pip can build h5py from a git branch. If you supply a list of undefined symbols, I can prepare a special version of h5py that should hopefully build with your old version of libhdf5.


That would be spectacular, Aleksandar! There are many undefined symbols. Here I installed current versions: cython 3.0.2 and h5py 3.9.0, compiled against HDF5 1.8.22. Then, running nm gives the following Undefined symbols (U):

                 U H5Aclose
                 U H5Acreate2
                 U H5Acreate_by_name
                 U H5Adelete
                 U H5Adelete_by_idx
                 U H5Adelete_by_name
                 U H5Aexists
                 U H5Aexists_by_name
                 U H5Aget_info
                 U H5Aget_info_by_idx
                 U H5Aget_info_by_name
                 U H5Aget_name
                 U H5Aget_num_attrs
                 U H5Aget_space
                 U H5Aget_storage_size
                 U H5Aget_type
                 U H5Aiterate2
                 U H5Aopen
                 U H5Aopen_by_idx
                 U H5Aopen_by_name
                 U H5Aopen_idx
                 U H5Aopen_name
                 U H5Aread
                 U H5Arename
                 U H5Arename_by_name
                 U H5Awrite
                 U H5check_version
                 U H5close
                 U H5Dclose
                 U H5Dcreate2
                 U H5Dcreate_anon
                 U H5Dextend
                 U H5Dfill
                 U H5Dget_access_plist
                 U H5Dget_create_plist
                 U H5Dget_offset
                 U H5Dget_space
                 U H5Dget_space_status
                 U H5Dget_storage_size
                 U H5Dget_type
                 U H5Diterate
                 U H5Dopen2
                 U H5DOwrite_chunk
                 U H5Dread
                 U H5DSattach_scale
                 U H5DSdetach_scale
                 U H5Dset_extent
                 U H5DSget_label
                 U H5DSget_num_scales
                 U H5DSget_scale_name
                 U H5DSis_attached
                 U H5DSis_scale
                 U H5DSiterate_scales
                 U H5DSset_label
                 U H5DSset_scale
                 U H5Dvlen_get_buf_size
                 U H5Dvlen_reclaim
                 U H5Dwrite
                 U H5Fclose
                 U H5Fcreate
                 U H5FDregister
                 U H5FDunregister
                 U H5Fflush
                 U H5Fget_access_plist
                 U H5Fget_create_plist
                 U H5Fget_file_image
                 U H5Fget_filesize
                 U H5Fget_freespace
                 U H5Fget_intent
                 U H5Fget_mdc_config
                 U H5Fget_mdc_hit_rate
                 U H5Fget_mdc_size
                 U H5Fget_name
                 U H5Fget_obj_count
                 U H5Fget_obj_ids
                 U H5Fget_vfd_handle
                 U H5Fis_hdf5
                 U H5Fmount
                 U H5Fopen
                 U H5free_memory
                 U H5Freopen
                 U H5Freset_mdc_hit_rate_stats
                 U H5Fset_mdc_config
                 U H5Funmount
                 U H5Gclose
                 U H5Gcreate2
                 U H5Gcreate_anon
                 U H5get_libversion
                 U H5Gget_comment
                 U H5Gget_create_plist
                 U H5Gget_info
                 U H5Gget_info_by_name
                 U H5Gget_linkval
                 U H5Gget_num_objs
                 U H5Gget_objinfo
                 U H5Gget_objname_by_idx
                 U H5Gget_objtype_by_idx
                 U H5Giterate
                 U H5Glink2
                 U H5Gmove2
                 U H5Gopen2
                 U H5Gset_comment
                 U H5Gunlink
                 U H5Idec_ref
                 U H5Iget_file_id
                 U H5Iget_name
                 U H5Iget_ref
                 U H5Iget_type
                 U H5Iinc_ref
                 U H5Iis_valid
                 U H5Lcopy
                 U H5Lcreate_external
                 U H5Lcreate_hard
                 U H5Lcreate_soft
                 U H5Ldelete
                 U H5Ldelete_by_idx
                 U H5Lexists
                 U H5Lget_info
                 U H5Lget_info_by_idx
                 U H5Lget_name_by_idx
                 U H5Lget_val
                 U H5Lget_val_by_idx
                 U H5Literate
                 U H5Literate_by_name
                 U H5Lmove
                 U H5LTopen_file_image
                 U H5Lunpack_elink_val
                 U H5Lvisit
                 U H5Lvisit_by_name
                 U H5Oclose
                 U H5Ocopy
                 U H5Odecr_refcount
                 U H5Oexists_by_name
                 U H5Oget_comment
                 U H5Oget_comment_by_name
                 U H5Oget_info
                 U H5Oget_info_by_idx
                 U H5Oget_info_by_name
                 U H5Oincr_refcount
                 U H5Olink
                 U H5Oopen
                 U H5Oopen_by_addr
                 U H5Oopen_by_idx
                 U H5open
                 U H5Oset_comment
                 U H5Oset_comment_by_name
                 U H5Ovisit
                 U H5Ovisit_by_name
                 U H5Pall_filters_avail
                 U H5Pclose
                 U H5Pclose_class
                 U H5Pcopy
                 U H5Pcreate
                 U H5Pequal
                 U H5Pfill_value_defined
                 U H5Pget_alignment
                 U H5Pget_alloc_time
                 U H5Pget_attr_creation_order
                 U H5Pget_attr_phase_change
                 U H5Pget_cache
                 U H5Pget_char_encoding
                 U H5Pget_chunk
                 U H5Pget_chunk_cache
                 U H5Pget_class
                 U H5Pget_copy_object
                 U H5Pget_create_intermediate_group
                 U H5Pget_driver
                 U H5Pget_driver_info
                 U H5Pget_edc_check
                 U H5Pget_efile_prefix
                 U H5Pget_elink_acc_flags
                 U H5Pget_elink_fapl
                 U H5Pget_elink_prefix
                 U H5Pget_est_link_info
                 U H5Pget_external
                 U H5Pget_external_count
                 U H5Pget_family_offset
                 U H5Pget_fapl_core
                 U H5Pget_fapl_family
                 U H5Pget_fclose_degree
                 U H5Pget_fill_time
                 U H5Pget_fill_value
                 U H5Pget_filter2
                 U H5Pget_filter_by_id2
                 U H5Pget_istore_k
                 U H5Pget_layout
                 U H5Pget_libver_bounds
                 U H5Pget_link_creation_order
                 U H5Pget_link_phase_change
                 U H5Pget_local_heap_size_hint
                 U H5Pget_mdc_config
                 U H5Pget_meta_block_size
                 U H5Pget_nfilters
                 U H5Pget_nlinks
                 U H5Pget_obj_track_times
                 U H5Pget_sieve_buf_size
                 U H5Pget_sizes
                 U H5Pget_sym_k
                 U H5Pget_userblock
                 U H5Pget_version
                 U H5Pmodify_filter
                 U H5Premove_filter
                 U H5Pset_alignment
                 U H5Pset_alloc_time
                 U H5Pset_attr_creation_order
                 U H5Pset_attr_phase_change
                 U H5Pset_cache
                 U H5Pset_char_encoding
                 U H5Pset_chunk
                 U H5Pset_chunk_cache
                 U H5Pset_copy_object
                 U H5Pset_create_intermediate_group
                 U H5Pset_deflate
                 U H5Pset_driver
                 U H5Pset_edc_check
                 U H5Pset_efile_prefix
                 U H5Pset_elink_acc_flags
                 U H5Pset_elink_fapl
                 U H5Pset_elink_prefix
                 U H5Pset_est_link_info
                 U H5Pset_external
                 U H5Pset_family_offset
                 U H5Pset_fapl_core
                 U H5Pset_fapl_family
                 U H5Pset_fapl_log
                 U H5Pset_fapl_multi
                 U H5Pset_fapl_sec2
                 U H5Pset_fapl_split
                 U H5Pset_fapl_stdio
                 U H5Pset_fclose_degree
                 U H5Pset_file_image
                 U H5Pset_fill_time
                 U H5Pset_fill_value
                 U H5Pset_filter
                 U H5Pset_fletcher32
                 U H5Pset_istore_k
                 U H5Pset_layout
                 U H5Pset_libver_bounds
                 U H5Pset_link_creation_order
                 U H5Pset_link_phase_change
                 U H5Pset_local_heap_size_hint
                 U H5Pset_mdc_config
                 U H5Pset_meta_block_size
                 U H5Pset_nlinks
                 U H5Pset_obj_track_times
                 U H5Pset_scaleoffset
                 U H5Pset_shuffle
                 U H5Pset_sieve_buf_size
                 U H5Pset_sizes
                 U H5Pset_sym_k
                 U H5Pset_szip
                 U H5Pset_userblock
                 U H5Rcreate
                 U H5Rdereference
                 U H5Rget_name
                 U H5Rget_obj_type2
                 U H5Rget_region
                 U H5Sclose
                 U H5Scopy
                 U H5Screate
                 U H5Screate_simple
                 U H5Sdecode
                 U H5Sencode
                 U H5Sextent_copy
                 U H5Sget_select_bounds
                 U H5Sget_select_elem_npoints
                 U H5Sget_select_elem_pointlist
                 U H5Sget_select_hyper_blocklist
                 U H5Sget_select_hyper_nblocks
                 U H5Sget_select_npoints
                 U H5Sget_select_type
                 U H5Sget_simple_extent_dims
                 U H5Sget_simple_extent_ndims
                 U H5Sget_simple_extent_npoints
                 U H5Sget_simple_extent_type
                 U H5Sis_simple
                 U H5Soffset_simple
                 U H5Sselect_all
                 U H5Sselect_elements
                 U H5Sselect_hyperslab
                 U H5Sselect_none
                 U H5Sselect_valid
                 U H5Sset_extent_none
                 U H5Sset_extent_simple
                 U H5Tarray_create2
                 U H5Tclose
                 U H5Tcommit2
                 U H5Tcommitted
                 U H5Tconvert
                 U H5Tcopy
                 U H5Tcreate
                 U H5Tdecode
                 U H5Tdetect_class
                 U H5Tencode
                 U H5Tenum_create
                 U H5Tenum_insert
                 U H5Tenum_nameof
                 U H5Tenum_valueof
                 U H5Tequal
                 U H5Tfind
                 U H5Tget_array_dims2
                 U H5Tget_array_ndims
                 U H5Tget_class
                 U H5Tget_create_plist
                 U H5Tget_cset
                 U H5Tget_ebias
                 U H5Tget_fields
                 U H5Tget_inpad
                 U H5Tget_member_class
                 U H5Tget_member_index
                 U H5Tget_member_name
                 U H5Tget_member_offset
                 U H5Tget_member_type
                 U H5Tget_member_value
                 U H5Tget_native_type
                 U H5Tget_nmembers
                 U H5Tget_norm
                 U H5Tget_offset
                 U H5Tget_order
                 U H5Tget_pad
                 U H5Tget_precision
                 U H5Tget_sign
                 U H5Tget_size
                 U H5Tget_strpad
                 U H5Tget_super
                 U H5Tget_tag
                 U H5Tinsert
                 U H5Tis_variable_str
                 U H5Tlock
                 U H5Topen2
                 U H5Tpack
                 U H5Tregister
                 U H5Tset_cset
                 U H5Tset_ebias
                 U H5Tset_fields
                 U H5Tset_inpad
                 U H5Tset_norm
                 U H5Tset_offset
                 U H5Tset_order
                 U H5Tset_pad
                 U H5Tset_precision
                 U H5Tset_sign
                 U H5Tset_size
                 U H5Tset_strpad
                 U H5Tset_tag
                 U H5Tunregister
                 U H5Tvlen_create
                 U H5Zfilter_avail
                 U H5Zget_filter_info
                 U H5Zregister
                 U H5Zunregister

and also some non-HDF5-specific symbols:

                 U PyBytes_FromStringAndSize
                 U PyCapsule_GetName
                 U PyCapsule_GetPointer
                 U PyCapsule_IsValid
                 U PyCapsule_New
                 U PyCode_NewEmpty
                 U _Py_Dealloc
                 U _PyDict_GetItem_KnownHash
                 U PyDict_GetItemString
                 U PyDict_New
                 U PyDict_SetItem
                 U PyDict_SetItemString
                 U Py_EnterRecursiveCall
                 U PyErr_Clear
                 U PyErr_ExceptionMatches
                 U PyErr_Format
                 U PyErr_Occurred
                 U PyErr_SetObject
                 U PyErr_SetString
                 U PyErr_WarnEx
                 U PyEval_RestoreThread
                 U PyEval_SaveThread
                 U PyExc_AttributeError
                 U PyExc_ImportError
                 U PyExc_NameError
                 U PyExc_RuntimeError
                 U PyExc_SystemError
                 U PyExc_TypeError
                 U PyExc_ValueError
                 U _Py_FalseStruct
                 U PyFrame_New
                 U Py_GetVersion
                 U PyImport_AddModule
                 U PyImport_GetModuleDict
                 U PyImport_ImportModule
000000000002fe10 T PyInit_defs
                 U PyInterpreterState_GetID
                 U Py_LeaveRecursiveCall
                 U PyMem_Malloc
                 U PyMem_Realloc
                 U PyModule_AddObject
                 U PyModuleDef_Init
                 U PyModule_GetDict
                 U PyModule_GetName
                 U PyModule_NewObject
                 U _Py_NoneStruct
                 U PyObject_Call
                 U PyObject_GetAttr
                 U PyObject_GetAttrString
                 U _PyObject_GetDictPtr
                 U PyObject_Hash
                 U PyObject_Not
                 U PyObject_SetAttr
                 U PyObject_SetAttrString
                 U PyOS_snprintf
                 U PyThreadState_Get
                 U _PyThreadState_UncheckedGet
                 U PyTraceBack_Here
                 U _Py_TrueStruct
                 U PyTuple_New
                 U PyTuple_Pack
                 U PyUnicode_AsUTF8
                 U PyUnicode_Decode
                 U PyUnicode_FromFormat
                 U PyUnicode_FromStringAndSize
                 U PyUnicode_InternFromString

Please advise if you need any more information.

Many thanks!

Hi @salmonson1,

Something is wrong… the list of undefined symbols is way too long. Many of the listed libhdf5 functions have existed since forever and therefore be correctly defined even in the h5py 2.10 code. Also, the Py... undefined symbols are for python and may indicate that the python development resources (packages) are missing or not found.

How did you install libhdf5-1.8.22? Can you run its tools, like h5dump or h5ls on an HDF5 file? How did you install Python 3.9? What’s the computer platform you are using?