We have been upgrading HDF5 from version 1.8.23 to 1.14.1. The following code where a file image of an in-memory file is created and loaded back in has then started to fail and consistently:
import h5py
print("H5Py version: ", h5py.version.version)
print("HDF5 version: ", h5py.version.hdf5_version_tuple)
print("HDF5 built version: ", h5py.version.hdf5_built_version_tuple)
def createFileImage():
filename = 'in_memory.hdf5'
hdf5 = h5py.File(
filename,
mode='w',
driver='core',
backing_store=False,
libver='latest')
hdf5.flush()
file_image = hdf5.id.get_file_image()
return file_image
def loadFileImage(file_image):
filename = 'in_memory.hdf5'
fapl = h5py.h5p.create(h5py.h5p.FILE_ACCESS)
fapl.set_fapl_core(backing_store=False)
fapl.set_file_image(file_image)
filename = bytes(filename, encoding='utf-8')
# The failure happens here ->
file_id = h5py.h5f.open(filename, fapl=fapl)
file = h5py.File(
file_id,
mode='w',
driver='core',
backing_store=False,
libver='latest')
return file
file_image = createFileImage()
read_obj = loadFileImage(file_image)
The failure is:
H5Py version: 3.10.0
HDF5 version: (1, 14, 1)
HDF5 built version: (1, 14, 1)
Traceback (most recent call last):
File "/home/jensenk/git/quantumsource-xtr/create_and_load_file_image_minimal_example.py", line 47, in <module>
read_obj = loadFileImage(file_image)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jensenk/git/quantumsource-xtr/create_and_load_file_image_minimal_example.py", line 34, in loadFileImage
file_id = h5py.h5f.open(filename, fapl=fapl)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 102, in h5py.h5f.open
OSError: Unable to synchronously open file (incorrect metadata checksum after all read attempts)
Is there something we are missing to create a proper file image. It used to work in the old version (1.8.23) of HDF5.