Unable to create hdf5 file from excel data for fixed length string

HDF5 Library h5py
1

I am trying to create an hdf5 file from an excel file and since I have mixed data in certain columns I am planning to store the object data as string with fixed size. However even after converting the column into string data type while creating data set I am getting error as below

File "/home/kmoza/Recommender/utils/hdf5_api.py", line 24, in create_hdf5_file
    dset = hdf_file.create_dataset(sanitized_column, data=col_data, dtype=fixed_str_dtype)
  File "/home/kmoza/.local/lib/python3.10/site-packages/h5py/_hl/group.py", line 183, in create_dataset
    dsid = dataset.make_new_dset(group, shape, dtype, data, name, **kwds)
  File "/home/kmoza/.local/lib/python3.10/site-packages/h5py/_hl/dataset.py", line 166, in make_new_dset
    dset_id.write(h5s.ALL, h5s.ALL, data)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5d.pyx", line 282, in h5py.h5d.DatasetID.write
  File "h5py/_proxy.pyx", line 115, in h5py._proxy.dset_rw
OSError: Can't synchronously write data (no appropriate function for conversion path)

I have made sure that the read data from excel and the data I am writing to hdf5 file has same dtype

(Pdb) fixed_str_dtype
dtype('S5')
(Pdb) col_data.dtype
dtype('S5')

Environment:
Python 3.10.12
h5py==3.11.0
numpy==1.26.4
openpyxl==3.1.2
pandas==2.2.2

2

Hi @krishoza15sep,

If you are not bound to a specific API, you may want to consider HDFql, a high-level declarative language to manage HDF5 data. HDFql fully supports reading data from an Excel file and writing it to an HDF5 file (and vice versa).