Compound datatype with int, float and array of floats

HDF5 Library h5py
1

I am trying to create a simple test HDF5 file with a dataset that has a compound datatype. I want 1 int,1 float and 1 array of floats. I can create the dataset with proper datatypes and can add data to the int and float entities. I can’t figure out how to add the data to the array entity. Code below. Suggestions are appreciated.

with h5py.File('test.h5', mode='w') as h5f:
    h5f.create_group('/group1')
    arr = np.random.random((10,2))
    h5f.create_dataset('/group1/ds1', shape=(10,),
           dtype=[ ('id' , int ), ('time', float), ('matrix', float, (10,2)) ] )
    
    h5f['group1']['ds1']['id'] = [ i for i in range(1,11,1) ]
    h5f['group1']['ds1']['time'] = [ i*0.125 for i in range(1,11,1) ]
    h5f['group1']['ds1'][0,'matrix'][:,:] = arr[:,:]
2

Hi @ken.walker,

I slightly modified your code and it worked:

import numpy as np
import h5py

dt = np.dtype([('id', 'i4'), ('time', 'f4'), ('matrix', 'f4', (10, 2))])

with h5py.File('hdf-forum-8083.h5', mode='w') as h5f:
    h5f.create_group('/group1')
    ds = h5f.create_dataset('/group1/ds1', shape=(10,), dtype=dt)
    for i in range(0, ds.shape[0]):
        arr = np.random.rand(10, 2)
        ds[i] = (i + 1, 0.125 * (i + 1), arr)

Take care,

       Aleksandar
3

Thanks for working on this and figuring it out. I think I tried every way to load the matrix data except row-by-row. LOL.

4

Glad it was still a timely reply. When working with h5py in general, it helps to use NumPy objects or Python objects for which conversion to NumPy objects is unambiguous. I also think storing data row-wise in this case is more efficient.

Aleksandar