H5Dget_chunk_info performance for many chunks?


I tried to read chunk offsets and sizes for a file with a lot of chunks and it seemed to be slower than expected. The problem is illustrated in this gist. Is it traversing a list from the beginning for each read?

Is there a way some way to get all of the chunk file offsets + sizes ?

Thanks for your help!


For context : I am trying to get multithreaded read-only access to hdf5 files and followed the suggestion in here to grab chunks for manual decoding:

Perhaps this is just a performance bug, should I open a github issue for it?



Could you please try H5Dchunk_iter function? It is in the HDF5 develop branch.

1 Like

Thanks! That looks like it is going to solve the problem.