How to visualize the allocation of metadata blocks given a hdf5 file?

Hi all, I’m using H5Pset_meta_block_size to change the minimum metadata block size of h5 files. I am wondering is there a way to check the number of metadata blocks or offsets of metadata blocks given a h5 file? I am looking for some tricks with command line tools to do that.

Try h5dump -B <yourfile.h5> .

If it doesn’t give you what you are looking for, I hope file format spec can help you to write a new tool or improve h5dump or h5ls.

I hope you can contribute it back to HDF community if you end up writing your own CLI tool. :pray:

Thanks for the suggestion. When I try h5dump -B, it simply prints out all contents of the h5 file. I’m on h5dump: Version 1.10.9. Do I need to upgrade it to the latest version to use this command?

Add -H (e.g., h5dump -B -H test.h5).

I use h5ls -va for this. Basically figure out where all the data is and then take the complement.

My problem with taking the complement of h5ls -va is it would include free space.

There is loads of talk about consolidating metadata for cloud optimization of HDF5, but I still can’t find any way to verify that metadata is consolidated. Any more suggestions?

Are you using file space management?

HDF5 File Space Management Documentation.

Yes, but whether using the default file space management or page aggregation, I cannot get information on how many metadata blocks exist or where they are located. It feels like guesswork to tune the various size parameters.

Under default file space management there is no free space tracking, so there is effectively no free space.

The specification is open so one could technically do a hexdump and take a look at the bytes:
https://docs.hdfgroup.org/hdf5/v1_14/_f_m_t3.html

You may be interested in the application of GNU Poke: