Profiling shows H5C_protect called by getObjnameByIdx is a bottleneck

HDF5 Library C++ Users' Group
1

I have a custom HDF file organization. Using HDF 1.12.1 on Windows x64 - Release build
I am reading with C++ code

		size_t nDS = resultGroup.getNumObjs();
		for ( int iDS = 0; iDS < nDS; iDS++ )
		{
			std::string    dsName = resultGroup.getObjnameByIdx ( iDS );
            DataSet        ds     = resultGroup.openDataSet ( dsName );

The call to getObjnameByIdx is a bottleneck, the code is spending 80% of it’s time there , not reading the DataSets!
There must be a more efficient way to do this.

2

How many datasets are in that group?

-Aleksandar

3

Hi,
100,000 + datasets.
It is very clear that when using getObjnameByIdx there is a linear search going on - it is not random access.
By replacing my loop with H5Giterate I was able to get a speed up of over 300x (!)
That would have been very useful to note in the HDF documentation that getObjnameByIdx is not random access and has performance issues with large # of subobjects.

Andrew

4

I suggest you submit an issue about this at Issues · HDFGroup/hdf5 · GitHub, otherwise your remark may get overlooked.

-Aleksandar