We have measured the time it takes to collect all the groups in hdf5 file
hierarchy. We did this loading time measurement on files located in the
computer and same files located on the remote server. The time it takes to
load all the group hierarchy is about 10 times more in the case of server
than loading straight from the computer. This time measurement was only
for reading group names from the file and not attributes or datasets. We
also observed that it took almost 5 seconds to load a local file with
20,000 groups with about 0.25(ms)/group. Are there any ideas to improve
these times for loading a group hierarchy?
We are considering all the group hierarchy and their attributes into a
single large byte array stored in the dataset of a special group. This
should speed up loading up all the hierarchy. Any opinions on this
approach?
Thank you
Asdaq Samir
Electrical Engineer | Caterpillar Inc.