Since I've spent oh, only the last 8 years beating you up a bit about HDF5 metadata reads at scale, it is only fair that I share with you some good news. Thanks for implementing a scalable metadata read. It's already paying off.
There was a user trying to read a large dataset on our Blue Gene. We got him using 1.10, and presto:
Here's a quote from the user:
Yes, I can now restore data (with hdf5 1.10). I did a large test simulation on cetus (with fewer nodes than I'd usually run, so a lot of data per node), and then followed up by restoring a real 1024-node simulation on mira: this read 6 "field files" of 3.8 GB each and 2 "particle files" of 396 GB and 2 "particle files" of 69 GB in 442 seconds. Previously it was taking on the order of an hour to read in a 4 GB field file.
==rob