Greetings,
I have written an experimental reader for a subset of HDF5 in rust (making things safer in terms of aliasing and concurrency) that supports concurrent/multi-threaded reads. The performance is on par or better for sequential reads with the official HDF5 library, and naturally way better for concurrent reads since blocking is unnecessary. It also has a streaming reader which is useful for network applications. Please take a look at: https://github.com/gauteh/hidefix. It is straightforward to create bindings for C/C++/Python, etc.
The approach is inspired by the DMR++ module in Hyrax, and requires that the chunks are accessible concurrently - so they are indexed at first. This is slow at the moment because of the way they are exposed by HDF5 library, but the changes in this PR make this pretty fast (200x speedup / 140 ms for a 1.4 gb file): https://github.com/HDFGroup/hdf5/pull/6 (applies to 1.12, needs updates for latest master). Also see this thread: Iterate over chunk info.
The project was started in order to provide fast concurrent reads for DARS, a DAP server written in rust: https://github.com/gauteh/dars.
– gaute