Hi Steven, it’s great to have more half-float support, though it would be great to have it as a piece of C code added to HDF5 on the same level like the existing standard types, even if just as an extension C library, such that it becomes available to all users.
I had been using OpenEXR’s half library https://www.openexr.com/ for a CPU implementation of half precision, do you have an experience in comparison with the half library that you use? It being a standalone header-only library looks like an advantage.
It seems the hurdle against making half-precision support part of the HDF5 library itself is that it’s not a standard data type defined in C99. I’d still place it there though because it’s a standard data type on GPUs though.
Concerning your question on time support, there’s a paper that discusses some aspects of how to specify time in HDF5: http://sciviz.cct.lsu.edu/papers/2007/F5TimeSemantics.pdf . It’s not necessarily “the” solution, but it may serve as inspiration to address some issues about handling time in HDF5. Certainly an actual implementation should be compatible with the C++2a standards by now, and allow to distinguish between UTC and GPS time and such issues. The topic seems to be complex enough to be placed in its own addon-library, and I am not sure if it can even be done reasonably well in C, as that would not benefit from the C++2a support for time.
Concerning support for std::complex and linear algebra: That raises the topic how to implement data types in a consistent scheme that also scales to higher-dimensional tensor types. Particularly, complex numbers are just a special 2D case of what is known in Geometric Algebra as a “rotor”. Their 3D equivalent is known as a quaternion. Quantum mechanics calls them “spinors” in the 4D case. The general treatment for arbitrary dimensions is via Geometric Algebra, so it would be good to have complex numbers implemented as the special 2D case of the nD case, rather than having a special case specific to complex numbers only. Such a scheme should then also be compatible with the specification of tensor and non-tensor data types. A while ago I wrote a paper on this topic that reviews Geometric Algebra and discusses some aspects and approaches also specifically with regards to storing those data types in HDF5: http://sciviz.cct.lsu.edu/papers/2009/GraVisMa09.pdf
It would be great to have a systematic, scalable approach here, but it’s not easy since not even the mathematical community is in agreement on a consistent notation for tensors. However, via HDF5 one could at least provide a reference and working model.