Std::complex support for H5CPP C++

[steven]

@werner I moved the comment on complex number here, to keep it in one place.

@werner

Complex numbers also raise the issue of multiple representations of the same quantity: A complex number can be given in Cartesian representation (x,y) as well as in polar representation (r,phi). Those are of course different numerical values and even different units, but mathematically it’s still the same number, only different numerical representations of the same. So that raises the question if maybe via HDF5, or some HDF5 filters, some automatic data conversion between representations can be - or should be - supported? Conversion from (x,y) to (y,x) is already intrinsically possible, provided a complex data type is stored as compound data type.

You are right: complex numbers, dual numbers, quaternions, … oh my! Storing and retrieving them in a unified is hard to do. Instead I take a set of libraries and furnish them with unified efficient IO layer, adding to their work what I have surplus: expert HPC quality tuned IO routines for modern C++. Similarly to the already existing linear algebra layer, if you save one object from one library you should be retrieve it from a different one. What makes is more complex that I also work with Julia and sometimes R and python; and I try to keep doors open for unified object model across platforms.

Let’s look at only complex numbers for now: {real,imaginary}. possible implementations:

1. a record type in contagious layout having real and imaginary consecutively; fields have some names; great for event based model, 50% throughput is lost when accessing only the imaginary part.
2. to have a block of imaginary and block of real numbers: this requires filtering technique and provide good balance between subsetting of fields while keeping throughput 100% in both cases. Harder to get it right… ( I have non-public multithreaded implementation for this )
3. two datasets within a directory: crude but simple and less opaque than the previous, requires interleaving for event based access resulting increased complexity on consumer side.

Indeed only the upcoming release will treat complex numbers in a flexible way providing options for 1 and 3:

using complex = std::complex<long double>;
std::array<complex,N> data; 
auto fd = h5::open(...);
h5::write(fd, "my_data", data, h5::name{"real","imaginary"},
   h5::single | h5::multi, [...]);

Where single | multi refers to whether we want single dataset of HDF5 compound type, or multiple datasets within a group/directory. As you are moving on a tradeoff curve: throughput vs. flexibility I provide choice. While one could argue that is over kill in the case of complex numbers, keep in mind there are other objects following the same pattern: certain sparse matrix layouts, hash tables, … .

To take this further @miller86 Mark Miller pointed out having the advantage to save property lists within the container. Currently H5CPP relies on attributes and custom dapl-s, but to take it to the next level Mark’s ideas are worth exploring.

hope it wasn’t too long,
steve