I noticed a serious issue this morning while tracking down some
performance problems in my code, which might affect other people here.
It seems that writing performance deteriorates significantly (roughly
a factor of 6) when the following conditions are met:
(1) a dataset is stored in a chunked layout,
(2) the memory dataspace and file dataspace have different ranks
I'm not aware of any warnings in the end-user documentation about
this. My application frequently performs writes from a single buffer
(say 10000 floats) to one "row" of a dataset (e.g. of dataspace (500,
10000)). If, in chunked mode, I use a rank-2 memory dataspace with
shape (1, 10000), it takes about 6 seconds to write, but if I use a
rank-1 memory dataspace with shape (10000), it takes 34 seconds (!).
I've verified this behavior occurs with HDF5 1.6.6 and 1.8.3. An
example C file is attached.
ctest.c (1.34 KB)