transfer between dataspaces of different layouts

HDF5 Library
1

I want to transfer data between multi-dimensional datasets storing the same data, but with a different interleaving (layout).
For instance, let’s assume a 2D dataset D1 with dimensions x,y, while D2 will rather store y,x
I assumed that the Dataspace abstraction would let me specify the data layout re-interleaving during dataset read/writes, but I can’t find where it is.
In this example, magic seems to occur if the dimensions do not match, (3x4 vs 4x3) but it does not describe what would happen if the max dimensions happened to be the same (4x4), how would it guess the transpose ?
What is the expected way to specify data [de/re ]interleaving during a data transfer ?

2

What appears to be a transposal in the example is only a reshape, and not a transposal. What I mean by that is this: multi-dimensional arrays of any rank are “really” reshaped one-dimensional arrays. In other words, int[12] and int[3][4] and int[4][3] and int[2][3][2], etc., are just different ways (shapes) of looking at the same contiguous region of elements (integers). A reshape does not require reordering the elements of that region. A transposal does require reordering the elements in that region:

Reshape:

int[12] -> {0, 1, 2, 3, ..., 11}

int[3][4] -> {{0, 1, 2, 3},
              {4, 5, 6, 7},
              {8, 9, 10, 11}}

int[4][3] -> {{0, 1, 2},
              {3, 4, 5},
              {6, 7, 8},
              {9, 10, 11}}

int[4][3] is not the transpose of int[3][4]. The transpose would be

{{0, 4, 8},
 {1, 5, 9},
 {2, 6, 10},
 {3, 7, 11}}

which is a reshape of

{0, 4, 8, 1, 5, 9, 2, 6, 10, 3, 7, 11}

The HDF5 library does on-the-fly reshape. It does not do on-the-fly transposal.

OK?
G.

3

Ok, the example I found was very misleading.
Now, the initial question remains : is there an HDF mechanism to de/re interleave multi-dimensional data ? (that would be a transpose in the 2D case)

It is a complex subject, the answer might be “no, do that yourself in an intermediate buffer”
I will check if “auto reshaping” (like you describe) and playing with strides will work.

4

Agree & apologies. H5Dgather could be used to implement transposal. You would still have to come up w/ a strategy to do this incrementally for a large dataset.

Best, G.