I am working with people to implement the parallel hdf5 writer into some code of ours, and I have a question regarding performance.
The objects we are dealing with, as an array, look like
[Nt,Nz,Ny,Nx,mu,a,b,…]
where Nt is the slowest index. The code is parallelized amongst the first four indices, and everything else is always stored locally on a node (memory). The indices Nt,…,Nx are typically of size 32 - 128, while all the other indices usually only are of size 3 or 4. It also happens that the indices mu,a,b,… are typically flattened in memory to a single index.
In terms of i/o performance, is there a recommendation for
1) writing an array [Nt,Nz,Ny,Nx,alpha], where alpha is a flattened array of size Nmu x Na x Nb x …
2) unflatten alpha so the array is [Nt,Nz,Ny,Nx,mu,a,b,…]
Since mu, a, and b are small, my intuition is you should just store them
in whatever way is most convenient for you. As long as you are using
collective I/O -- and you have to ask for it explicitly via property
list -- I'm pretty sure you'd be OK.
Would be a good exercise to try out both approaches, see if there is a
difference, then look at something like Darshan output to see why that
difference might exist.
==rob
···
On Wed, 2014-02-12 at 11:49 -0500, "André Walker-Loud " wrote:
Hi HDF-ers,
I am working with people to implement the parallel hdf5 writer into some code of ours, and I have a question regarding performance.
The objects we are dealing with, as an array, look like
[Nt,Nz,Ny,Nx,mu,a,b,…]
where Nt is the slowest index. The code is parallelized amongst the first four indices, and everything else is always stored locally on a node (memory). The indices Nt,…,Nx are typically of size 32 - 128, while all the other indices usually only are of size 3 or 4. It also happens that the indices mu,a,b,… are typically flattened in memory to a single index.
In terms of i/o performance, is there a recommendation for
1) writing an array [Nt,Nz,Ny,Nx,alpha], where alpha is a flattened array of size Nmu x Na x Nb x …
2) unflatten alpha so the array is [Nt,Nz,Ny,Nx,mu,a,b,…]
