# parallel i/o performance for writing arrays

Hi HDF-ers,

I am working with people to implement the parallel hdf5 writer into some code of ours, and I have a question regarding performance.

The objects we are dealing with, as an array, look like

[Nt,Nz,Ny,Nx,mu,a,b,…]

where Nt is the slowest index. The code is parallelized amongst the first four indices, and everything else is always stored locally on a node (memory). The indices Nt,…,Nx are typically of size 32 - 128, while all the other indices usually only are of size 3 or 4. It also happens that the indices mu,a,b,… are typically flattened in memory to a single index.

In terms of i/o performance, is there a recommendation for

1) writing an array [Nt,Nz,Ny,Nx,alpha], where alpha is a flattened array of size Nmu x Na x Nb x …

2) unflatten alpha so the array is [Nt,Nz,Ny,Nx,mu,a,b,…]

Thanks,

Andre

Since mu, a, and b are small, my intuition is you should just store them
in whatever way is most convenient for you. As long as you are using
collective I/O -- and you have to ask for it explicitly via property
list -- I'm pretty sure you'd be OK.

Would be a good exercise to try out both approaches, see if there is a
difference, then look at something like Darshan output to see why that
difference might exist.

==rob

···

On Wed, 2014-02-12 at 11:49 -0500, "André Walker-Loud " wrote:

Hi HDF-ers,

I am working with people to implement the parallel hdf5 writer into some code of ours, and I have a question regarding performance.

The objects we are dealing with, as an array, look like

[Nt,Nz,Ny,Nx,mu,a,b,…]

where Nt is the slowest index. The code is parallelized amongst the first four indices, and everything else is always stored locally on a node (memory). The indices Nt,…,Nx are typically of size 32 - 128, while all the other indices usually only are of size 3 or 4. It also happens that the indices mu,a,b,… are typically flattened in memory to a single index.

In terms of i/o performance, is there a recommendation for

1) writing an array [Nt,Nz,Ny,Nx,alpha], where alpha is a flattened array of size Nmu x Na x Nb x …

2) unflatten alpha so the array is [Nt,Nz,Ny,Nx,mu,a,b,…]