# chunks of different sizes

Hi,

Is it possible to write chunks of different sizes? I mean, in a
parallel write, can chunks with different dimensions be written
collectively? For example, lets say that I have 4 processors and a
8-row by 6-column matrix. Mpi rank 0 writes from row 0 to 3 and column
0 to 3; Mpi rank 1 writes from 0 to 3 and 4 to 5; Mpi rank 2 writes
from 4 to 7 and 0 to 1; and finally Mpi rank 3 writes from 4 to 7 and
2 to 5. I am assuming C indexing. Notice that, although the processors
write chunks of different sizes, there is no overlap nor gaps. How can
I do that?

Eduardo

Hi Eduardo,

···

On May 2, 2011, at 8:18 AM, Eduardo wrote:

Hi,

Is it possible to write chunks of different sizes? I mean, in a
parallel write, can chunks with different dimensions be written
collectively? For example, lets say that I have 4 processors and a
8-row by 6-column matrix. Mpi rank 0 writes from row 0 to 3 and column
0 to 3; Mpi rank 1 writes from 0 to 3 and 4 to 5; Mpi rank 2 writes
from 4 to 7 and 0 to 1; and finally Mpi rank 3 writes from 4 to 7 and
2 to 5. I am assuming C indexing. Notice that, although the processors
write chunks of different sizes, there is no overlap nor gaps. How can
I do that?

I think you want to make different selections on different processes, not different chunk sizes. Try H5Sselect_hyperslab().

Quincey

Is it possible to write chunks of different sizes? I mean, in a
parallel write, can chunks with different dimensions be written
collectively?

``````   I think you want to make different selections on different processes, not different chunk sizes\.  Try H5Sselect\_hyperslab\(\)\.
``````

Different chunk sizes would be interesting to avoid IO contention when
the hyperslabs are not regular.

- Rhys

Different chunk sizes would be interesting to avoid IO contention when
the hyperslabs are not regular.

So, if my hyperslabs are not regular (similarly to my previous
example), does my application suffer IO contention? I mean, should I
avoid irregular hyperslabs, even if there is no overlap or gaps?

Best regards,
Eduardo

Hi Eduardo,

···

On May 3, 2011, at 12:36 PM, Eduardo wrote:

Different chunk sizes would be interesting to avoid IO contention when
the hyperslabs are not regular.

So, if my hyperslabs are not regular (similarly to my previous
example), does my application suffer IO contention? I mean, should I
avoid irregular hyperslabs, even if there is no overlap or gaps?

Yes, independent access I/O performance will be best when there is a direct mapping of each process's selection to 1+ chunks and those chunks aren't accessed by other processes. Enabling collective I/O access may reduce or eliminate the penalty for accessing one chunk with multiple processes however.

Quincey