Hello all,
I have been beating my head to find out the difference between Collective and
Independent IO, so as to make sense of what is happening in my code. The
reason for this is that I was trying to write a file in parallel and to my
surprise I found that Independent IO performs better than Collective IO.
In collective IO, all the processors start writing the file simultaneously(in
a blocking manner) and write their chunk of the file. In independent IO, each
processor starts writing the file as and when there is a library call to
H5dwrite_f(), so if some(and not all) processors call h5dwrite() at the same
time, it is as good as collective IO.
What my best guess for the slow operation for collective mode is the blocking
nature of collective. For ex. if 9/10 procs are ready to write, they will wait
for the 10 proc to arrive. But in Independent IO, the 9 procs will write their
chunk and proceed. The last processor will write its part when it calls the
API.
Please correct me if I am wrong about Collective vs Independent IO.
I also tried to write the file using just one processor and found that it was
noticably slow, so I am assuming parallel IO is working good. Another thing I
noticed that Parallel HDF5(6 procs) performed slower than binary(only one
processor) for an array of 17725 reals but performs better(3 times faster)
than binary(one processor doin IO) for an array of 1.2 million and using 9
processors. This was a heartening sign and the results look good.
Regards,
Nikhil
ยทยทยท
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.