I am just learning to use HDF5 , and I have a question on how to use the mpiposix driver.
Attached herewith is a simple mpi program where each process does the following:
open hdf file;
write data into a 2x2 dataset;
close file;
re-open file;
read dataset back;
I ran this with 2 processes and h5dump of the file shows that the file only contains data from the first process. I dont see data from the second process. I suspect I am not somehow specifying the property lists correctly. Can someone point out what I am doing wrong?
Thanks,
Kshitij Mehta
PhD Candidate
Parallel Software Technologies Lab
University of Houston
Texas, USA
I am just learning to use HDF5 , and I have a question on how to use the mpiposix driver.
Attached herewith is a simple mpi program where each process does the following:
open hdf file;
write data into a 2x2 dataset;
close file;
re-open file;
read dataset back;
I ran this with 2 processes and h5dump of the file shows that the file only contains data from the first process. I dont see data from the second process. I suspect I am not somehow specifying the property lists correctly. Can someone point out what I am doing wrong?
The property list setting is fine, but you are creating 1 dataset, and having two processes write to the same dataset (over each other).. So in your case, process 1 data is overwriting process 2's data.. If you delay process 1 a little before the write call, using sleep for example, you will read back process's 1 data from both processes.
Probably what you want is create n datasets, and have each process write to each dataset?
or create 1 4xn, or nx4 dataset and have each process select a portion of the dataset to write and read from?
Thanks,
Mohamad
···
On 5/24/2012 4:32 PM, Kshitij Mehta wrote:
Thanks,
Kshitij Mehta
PhD Candidate
Parallel Software Technologies Lab
University of Houston
Texas, USA
Rob is definitely correct.. The question that I should have asked also in the previous email:
Is there a reason why you are using the mpiposix VFD and not the mpiio VFD?
Mohamad
···
On 5/25/2012 11:09 AM, Rob Latham wrote:
On Fri, May 25, 2012 at 09:41:57AM -0500, Mohamad Chaarawi wrote:
Probably what you want is create n datasets, and have each process
write to each dataset?
I guess this is MPIPOSIX, so nothing clever can happen?
or create 1 4xn, or nx4 dataset and have each process select a
portion of the dataset to write and read from?
If you use MPIIO, doing things this way gets you a lot of
optimizations. I imagine one day you'll want to scale up beyond 4
processors..
I am writing a driver for Hdf5 and was looking at the existing VFDs. I wrote some test programs to learn more about their internal working. So not writing any performance critical stuff.
···
On May 25, 2012, at 1:52 PM, Mohamad Chaarawi wrote:
Rob is definitely correct.. The question that I should have asked also in the previous email:
Is there a reason why you are using the mpiposix VFD and not the mpiio VFD?
Mohamad
On 5/25/2012 11:09 AM, Rob Latham wrote:
On Fri, May 25, 2012 at 09:41:57AM -0500, Mohamad Chaarawi wrote:
Probably what you want is create n datasets, and have each process
write to each dataset?
I guess this is MPIPOSIX, so nothing clever can happen?
or create 1 4xn, or nx4 dataset and have each process select a
portion of the dataset to write and read from?
If you use MPIIO, doing things this way gets you a lot of
optimizations. I imagine one day you'll want to scale up beyond 4
processors..
