HDF5 Performance


I am sorry I did not attach any files since it wasn't possible to attach a 64
MB file. I am attaching the program for a smaller run.

The folder ptestrun contains the program to read a binary file "mfdn.smwf001"
and write the same binary file into another file "binopfile". It also uses a
HDF5 file hdfref, to read the lengths being read and written by the processors.
The program does IO in exactly the same way our actual code does. IO happens as
1. Proc 0 reads its part of the vector. Then it reads the vector for the
remaining processors and sends(tag=1) it to the respective processors.
2. Proc 0 writes its part of the vector. Then it writes the vector for the
remaining processors that it receives(tag=2) from the respective processors.

The folder testrun contains the program to write the output file in a HDF5
file. It reads the contents from a HDF5 file "mfdn.smwf001" and writes the
dataset named "EigenVectors__" to a new HDF5 file "hdfopfile". The lengths
written by various procs can stored in the dataset "LENGTHS". The IO happens as

1. Get the dataset dims from the datasets named "EigenVectors__"
2. Create the HDF5 o/p file accordingly.
3. Read the datasets from "mfdn.smwf001" and write it to "hdfopfile"

Although this is a small run and may not fully utilize PHDF5 but may be someone
can tell if I am making any mistakes that is leading to the slow IO. I can't
send the larger runs as attachments since they are around 64 MB and 2 GB.

This program should run exactly on 5 processors.

Thanks a lot. Hope to get some pointers for improving the code.



This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.

2 posts were merged into an existing topic: HDF5 performance issue