How to trace MPI calls in HDF5 application using MPE?

Hi Rob,

That's exactly my problem. I need to record those MPI functions called by
HDF5. All the I/O in my application is done using HDF5 (there's no explicit
MPI I/O function like MPI_File_write ..). So when I link my application
with just mpe as usual, I can't see those I/O functions.

I think that's the reason why I need to enable MPE with HDF5.

Thank you very much,
Huong

Oh, you can, but it can be tricky to get the libraries in the right
order.

For example, I'm using the mpi compiler wrappers that came with MPICH
and a separate build of MPE:

$ mpicc h5test.c -o h5test -L${HDF5}/lib -lhdf5 -lm -L/soft/apps/mpe/install_ibm/lib -llmpe -lmpe

you can also sometimes use your mpi wrapper scripts to do some of this
for you, if you built MPICH2 with MPE support:

$ mpicc -mpe=mpilog h5test.c -o h5test -L${HDF5}/lib -lhdf5 -lm

==rob

···

On Wed, Apr 04, 2012 at 04:18:35PM -0500, huong luu wrote:

Hi Rob,

That's exactly my problem. I need to record those MPI functions called by
HDF5. All the I/O in my application is done using HDF5 (there's no explicit
MPI I/O function like MPI_File_write ..). So when I link my application
with just mpe as usual, I can't see those I/O functions.

--
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA

There's also some broken code in the HDF5 library API enter/leave macros, which I'm working with Huong off-list to correct.

  Quincey

···

On Apr 4, 2012, at 4:48 PM, Rob Latham wrote:

On Wed, Apr 04, 2012 at 04:18:35PM -0500, huong luu wrote:

Hi Rob,

That's exactly my problem. I need to record those MPI functions called by
HDF5. All the I/O in my application is done using HDF5 (there's no explicit
MPI I/O function like MPI_File_write ..). So when I link my application
with just mpe as usual, I can't see those I/O functions.

Oh, you can, but it can be tricky to get the libraries in the right
order.

For example, I'm using the mpi compiler wrappers that came with MPICH
and a separate build of MPE:

$ mpicc h5test.c -o h5test -L${HDF5}/lib -lhdf5 -lm -L/soft/apps/mpe/install_ibm/lib -llmpe -lmpe

you can also sometimes use your mpi wrapper scripts to do some of this
for you, if you built MPICH2 with MPE support:

$ mpicc -mpe=mpilog h5test.c -o h5test -L${HDF5}/lib -lhdf5 -lm

Hi Rob,

I also tried your suggestion but after running, there was no trace file
created.

cc h5ex_d_chunk.c -o h5ex_d_chunk -lhdf5 -lm
-L/nics/c/home/hluu/install/mpe-orig/lib -llmpe -lmpe

I'm running on Kraken (NICS) cluster. cc is the compiler wrapper they use
on Kraken. And I NEED to use a separate build of MPE for my research.

With regards,
Huong

···

On Wed, Apr 4, 2012 at 4:48 PM, Rob Latham <robl@mcs.anl.gov> wrote:

On Wed, Apr 04, 2012 at 04:18:35PM -0500, huong luu wrote:
> Hi Rob,
>
> That's exactly my problem. I need to record those MPI functions called by
> HDF5. All the I/O in my application is done using HDF5 (there's no
explicit
> MPI I/O function like MPI_File_write ..). So when I link my application
> with just mpe as usual, I can't see those I/O functions.

Oh, you can, but it can be tricky to get the libraries in the right
order.

For example, I'm using the mpi compiler wrappers that came with MPICH
and a separate build of MPE:

$ mpicc h5test.c -o h5test -L${HDF5}/lib -lhdf5 -lm
-L/soft/apps/mpe/install_ibm/lib -llmpe -lmpe

you can also sometimes use your mpi wrapper scripts to do some of this
for you, if you built MPICH2 with MPE support:

$ mpicc -mpe=mpilog h5test.c -o h5test -L${HDF5}/lib -lhdf5 -lm

==rob

--
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA