Hello,
I am trying to update a numerical model (RAMS) so that it has distributive memory capabilities when run in parallel. I would like to do this using PHDF5. Currently, RAMS uses MPI for the parallel computing, but it assembles the grids on a single node before writing to HDF5 in serial. I have re-built both MPI and HDF5 locally for this upgrade. Here is my problem:
While compiling RAMS with the new parallel HDF5 libraries, the loader stage complains that within the HDF5 libraries it has "undefined reference to" MANY MPI routines. If I look in the HDF5 .a libraries (using nm), I see the references. I have tried adding MPI libraries to the loader using -lxxx, but that does not seem to solve my problem. Because of this, I have tried many different configurations of the libraries. My latest configurations are:
MPI
./configure --prefix=/home/rseigel/mpich2-1.5-install --enable-romio --enable-mpe 2>&1 | tee c.txt
HDF5
./configure --prefix=/home/rseigel/hdf5-1.8.10 --enable-fortran --enable-parallel --enable-shared=no CFLAGS="-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64" CC=/home/rseigel/mpich2-1.5-install/bin/mpicc
I have checked all of the paths in my environment and the makefile for RAMS numerous times. Everything is okay from that standpoint. I am not sure what to do next, as I do not know why there are undefined references.
Thanks!
Rob
Here are the loader errors:
/usr/local/pgi/linux86-64/8.0-4/bin/pgf90 -o rams-6.0-opt rammain.o -v -Wl,-zmuldefs -Wl, -ldl -lc -lgcc_eh -lpthread -lsz rams-6.0-opt.a \
./utils/libutils-2.4-opt.a -L./utils -lncarg-2.4 -L/home/rseigel/mpich2-1.5-install/lib -lmpich -lmpl -lmpichf90 -ltmpe -lampe -lfmpich -llmpe -lmpe -lmpe_collchk -lmpe_f2cmpi -lmpe_nompi -lmpe_nompi_null -lmpe_null -lmpichcxx -lopa -L/usr/lib64 -L/home/rseigel/hdf5-1.8.10/lib -lhdf5 -L/usr/lib64 -lsz -lz -lm
/usr/bin/ld /usr/lib64/crt1.o /usr/lib64/crti.o /usr/local/pgi/linux86-64/8.0-4/lib/trace_init.o /usr/lib/gcc/x86_64-redhat-linux/4.1.2/crtbegin.o /usr/local/pgi/linux86-64/8.0-4/lib/f90main.o -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /usr/local/pgi/linux86-64/8.0-4/lib/pgi.ld -L./utils -L/home/rseigel/mpich2-1.5-install/lib -L/usr/lib64 -L/home/rseigel/hdf5-1.8.10/lib -L/usr/lib64 -L/usr/local/pgi/linux86-64/8.0-4/lib -L/usr/lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 rammain.o -zmuldefs -ldl -lc -lgcc_eh -lpthread -lsz rams-6.0-opt.a ./utils/libutils-2.4-opt.a -lncarg-2.4 -lmpich -lmpl -lmpichf90 -ltmpe -lampe -lfmpich -llmpe -lmpe -lmpe_collchk -lmpe_f2cmpi -lmpe_nompi -lmpe_nompi_null -lmpe_null -lmpichcxx -lopa -lhdf5 -lsz -lz -lm -rpath /usr/local/pgi/linux86-64/8.0-4/lib -o rams-6.0-opt -lpgf90 -lpgf90_rpm1 -lpgf902 -lpgf90rtl -lpgftnrtl -lnspgc -lpgc -lrt -lpthread -lm -lgcc -lc -lgcc /usr/lib/gcc/x86_64-redhat-linux/4.1.2/crtend.o /usr/lib64/crtn.o
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5AC.o): In function `H5AC_propagate_flushed_and_still_clean_entries_list':
H5AC.c:(.text+0xf03): undefined reference to `MPI_Error_string'
H5AC.c:(.text+0xfdd): undefined reference to `MPI_Error_string'
H5AC.c:(.text+0x10aa): undefined reference to `MPI_Error_string'
H5AC.c:(.text+0x12ea): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5AC.o): In function `H5AC_rsp__p0_only__flush_to_min_clean':
H5AC.c:(.text+0x15a2): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5AC.o):H5AC.c:(.text+0x177c): more undefined references to `MPI_Error_string' follow
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5Eint.o): In function `H5E_walk2_cb':
H5Eint.c:(.text+0xa52): undefined reference to `MPI_Initialized'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5Eint.o): In function `H5E_walk1_cb':
H5Eint.c:(.text+0xcc7): undefined reference to `MPI_Initialized'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpi.o): In function `H5FD_mpi_comm_info_free':
H5FDmpi.c:(.text+0x273): undefined reference to `MPI_Info_free'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpi.o): In function `H5FD_mpi_comm_info_dup':
H5FDmpi.c:(.text+0x337): undefined reference to `MPI_Info_dup'
H5FDmpi.c:(.text+0x3c8): undefined reference to `MPI_Info_free'
H5FDmpi.c:(.text+0x3ed): undefined reference to `MPI_Error_string'
H5FDmpi.c:(.text+0x460): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpio.o): In function `H5FD_get_mpio_atomicity':
H5FDmpio.c:(.text+0xf2): undefined reference to `MPI_File_get_atomicity'
H5FDmpio.c:(.text+0x12b): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpio.o): In function `H5FD_set_mpio_atomicity':
H5FDmpio.c:(.text+0x1c3): undefined reference to `MPI_File_set_atomicity'
H5FDmpio.c:(.text+0x1e1): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpio.o): In function `H5FD_mpio_truncate':
H5FDmpio.c:(.text+0x29c): undefined reference to `MPI_File_set_size'
H5FDmpio.c:(.text+0x2d3): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x35e): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpio.o): In function `H5FD_mpio_flush':
H5FDmpio.c:(.text+0x405): undefined reference to `MPI_File_sync'
H5FDmpio.c:(.text+0x41a): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpio.o): In function `H5FD_mpio_write':
H5FDmpio.c:(.text+0x541): undefined reference to `MPI_File_write_at'
H5FDmpio.c:(.text+0x55d): undefined reference to `MPI_Get_elements'
H5FDmpio.c:(.text+0x573): undefined reference to `MPI_Type_size'
H5FDmpio.c:(.text+0x63c): undefined reference to `MPI_File_set_view'
H5FDmpio.c:(.text+0x681): undefined reference to `MPI_File_write_at_all'
H5FDmpio.c:(.text+0x6a8): undefined reference to `MPI_File_set_view'
H5FDmpio.c:(.text+0x6c1): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x773): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x81c): undefined reference to `MPI_File_write_at'
H5FDmpio.c:(.text+0x835): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x8ad): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x920): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x9e7): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0xaae): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpio.o): In function `H5FD_mpio_read':
H5FDmpio.c:(.text+0xbc1): undefined reference to `MPI_File_read_at'
H5FDmpio.c:(.text+0xbdd): undefined reference to `MPI_Get_elements'
H5FDmpio.c:(.text+0xbf3): undefined reference to `MPI_Type_size'
H5FDmpio.c:(.text+0xd1d): undefined reference to `MPI_File_set_view'
H5FDmpio.c:(.text+0xd64): undefined reference to `MPI_File_read_at_all'
H5FDmpio.c:(.text+0xd8d): undefined reference to `MPI_File_set_view'
H5FDmpio.c:(.text+0xda6): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0xe47): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0xeca): undefined reference to `MPI_File_read_at'
H5FDmpio.c:(.text+0xee3): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0xf5d): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0xfd0): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x1097): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x115e): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpio.o): In function `H5FD_mpio_close':
H5FDmpio.c:(.text+0x11dd): undefined reference to `MPI_File_close'
H5FDmpio.c:(.text+0x120f): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpio.o): In function `H5FD_mpio_open':
H5FDmpio.c:(.text+0x1ecf): undefined reference to `MPI_File_open'
H5FDmpio.c:(.text+0x1ee8): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x1f82): undefined reference to `MPI_Info_free'
H5FDmpio.c:(.text+0x1fc3): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x203d): undefined reference to `MPI_File_close'
H5FDmpio.c:(.text+0x216d): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x2231): undefined reference to `MPI_File_get_size'
H5FDmpio.c:(.text+0x2253): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x22d9): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x2347): undefined reference to `MPI_File_set_size'
H5FDmpio.c:(.text+0x2360): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x2442): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpio.o): In function `H5Pget_fapl_mpio':
H5FDmpio.c:(.text+0x2909): undefined reference to `MPI_Info_dup'
H5FDmpio.c:(.text+0x29aa): undefined reference to `MPI_Error_string'
H5FDmpio.c:(.text+0x2a1d): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpiposix.o): In function `H5FD_mpiposix_truncate':
H5FDmpiposix.c:(.text+0x1f8): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpiposix.o): In function `H5FD_mpiposix_fapl_get':
H5FDmpiposix.c:(.text+0x9c3): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpiposix.o): In function `H5FD_mpiposix_fapl_copy':
H5FDmpiposix.c:(.text+0xac2): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5FDmpiposix.o):H5FDmpiposix.c:(.text+0xe29): more undefined references to `MPI_Error_string' follow
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5Smpio.o): In function `H5S_obtain_datatype':
H5Smpio.c:(.text+0x462): undefined reference to `MPI_Type_hindexed'
H5Smpio.c:(.text+0x47b): undefined reference to `MPI_Error_string'
H5Smpio.c:(.text+0x52a): undefined reference to `MPI_Error_string'
H5Smpio.c:(.text+0x59d): undefined reference to `MPI_Error_string'
H5Smpio.c:(.text+0x744): undefined reference to `MPI_Error_string'
H5Smpio.c:(.text+0x7b7): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5Smpio.o): In function `H5S_mpio_space_type':
H5Smpio.c:(.text+0xecd): undefined reference to `MPI_Type_extent'
H5Smpio.c:(.text+0x10a2): undefined reference to `MPI_Error_string'
H5Smpio.c:(.text+0x1128): undefined reference to `MPI_Error_string'
H5Smpio.c:(.text+0x1202): undefined reference to `MPI_Error_string'
H5Smpio.c:(.text+0x1275): undefined reference to `MPI_Error_string'
H5Smpio.c:(.text+0x1302): undefined reference to `MPI_Error_string'
/home/rseigel/hdf5-1.8.10/lib/libhdf5.a(H5Smpio.o):H5Smpio.c:(.text+0x13a7): more undefined references to `MPI_Error_string' follow
pgf90-Fatal-linker completed with exit code 1
make: *** [rams-6.0-opt] Error 2
···
____________________________
Robert Seigel, Ph.D.
Colorado State University
Department of Atmospheric Science
1371 Campus Delivery
Fort Collins, CO 80523
(970) 491-8331
rseigel@atmos.colostate.edu