HDF5 using my own libz (zlib)


#1

I am trying to use a newer version of zlib (which I compiled myself) with hdf5-1.8.20, using pgfortran. I’ve done like this (in tcsh):

setenv ZLIB /usr/local/src/zlib-1.2.11

setenv CPPFLAGS “-I$ZLIB/include”
setenv LDFLAGS “-L$ZLIB/lib”
setenv LD_LIBRARY_PATH $ZLIB/lib:$LD_LIBRARY_PATH # note: first in the list

./configure
–prefix=pwd
–enable-fortran
–with-zlib=/usr/local/zlib-1.2.11
>&! my.configure.out

I tried using LDFLAGS “-L$ZLIB/lib -lz” like I would have expected, but had to instead use sed after configure:

sed -i “s|-L/usr/local/src/zlib-1.2.11/lib|-L/usr/local/src/zlib-1.2.11/lib -lz|g” Makefile
sed -i “s|-lz -ldl -lm|-ldl -lm|g” Makefile

After compilation, it still appears that the executables will use the system’s (gcc) version of libz:

ldd bin/h5dump | grep libz

    libz.so.1 => /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/libz.so.1 (0x00007f41f6c9f000)

What am I doing wrong? /usr/local/src/zlib-1.2.11/lib is the first element in $LD_LIBRARY_PATH in my normal environment.

Bart


#2

I have just tried again with hdf5-1.10.6, still no luck. Here’s the only mentions of -lz in the Makefile:

FCLIBS = -L/usr/local/src/zlib-1.2.11/lib -lz -L/usr/local/pgi/linux86-64-nollvm/19.10/lib -L/usr/lib64 -L/usr/li
b/gcc/x86_64-redhat-linux/4.8.5 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/…/…/…/…/lib64 -lpgf90rtl -lpgf90 -lpg
f90_rpm1 -lpgf902 -lpgftnrtl -lpgmp -lnuma -lpthread -lpgmath -lnspgc -lpgc -lrt -lm

LDFLAGS = -L/usr/local/src/zlib-1.2.11/lib -lz

LIBS = -lz -ldl -lm

(Note that I edited the Makefile to move the -lz from after -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 to after -L/usr/local/src/zlib-1.2.11/lib.)

hdf5-1.10.6 already has a bin/ directory, so I could not use --prefix=pwd in the call to configure, I had to use --prefix=pwd/build.

After compilation, it still looks like it’s using the system’s version of zlib:

ldd build/bin/h5copy | eg libz
libz.so.1 => /usr/lib/gcc/x86_64-redhat-linux/4.8.5/…/…/…/…/lib64/libz.so.1 (0x00007f93b434f000)

Does anyone have any hints for me?

Bart


#3

The output of ldd depends on the shell environment. Assuming Bash, what if you export LD_LIBRARY_PATH=/usr/local/src/zlib-1.2.11/lib:$LD_LIBRARY_PATH and then run ldd? G.


#4

Hi Bart,

I see that in your original thread on the topic you already tried the LD_LIBRARY_PATH method (in tcsh) without success. In a case like this I would resort to strace. It can give you some hints as to the actual search which ld is performing when you run the code. For example:

strace -f /home/dkahn/HDF5LinkTest/bin/h5copy | & grep open | grep libz

In my test case it shows both the failures and success of the search (ld always find the library in the last place it looks:wink:):

    ahn: ~/src/hdf5-1.10.6 % setenv LD_LIBRARY_PATH /home/dkahn/lib
    ahn: ~/src/hdf5-1.10.6 % strace -f /home/dkahn/HDF5LinkTest/bin/h5copy | & grep open | grep libz
    open("/home/dkahn/HDF5LinkTest/lib/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    open("/home/dkahn/lib/tls/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    open("/home/dkahn/lib/tls/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    open("/home/dkahn/lib/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    **open("/home/dkahn/lib/libz.so.1", O_RDONLY|O_CLOEXEC) = 3**

If I change the search path to something non-nonsensical we’ll see that ld finds the system installation of the library:

ahn: ~/src/hdf5-1.10.6 % setenv LD_LIBRARY_PATH /home/dkahn/lutfisk
ahn: ~/src/hdf5-1.10.6 % strace -f /home/dkahn/HDF5LinkTest/bin/h5copy | & grep open | grep libz
open("/home/dkahn/HDF5LinkTest/lib/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/dkahn/lutfisk/tls/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/dkahn/lutfisk/tls/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/dkahn/lutfisk/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/dkahn/lutfisk/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/lib64/libz.so.1", O_RDONLY|O_CLOEXEC) = 3

During compilation the linker (at least in the static linking case, so this might not be useful to you) is sensitive to the order in which paths and libraries appear on the command line. The variables names alone (LDFLAGS, LIBS, etc) don’t give the whole picture. You can build the code with:

make V=1

and that will print the full command line to the screen (in the case of HDF5 its a lot of output) so you can see the order in which the linker processing arguments. Sometimes this is useful for identifying library problems.

Good luck.


#5

Thanks, this would apply at runtime too, not just in the call to ldd. I believe my memory leak is coming from my program using /usr/lib64/libz.so.1 instead of /usr/local/src/zlib-1.2.11/lib.

I have $LD_LIBRARY_PATH set in /etc/profile.d/zlib.csh. Here’s the values (with a little tr magic to make it easier to read). You can see that my local self-compiled zlib is early in the list.

echo $LD_LIBRARY_PATH | tr : “\n”

/opt/ohpc/pub/compiler/gcc/7.3.0/lib64

/usr/local/src/zlib-1.2.11/lib

/usr/local/src/hdf5-1.8.20.pgi/lib

/usr/local/src/libpng-1.6.37/lib

/usr/local/src/netcdf-c-4.7.2.pgi/build/lib

/usr/local/pgi/linux86-64/2019/lib

/usr/local/pgi/linux86-64/2019/libso

/usr/local/src/szip-2.1.1/lib

I also have added it like so:

cat /etc/ld.so.conf

include ld.so.conf.d/*.conf

cat /etc/ld.so.conf.d/zlib.conf

/usr/local/src/zlib-1.2.11/lib

sudo ldconfig -v | grep -e libz -e zlib

/usr/local/src/zlib-1.2.11/lib:

libz.so.1 -> libz.so.1.2.11

libzip.so.2 -> libzip.so.2.1.0

libz.so.1 -> libz.so.1.2.7

I believe the fundamental problem is that /usr/lib64/libz.so.1 is the one it’s finding first, and using. /usr/lib64 is not in /etc/csh* or /etc/bash*. It’s not in /etc/ld.so.conf.d:

grep /usr/lib /etc/ld.so.conf.d/*

/etc/ld.so.conf.d/atlas-x86_64.conf:/usr/lib64/atlas

/etc/ld.so.conf.d/dyninst-x86_64.conf:/usr/lib64/dyninst

/etc/ld.so.conf.d/llvm5.0-x86_64.conf:/usr/lib64/llvm5.0/lib

/etc/ld.so.conf.d/llvm-x86_64.conf:/usr/lib64/llvm

/etc/ld.so.conf.d/mariadb-x86_64.conf:/usr/lib64/mysql

/etc/ld.so.conf.d/R-x86_64.conf:/usr/lib64/R/lib

/usr/lib64 is one of the default ‘trusted’ locations, according to the ldconfig man page. But I would think it sane to add user-specified paths before the system-trusted defaults. I can’t figure out how to change that.

Any of this making sense to you?

  • Bart

#6

OK, I recompiled using make V=1 and re-directed to a file. Here’s my current compile script:

cat my.configure

#!/bin/csh -f

make distclean >& /dev/null

setenv FC pgfortran

setenv CC pgcc

setenv CXX pgc++

setenv FFLAGS “-tp=istanbul”

setenv FCFLAGS “-tp=istanbul”

setenv CFLAGS “-tp=istanbul”

setenv ZLIB /usr/local/src/zlib-1.2.11

setenv CPPFLAGS “-I$ZLIB/include”

setenv LDFLAGS “-L$ZLIB/lib”

if !($?LD_LIBRARY_PATH) then

setenv LD_LIBRARY_PATH $ZLIB/lib

else

if ( “$LD_LIBRARY_PATH” !~ $ZLIB/lib ) then

setenv LD_LIBRARY_PATH $ZLIB/lib:${LD_LIBRARY_PATH}

endif

endif

echo $LD_LIBRARY_PATH | tr : “\n”

./configure \

–prefix=pwd/build \

–enable-fortran \

–with-zlib=$ZLIB \

&! my.configure.out

sed -i “s|-L/usr/local/src/zlib-1.2.11/lib|-L/usr/local/src/zlib-1.2.11/lib -lz|g” Makefile

sed -i “s|-L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 -lz|-L/usr/lib/gcc/x86_64-redhat-linux/4.8.5|g” Makefile

make V=1 >&! make.out

make install >&! make.install.out

Yes, lots of output to make.out, a bit hard to parse, but I do see lots of -L/usr/local/src/zlib-1.2.11/lib -lz strings. Some redundancies, like

/bin/sh …/libtool --tag=CC --mode=link pgcc -c99 -Minform=inform -fast -s -tp=istanbul -L/usr/local/src/zlib-1.2.11/lib -L/usr/local/src/zlib-1.2.11/lib -o H5detect H5detect.o -lz -ldl -lm

Calls to ldd still show it using the /usr/lib64 version of zlib. As does strace:

strace -f build/bin/h5copy | & grep open | grep libz

open("/usr/local/src/hdf5-1.10.6.pgi/build/lib/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64-nollvm/19.10/lib/tls/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64-nollvm/19.10/lib/tls/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64-nollvm/19.10/lib/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64-nollvm/19.10/lib/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/lib/gcc/x86_64-redhat-linux/4.8.5/…/…/…/…/lib64/tls/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/lib/gcc/x86_64-redhat-linux/4.8.5/…/…/…/…/lib64/tls/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/lib/gcc/x86_64-redhat-linux/4.8.5/…/…/…/…/lib64/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/lib/gcc/x86_64-redhat-linux/4.8.5/…/…/…/…/lib64/libz.so.1", O_RDONLY|O_CLOEXEC) = 3

Any further ideas? I really appreciate the help.

  • Bart

#7

The results of the strace indicate that the paths in LD_LIBRARY_PATH are being ignored. (There is small possibility that strace is truncating its output such that libz doesn’t appear. You can rerun strace adding the -v option to prevent the truncation.)

The only thing know that would cause this is having the setuid or setgid bits set on the executable, namely h5copy in your test case. By the way I’m assuming that the h5copy in your Build directory is truly the Linux binary–it seems that the HDF5 “make build” process produces an h5copy wrapped in a sh script, and it isn’t until you “make install” that a true binary shows up. You can check this with the file command.

Observe the following examples in which the dynamic linker searches for libc

~/src/hdf5-1.10.6 % echo $LD_LIBRARY_PATH
/home/dkahn/lutfisk

First a program without setuid bit set, we check the permissions for dc

~/src/hdf5-1.10.6 % ls -l /usr/bin/dc
-rwxr-xr-x. 1 root root 45392 Jun 9 2014 /usr/bin/dc

and run using strace, notice it includes the futile search of our LD_LIBRARY_PATH directory path.

~/src/hdf5-1.10.6 % strace -f /usr/bin/dc | & grep libc
open("/home/dkahn/lutfisk/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/dkahn/lutfisk/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/dkahn/lutfisk/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/home/dkahn/lutfisk/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3

Now lets try /usr/bin/crontab, note the ‘s’ in permissions indicates setuid bit set:

~/src/hdf5-1.10.6 % ls -l /usr/bin/crontab
-rwsr-xr-x. 1 root root 57656 Aug 8 2019 /usr/bin/crontab

And the we see that LD_LIBRARY_PATH is not searched.

~/src/hdf5-1.10.6 % strace -f crontab -l | & grep libc
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
open("/lib64/libcap-ng.so.0", O_RDONLY|O_CLOEXEC) = 3

This last behavior reminds me of what you reported.

My understanding of this is that the restriction kicks in if the setuid or setgid bits are set and you running as a user id or group id other than owner or group id of the file. This is a Unix security measure to prevent someone from substituting a system library when running a privileged code. If you inadvertently installed h5copy with these bits sets that could explain what you are seeing.


#8

Yes, the build/bin directory contains the executable, placed there by make install.

file h5copy

h5copy: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, stripped

It does not appear to me that the setuid or setgid bits are set:

\ls -lF h5copy

-rwxr-xr-x. 1 bbrashers users 219696 Mar 3 16:45 h5copy*

(cd …/ ; ls -dlF bin)

drwxrwxr-x. 2 bbrashers 4.0K Mar 3 16:45 bin/

I thought I was on to something when I remembered that CentOS7 assumes 0 < uid < 1000 are reserved for “special” accounts (it changed from 500 to 1000 with CentOS 7) and I had imported my account across many generations of CentOS machines as I upgraded. So I tested using another user’s account:

sudo su –

su – someuser

cd /usr/local/src/hdf5-1.10.6.pgi/build/bin/

echo $uid $gid

17887 100

echo $LD_LIBRARY_PATH

/usr/local/src/zlib-1.2.11/lib:/usr/local/src/szip-2.1.1/lib:/usr/local/pgi/linux86-64/2019/libso:/usr/local/pgi/linux86-64/2019/lib:/usr/local/src/netcdf-c-4.7.2.pgi/build/lib:/opt/ohpc/pub/compiler/gcc/7.3.0/lib64:/usr/local/src/libpng-1.6.37/lib:/usr/local/src/hdf5-1.10.6.pgi/lib

[wclyn-cluster1 build/bin]# strace -fv h5copy | & grep open | grep libz

open("/usr/local/src/hdf5-1.10.6.pgi/build/lib/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64-nollvm/19.10/lib/tls/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64-nollvm/19.10/lib/tls/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64-nollvm/19.10/lib/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64-nollvm/19.10/lib/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/lib/gcc/x86_64-redhat-linux/4.8.5/…/…/…/…/lib64/tls/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/lib/gcc/x86_64-redhat-linux/4.8.5/…/…/…/…/lib64/tls/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/lib/gcc/x86_64-redhat-linux/4.8.5/…/…/…/…/lib64/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/lib/gcc/x86_64-redhat-linux/4.8.5/…/…/…/…/lib64/libz.so.1", O_RDONLY|O_CLOEXEC) = 3

But that’s the same output I get from my account, which is uid = 512, gid = 100.

Would not the sudo su - ; su – someuser completely discard anything associated with uid = 512?

In /etc/login.defs, I see that SYS_GID_MIN = 200, but I have the group users = 100. Could that be the fundamental issue?

I’m using autofs to mount all the contents of /usr/local from another machine (a RAID array which survives OS upgrades) to all the nodes in my cluster, both headnode and compute nodes (RocksClusters.org styling). But the mount options look OK to me:

grep src /etc/auto.local

src -rw,hard,intr,nfsvers=3,noacl,rsize=32768,wsize=32768 storage.local:/state/partition1/local/&

On the storage node, \ls -lF does not show the “s” sticky bit.

Bart


#9

Bart,

You can try the same test I did to see if LD_LIBRARY_PATH is searched before the defaults ones for a system program, namely:

strace -f /usr/bin/dc | & grep libc

If you don’t see it searching /usr/local/src/zlib-1.2.11/lib then I think you’ll know this problem goes beyond the HDF library and tools, and if that’s the case it’s possible that you’re system is running in some kind of secure mode. I’m not very familiar with the this, but you could try this:

~/src/hdf5-1.10.6 % ( setenv LD_SHOW_AUXV 1 ; sleep 1 ) |& grep AT_SECURE
AT_SECURE: 0

If the result is non-zero I think that means that run time linker will disable searching LD_LIBRARY_PATH for all programs. I’m not sure if that’s done on a user-by-user basis for across the board.


#10

Good ideas, both. I think the line in red is the one we should expect to see – and proof the system is searching $LD_LIBRARAY_PATH.

strace -f /usr/bin/dc | & grep libc

open("/opt/ohpc/pub/compiler/gcc/7.3.0/lib64/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/opt/ohpc/pub/compiler/gcc/7.3.0/lib64/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/opt/ohpc/pub/compiler/gcc/7.3.0/lib64/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/opt/ohpc/pub/compiler/gcc/7.3.0/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/zlib-1.2.11/lib/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/zlib-1.2.11/lib/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/zlib-1.2.11/lib/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/zlib-1.2.11/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/szip-2.1.1/lib/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/szip-2.1.1/lib/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/szip-2.1.1/lib/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/szip-2.1.1/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64/2019/libso/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64/2019/libso/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64/2019/libso/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64/2019/libso/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64/2019/lib/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64/2019/lib/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64/2019/lib/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/pgi/linux86-64/2019/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/netcdf-c-4.7.2.pgi/build/lib/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/netcdf-c-4.7.2.pgi/build/lib/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/netcdf-c-4.7.2.pgi/build/lib/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/netcdf-c-4.7.2.pgi/build/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/libpng-1.6.37/lib/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/libpng-1.6.37/lib/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/libpng-1.6.37/lib/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/libpng-1.6.37/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/hdf5-1.10.6.pgi/lib/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/hdf5-1.10.6.pgi/lib/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/hdf5-1.10.6.pgi/lib/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/usr/local/src/hdf5-1.10.6.pgi/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3

( setenv LD_SHOW_AUXV 1 ; sleep 1 ) |& grep AT_SECURE

AT_SECURE: 0

  • Bart

#11

Hi Bart,

I’m interpreting your results to mean that there is one set of search paths used for /usr/bin/dc and other for your install of h5copy. I’m not sure what difference could be causing this. Am I right in assuming that your HDF5 was built from source using PGI? You could try a build using GCC and see that changes things.

On the off chance that the PGI compiler does use a different dynamic linker you could also try readelf, note the line with ‘Requesting program interpreter:’ in the output below which shows the path the the dynamic linker, if it’s different for /usr/bin/dc and your h5copy that could be a clue:

/src/hdf5-1.10.6 % readelf -l /usr/bin/dc

Elf file type is EXEC (Executable file)
Entry point 0x4012f4
There are 9 program headers, starting at offset 64

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
PHDR 0x0000000000000040 0x0000000000400040 0x0000000000400040
0x00000000000001f8 0x00000000000001f8 R E 8
INTERP 0x0000000000000238 0x0000000000400238 0x0000000000400238
0x000000000000001c 0x000000000000001c R 1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000
0x0000000000009974 0x0000000000009974 R E 200000
LOAD 0x0000000000009e10 0x0000000000609e10 0x0000000000609e10
0x0000000000000368 0x0000000000000c38 RW 200000
DYNAMIC 0x0000000000009e28 0x0000000000609e28 0x0000000000609e28
0x00000000000001d0 0x00000000000001d0 RW 8
NOTE 0x0000000000000254 0x0000000000400254 0x0000000000400254
0x0000000000000044 0x0000000000000044 R 4
GNU_EH_FRAME 0x00000000000082a4 0x00000000004082a4 0x00000000004082a4
0x000000000000035c 0x000000000000035c R 4
GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 RW 10
GNU_RELRO 0x0000000000009e10 0x0000000000609e10 0x0000000000609e10
0x00000000000001f0 0x00000000000001f0 R 1


#12

Hi Daniel,

Yes, my HDF5 was built from source using PGI. My ultimate goal is to find a memory leak in a pre-processor for the WRF meteorological program, which requires netCDF, which requires HDF5. I was trying to make sure as many components as possible in the software stack were compiled by the same compiler.

But I will try to compile hdf5-1.10.6 with GCC and get back to you.

Did you see my previous note about my $uid being < 1000 on CentOS 7? It’s a side-effect of retaining the same uid since CentOS 5.x days.

The results from the readelf tests have the same value for “Requesting program interpreter”.

readelf -l /usr/bin/dc

Elf file type is EXEC (Executable file)

Entry point 0x4012f4

There are 9 program headers, starting at offset 64

Program Headers:

Type Offset VirtAddr PhysAddr

FileSiz MemSiz Flags Align

PHDR 0x0000000000000040 0x0000000000400040 0x0000000000400040

0x00000000000001f8 0x00000000000001f8 R E 8

INTERP 0x0000000000000238 0x0000000000400238 0x0000000000400238

0x000000000000001c 0x000000000000001c R 1

** [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]**

LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000

0x0000000000009974 0x0000000000009974 R E 200000

LOAD 0x0000000000009e10 0x0000000000609e10 0x0000000000609e10

0x0000000000000368 0x0000000000000c38 RW 200000

DYNAMIC 0x0000000000009e28 0x0000000000609e28 0x0000000000609e28

0x00000000000001d0 0x00000000000001d0 RW 8

NOTE 0x0000000000000254 0x0000000000400254 0x0000000000400254

0x0000000000000044 0x0000000000000044 R 4

GNU_EH_FRAME 0x00000000000082a4 0x00000000004082a4 0x00000000004082a4

0x000000000000035c 0x000000000000035c R 4

GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000

             0x0000000000000000 0x0000000000000000  RW     10

GNU_RELRO 0x0000000000009e10 0x0000000000609e10 0x0000000000609e10

             0x00000000000001f0 0x00000000000001f0  R      1

Section to Segment mapping:

Segment Sections…

00

01 .interp

02 .interp .note.ABI-tag .note.gnu.build-id .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame_hdr .eh_frame

03 .init_array .fini_array .jcr .dynamic .got .got.plt .data .bss

04 .dynamic

05 .note.ABI-tag .note.gnu.build-id

06 .eh_frame_hdr

07

08 .init_array .fini_array .jcr .dynamic .got

readelf -l /usr/local/src/hdf5-1.10.6.pgi/build/bin/h5copy

Elf file type is EXEC (Executable file)

Entry point 0x404d60

There are 9 program headers, starting at offset 64

Program Headers:

Type Offset VirtAddr PhysAddr

FileSiz MemSiz Flags Align

PHDR 0x0000000000000040 0x0000000000400040 0x0000000000400040

0x00000000000001f8 0x00000000000001f8 R E 8

INTERP 0x0000000000000238 0x0000000000400238 0x0000000000400238

0x000000000000001c 0x000000000000001c R 1

** [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]**

LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000

0x000000000003326c 0x000000000003326c R E 200000

LOAD 0x0000000000033bc8 0x0000000000633bc8 0x0000000000633bc8

0x0000000000001530 0x0000000000001e78 RW 200000

DYNAMIC 0x0000000000033be0 0x0000000000633be0 0x0000000000633be0

0x0000000000000280 0x0000000000000280 RW 8

NOTE 0x0000000000000254 0x0000000000400254 0x0000000000400254

0x0000000000000020 0x0000000000000020 R 4

GNU_EH_FRAME 0x0000000000031fac 0x0000000000431fac 0x0000000000431fac

0x00000000000003b4 0x00000000000003b4 R 4

GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000

             0x0000000000000000 0x0000000000000000  RW     10

GNU_RELRO 0x0000000000033bc8 0x0000000000633bc8 0x0000000000633bc8

             0x0000000000000438 0x0000000000000438  R      1

Section to Segment mapping:

Segment Sections…

00

01 .interp

02 .interp .note.ABI-tag .hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .plt.got .text .fini .rodata .eh_frame_hdr .eh_frame

03 .init_array .fini_array .jcr .dynamic .got .got.plt .data .bss

04 .dynamic

05 .note.ABI-tag

06 .eh_frame_hdr

07

08 .init_array .fini_array .jcr .dynamic .got

  • Bart