Success

lewisl · February 23, 2014, 1:17am

Would help if install doc was accurate per platform.

This simple set of instructions worked:

./configure --prefix=/usr/local/hdf5 \
--enable-shared
make
make check
make install
make check-install

And be sure to remind everyone no spaces in directory names. (The shining achievement of 1984—long file names in the Mac OS—has not caught up to Unix tools shipped by Apple.)

···

From: Lewis Levin <lewis@neilson-levin.org<mailto:lewis@neilson-levin.org>>
Date: Saturday, February 22, 2014 at 4:48 PM
To: "hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>" <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: Re: followed install instructions to the letter results in failure on OS X

On July 17, Elena Pourmal writes that thread safe doesn’t work on os x. How is someone to know this if you fail to document it in the supplied installation instructions. If you need to include platform specific instructions, then do so. Please don’t respond, “we’re volunteers; it’s open source. Please—you fix it. “ You are the maintainers. You are getting stipends, however inadequate, from the University of Illinois. Really—it IS your responsibility.

From: Lewis Levin <lewis@neilson-levin.org<mailto:lewis@neilson-levin.org>>
Date: Saturday, February 22, 2014 at 4:44 PM
To: "hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>" <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: followed install instructions to the letter results in failure on OS X

I decided to bite the bullet and just compile instead of expecting you to care enough about your binary distributions to actually test them. Seemed pretty easy from the instructions.

Here is what I did:

./configure --prefix=/usr/local/hdf5 \
    --enable-shared \
    --enable-production \
    --enable-threadsafe
make
make check
make install
make check-install

In addition to the snippets below, I can send the entire output stream, which includes the config info and perhaps hundreds of warnings. Needless to say, the library was not installed. The error messages are useless to me. I have read & write privileges on /usr/local so that does not appear to be the problem. Perhaps you can figure it out.

After many checks and lots of warnings the build errored out. Wish you guys would ship stuff that actually worked with documentation that was accurate.

Here are the error snippets:

~~ lots of messages… …then

16 warnings and 2 errors generated.
make[2]: *** [H5TS.lo] Error 1
make[1]: *** [all] Error 2
make: *** [all-recursive] Error 1

~~ lots of messages… …then
16 warnings and 2 errors generated.
make[1]: *** [H5TS.lo] Error 1
make: *** [check-recursive] Error 1

~~ lots of messages… …then

make[3]: *** No rule to make target `/usr/local/hdf5/bin/h5cc', needed by `h5_write'. Stop.
test -z "h5_write h5_read h5_extend_write h5_chunk_read h5_compound h5_crtgrpd h5_subset h5_cmprss h5_rdwt h5_crtgrpar h5_extend h5_crtatt h5_crtgrp h5_crtdat h5_group h5_select h5_attribute h5_mount h5_reference h5_drivers h5_ref2reg h5_extlink h5_elink_unix2win h5_shared_mesg " || rm -f h5_write h5_read h5_extend_write h5_chunk_read h5_compound h5_crtgrpd h5_subset h5_cmprss h5_rdwt h5_crtgrpar h5_extend h5_crtatt h5_crtgrp h5_crtdat h5_group h5_select h5_attribute h5_mount h5_reference h5_drivers h5_ref2reg h5_extlink h5_elink_unix2win h5_shared_mesg
rm -rf .libs _libs
test -z "*.raw *.meta *.o" || rm -f *.raw *.meta *.o
rm -f *.lo
rm -f -rf *.chkexe *.chklog *.clog red blue u2w *.h5
make[3]: *** No rule to make target `/usr/local/hdf5/bin/h5cc', needed by `h5_write'. Stop.
make[2]: *** [installcheck-local] Error 2
make[1]: *** [installcheck-local] Error 1
make: *** [installcheck-recursive] Error 1

This is the config output from the process:

SUMMARY OF THE HDF5 CONFIGURATION
=================================

General Information:
-------------------
                   HDF5 Version: 1.8.12
                  Configured on: Sat Feb 22 16:23:25 PST 2014
                  Configured by: lewislevinmbr@Lewiss-MacBook-Pro.local<mailto:lewislevinmbr@Lewiss-MacBook-Pro.local>
                 Configure mode: production
                    Host system: i386-apple-darwin13.0.2
              Uname information: Darwin Lewiss-MacBook-Pro.local 13.0.2 Darwin Kernel Version 13.0.2: Sun Sep 29 19:38:57 PDT 2013; root:xnu-2422.75.4~1/RELEASE_X86_64 x86_64
                       Byte sex: little-endian
                      Libraries: static, shared
             Installation point: /usr/local/hdf5

Compiling Options:
------------------
               Compilation Mode: production
                     C Compiler: /usr/bin/clang ( Apple LLVM version 5.0 )
                         CFLAGS:
                      H5_CFLAGS:
                      AM_CFLAGS:
                       CPPFLAGS:
                    H5_CPPFLAGS: -DNDEBUG -UH5_DEBUG_API
                    AM_CPPFLAGS:
               Shared C Library: yes
               Static C Library: yes
  Statically Linked Executables: no
                        LDFLAGS:
                     H5_LDFLAGS:
                     AM_LDFLAGS:
                Extra libraries: -lz -ldl -lm
                       Archiver: ar
                         Ranlib: ranlib
              Debugged Packages:
                    API Tracing: no

Languages:
----------
Fortran: no

C++: no

Features:
---------
                  Parallel HDF5: no
             High Level library: yes
                   Threadsafety: yes
            Default API Mapping: v18
With Deprecated Public Symbols: yes
         I/O filters (external): deflate(zlib)
         I/O filters (internal): shuffle,fletcher32,nbit,scaleoffset
                            MPE: no
                     Direct VFD: no
                        dmalloc: no
Clear file buffers before write: yes
           Using memory checker: no
         Function Stack Tracing: no
                           GPFS: no
      Strict File Format Checks: no
   Optimization Instrumentation: no
       Large File Support (LFS): yes

andre · February 24, 2014, 1:54pm

Hi Lewis,

Your frustration is clear and understandable.
Not to distract from your point - hdf5 library should ship without a hardlinked dynlib location that does not exist on your machine - just trying to help your goal of a simple install script.

I know you do not want to rely on homebrew, but since they have figured out how to get it to install successfully, what about copying their instal script when designing yours? (I guess it is now hosted at homebrew-science). I suppose your original point, that your users may not have xcode clt makes that an unsatisfactory solution.

For using the pre-built binary, can you use “automator” to manage the installation with a step of using xcode to alter the dynlib path? I know one can do this with xcode clt with the `install_name_tool` - I assume xcode can do this without clt, but I have never tried.

I think it would be sage advice to anyone doing scientific applications with python and a mac to install xcode and clt. Maybe a standalone script which accomplishes this? Even if users are not comfortable with installations other than .dmg style, if they are in the sciences, it is to their benefit to become more familiar with it. And you goal of making simple install scripts could help encourage them in this direction also.

Just some advice from a person who has recently been in the frustrated position of installing all my software on a clean mavericks OS (this time I bit the bullet to build it all from the ground up with only the help of google and homebrew).

Regards,

Andre

···

On Feb 22, 2014, at 8:17 PM, Lewis Levin <lewis@neilson-levin.org> wrote:

Would help if install doc was accurate per platform.

This simple set of instructions worked:

./configure --prefix=/usr/local/hdf5 \
    --enable-shared
make
make check
make install
make check-install

And be sure to remind everyone no spaces in directory names. (The shining achievement of 1984—long file names in the Mac OS—has not caught up to Unix tools shipped by Apple.)

From: Lewis Levin <lewis@neilson-levin.org>
Date: Saturday, February 22, 2014 at 4:48 PM
To: "hdf-forum@lists.hdfgroup.org" <hdf-forum@lists.hdfgroup.org>
Subject: Re: followed install instructions to the letter results in failure on OS X

On July 17, Elena Pourmal writes that thread safe doesn’t work on os x. How is someone to know this if you fail to document it in the supplied installation instructions. If you need to include platform specific instructions, then do so. Please don’t respond, “we’re volunteers; it’s open source. Please—you fix it. “ You are the maintainers. You are getting stipends, however inadequate, from the University of Illinois. Really—it IS your responsibility.

From: Lewis Levin <lewis@neilson-levin.org>
Date: Saturday, February 22, 2014 at 4:44 PM
To: "hdf-forum@lists.hdfgroup.org" <hdf-forum@lists.hdfgroup.org>
Subject: followed install instructions to the letter results in failure on OS X

I decided to bite the bullet and just compile instead of expecting you to care enough about your binary distributions to actually test them. Seemed pretty easy from the instructions.

Here is what I did:

./configure --prefix=/usr/local/hdf5 \
    --enable-shared \
    --enable-production \
    --enable-threadsafe
make
make check
make install
make check-install

In addition to the snippets below, I can send the entire output stream, which includes the config info and perhaps hundreds of warnings. Needless to say, the library was not installed. The error messages are useless to me. I have read & write privileges on /usr/local so that does not appear to be the problem. Perhaps you can figure it out.

After many checks and lots of warnings the build errored out. Wish you guys would ship stuff that actually worked with documentation that was accurate.

Here are the error snippets:

~~ lots of messages… …then

16 warnings and 2 errors generated.
make[2]: *** [H5TS.lo] Error 1
make[1]: *** [all] Error 2
make: *** [all-recursive] Error 1

~~ lots of messages… …then
16 warnings and 2 errors generated.
make[1]: *** [H5TS.lo] Error 1
make: *** [check-recursive] Error 1

~~ lots of messages… …then

make[3]: *** No rule to make target `/usr/local/hdf5/bin/h5cc', needed by `h5_write'. Stop.
test -z "h5_write h5_read h5_extend_write h5_chunk_read h5_compound h5_crtgrpd h5_subset h5_cmprss h5_rdwt h5_crtgrpar h5_extend h5_crtatt h5_crtgrp h5_crtdat h5_group h5_select h5_attribute h5_mount h5_reference h5_drivers h5_ref2reg h5_extlink h5_elink_unix2win h5_shared_mesg " || rm -f h5_write h5_read h5_extend_write h5_chunk_read h5_compound h5_crtgrpd h5_subset h5_cmprss h5_rdwt h5_crtgrpar h5_extend h5_crtatt h5_crtgrp h5_crtdat h5_group h5_select h5_attribute h5_mount h5_reference h5_drivers h5_ref2reg h5_extlink h5_elink_unix2win h5_shared_mesg
rm -rf .libs _libs
test -z "*.raw *.meta *.o" || rm -f *.raw *.meta *.o
rm -f *.lo
rm -f -rf *.chkexe *.chklog *.clog red blue u2w *.h5
make[3]: *** No rule to make target `/usr/local/hdf5/bin/h5cc', needed by `h5_write'. Stop.
make[2]: *** [installcheck-local] Error 2
make[1]: *** [installcheck-local] Error 1
make: *** [installcheck-recursive] Error 1

This is the config output from the process:

            SUMMARY OF THE HDF5 CONFIGURATION
            =================================

General Information:
-------------------
                   HDF5 Version: 1.8.12
                  Configured on: Sat Feb 22 16:23:25 PST 2014
                  Configured by: lewislevinmbr@Lewiss-MacBook-Pro.local
                 Configure mode: production
                    Host system: i386-apple-darwin13.0.2
              Uname information: Darwin Lewiss-MacBook-Pro.local 13.0.2 Darwin Kernel Version 13.0.2: Sun Sep 29 19:38:57 PDT 2013; root:xnu-2422.75.4~1/RELEASE_X86_64 x86_64
                       Byte sex: little-endian
                      Libraries: static, shared
             Installation point: /usr/local/hdf5

Compiling Options:
------------------
               Compilation Mode: production
                     C Compiler: /usr/bin/clang ( Apple LLVM version 5.0 )
                         CFLAGS:
                      H5_CFLAGS:
                      AM_CFLAGS:
                       CPPFLAGS:
                    H5_CPPFLAGS: -DNDEBUG -UH5_DEBUG_API
                    AM_CPPFLAGS:
               Shared C Library: yes
               Static C Library: yes
  Statically Linked Executables: no
                        LDFLAGS:
                     H5_LDFLAGS:
                     AM_LDFLAGS:
                Extra libraries: -lz -ldl -lm
                       Archiver: ar
                         Ranlib: ranlib
              Debugged Packages:
                    API Tracing: no

Languages:
----------
                        Fortran: no

                            C++: no

Features:
---------
                  Parallel HDF5: no
             High Level library: yes
                   Threadsafety: yes
            Default API Mapping: v18
With Deprecated Public Symbols: yes
         I/O filters (external): deflate(zlib)
         I/O filters (internal): shuffle,fletcher32,nbit,scaleoffset
                            MPE: no
                     Direct VFD: no
                        dmalloc: no
Clear file buffers before write: yes
           Using memory checker: no
         Function Stack Tracing: no
                           GPFS: no
      Strict File Format Checks: no
   Optimization Instrumentation: no
       Large File Support (LFS): yes
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

mike.jackson · February 24, 2014, 2:30pm

Lewis,
I'll acknowledge your frustration in trying to compile a library that is not your own. With the myriad of different build systems this can get difficult to remember all of the nuances of each build system and project. With that in mind here is some advice for using/building HDF5 on OS X.

On OS X there are several different ways to build dynamic libraries with respect to the "rpath" and either way the HDF Group builds the libraries a certain set of developers are going to be upset so it is a "no win" situation.

OS X's linker can encode the install_name into the library as a full absolute path if given the arguments. This is VERY useful when doing development because when I run my executable I don't have to have a copy of the library in the local directory. My executable simply knows where to find the library. This is GOOD. If the libraries are NOT encoded with the full path then I have to set up environment variables such as DYLD_LIBRARY_PATH and/or monkey with launch scripts and/or copy the library into the same library as the executable. So for development we usually want the path in the library.

I personally use CMake to build HDF5 on OS X. There are some flags in the CMake build that help the developer figure out if they should be building the libraries with or without the full installation path encoded in the libraries. Also if your own project uses CMake then there are CMake infrastructure scripts that can help build proper OS X .app bundles with all libraries included and "cleaned up" correctly.

So as an OS X developer you should understand all of these issues and take the appropriate action for your own build system and environment.

My guess is that when the binaries were posted a flag was simply not set correctly and a mistake was made. That's all. I am sure it will be corrected soon.

Also, there is a tool called "install_name_tool" that you can use to actually _change_ the encoded path to match where the libraries are installed on your system. do "man install_name_tool" to see how to use the tool.

If you use CMake to build HDF5 then setting the cmake variable HDF5_BUILD_WITH_INSTALL_NAME to ON or OFF will control if the install path is encoded in the library.

cd hdf5-1.8.12
mkdir Build
cd Build
cmake -DHDF5_BUILD_WITH_INSTALL_NAME=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local/hdf5-1.8.12 -DBUILD_SHARED_LIBS=ON ../
make -j
make install

Hope all this helps you get HDF5 running a bit quicker on your system
Thanks
Mike Jackson

···

On Feb 22, 2014, at 8:17 PM, Lewis Levin <lewis@neilson-levin.org> wrote:

Would help if install doc was accurate per platform.

This simple set of instructions worked:

./configure --prefix=/usr/local/hdf5 \
    --enable-shared
make
make check
make install
make check-install

And be sure to remind everyone no spaces in directory names. (The shining achievement of 1984—long file names in the Mac OS—has not caught up to Unix tools shipped by Apple.)

From: Lewis Levin <lewis@neilson-levin.org>
Date: Saturday, February 22, 2014 at 4:48 PM
To: "hdf-forum@lists.hdfgroup.org" <hdf-forum@lists.hdfgroup.org>
Subject: Re: followed install instructions to the letter results in failure on OS X

On July 17, Elena Pourmal writes that thread safe doesn’t work on os x. How is someone to know this if you fail to document it in the supplied installation instructions. If you need to include platform specific instructions, then do so. Please don’t respond, “we’re volunteers; it’s open source. Please—you fix it. “ You are the maintainers. You are getting stipends, however inadequate, from the University of Illinois. Really—it IS your responsibility.

From: Lewis Levin <lewis@neilson-levin.org>
Date: Saturday, February 22, 2014 at 4:44 PM
To: "hdf-forum@lists.hdfgroup.org" <hdf-forum@lists.hdfgroup.org>
Subject: followed install instructions to the letter results in failure on OS X

I decided to bite the bullet and just compile instead of expecting you to care enough about your binary distributions to actually test them. Seemed pretty easy from the instructions.

Here is what I did:

./configure --prefix=/usr/local/hdf5 \
    --enable-shared \
    --enable-production \
    --enable-threadsafe
make
make check
make install
make check-install

In addition to the snippets below, I can send the entire output stream, which includes the config info and perhaps hundreds of warnings. Needless to say, the library was not installed. The error messages are useless to me. I have read & write privileges on /usr/local so that does not appear to be the problem. Perhaps you can figure it out.

After many checks and lots of warnings the build errored out. Wish you guys would ship stuff that actually worked with documentation that was accurate.

Here are the error snippets:

~~ lots of messages… …then

16 warnings and 2 errors generated.
make[2]: *** [H5TS.lo] Error 1
make[1]: *** [all] Error 2
make: *** [all-recursive] Error 1

~~ lots of messages… …then
16 warnings and 2 errors generated.
make[1]: *** [H5TS.lo] Error 1
make: *** [check-recursive] Error 1

~~ lots of messages… …then

make[3]: *** No rule to make target `/usr/local/hdf5/bin/h5cc', needed by `h5_write'. Stop.
test -z "h5_write h5_read h5_extend_write h5_chunk_read h5_compound h5_crtgrpd h5_subset h5_cmprss h5_rdwt h5_crtgrpar h5_extend h5_crtatt h5_crtgrp h5_crtdat h5_group h5_select h5_attribute h5_mount h5_reference h5_drivers h5_ref2reg h5_extlink h5_elink_unix2win h5_shared_mesg " || rm -f h5_write h5_read h5_extend_write h5_chunk_read h5_compound h5_crtgrpd h5_subset h5_cmprss h5_rdwt h5_crtgrpar h5_extend h5_crtatt h5_crtgrp h5_crtdat h5_group h5_select h5_attribute h5_mount h5_reference h5_drivers h5_ref2reg h5_extlink h5_elink_unix2win h5_shared_mesg
rm -rf .libs _libs
test -z "*.raw *.meta *.o" || rm -f *.raw *.meta *.o
rm -f *.lo
rm -f -rf *.chkexe *.chklog *.clog red blue u2w *.h5
make[3]: *** No rule to make target `/usr/local/hdf5/bin/h5cc', needed by `h5_write'. Stop.
make[2]: *** [installcheck-local] Error 2
make[1]: *** [installcheck-local] Error 1
make: *** [installcheck-recursive] Error 1

This is the config output from the process:

            SUMMARY OF THE HDF5 CONFIGURATION
            =================================

General Information:
-------------------
                   HDF5 Version: 1.8.12
                  Configured on: Sat Feb 22 16:23:25 PST 2014
                  Configured by: lewislevinmbr@Lewiss-MacBook-Pro.local
                 Configure mode: production
                    Host system: i386-apple-darwin13.0.2
              Uname information: Darwin Lewiss-MacBook-Pro.local 13.0.2 Darwin Kernel Version 13.0.2: Sun Sep 29 19:38:57 PDT 2013; root:xnu-2422.75.4~1/RELEASE_X86_64 x86_64
                       Byte sex: little-endian
                      Libraries: static, shared
             Installation point: /usr/local/hdf5

Compiling Options:
------------------
               Compilation Mode: production
                     C Compiler: /usr/bin/clang ( Apple LLVM version 5.0 )
                         CFLAGS:
                      H5_CFLAGS:
                      AM_CFLAGS:
                       CPPFLAGS:
                    H5_CPPFLAGS: -DNDEBUG -UH5_DEBUG_API
                    AM_CPPFLAGS:
               Shared C Library: yes
               Static C Library: yes
  Statically Linked Executables: no
                        LDFLAGS:
                     H5_LDFLAGS:
                     AM_LDFLAGS:
                Extra libraries: -lz -ldl -lm
                       Archiver: ar
                         Ranlib: ranlib
              Debugged Packages:
                    API Tracing: no

Languages:
----------
                        Fortran: no

                            C++: no

Features:
---------
                  Parallel HDF5: no
             High Level library: yes
                   Threadsafety: yes
            Default API Mapping: v18
With Deprecated Public Symbols: yes
         I/O filters (external): deflate(zlib)
         I/O filters (internal): shuffle,fletcher32,nbit,scaleoffset
                            MPE: no
                     Direct VFD: no
                        dmalloc: no
Clear file buffers before write: yes
           Using memory checker: no
         Function Stack Tracing: no
                           GPFS: no
      Strict File Format Checks: no
   Optimization Instrumentation: no
       Large File Support (LFS): yes
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org