Hi,
I understand that HDF5 cannot be build as universal binary directly. We need to build e.g. i386 and ppc architecture separately and combined them using lipo.
Is there a write up on how one can go about doing that? i.e. best practice approach.
Thanks in advanced.
Regards
Hi Nicholas,
Hi,
I understand that HDF5 cannot be build as universal binary directly. We need to build e.g. i386 and ppc architecture separately and combined them using lipo.
Is there a write up on how one can go about doing that? i.e. best practice approach.
In the past I only built universal binaries for HDF5 and HDF4 command line utilities (not libraries). I used different machines and didn't not use compiler flags on the same machine (but tried it both ways; unfortunately I couldn't make binaries built on 64-bit Intel to work on 32-bit Intel). Here is the procedure I followed:
0. Make sure that the same versions of GNU compilers are used on all desired platforms (I suspect this may be not required).
1. Build and install universal binaries for ZLIB and SZIP libraries for desired architectures (build static versions, and then use lipo to create universal binaries for libraries; test them)
2. For each architecture configure and build static HDF5 C library and utilities by using --disable-shared; link with the universal binaries for ZLIB and SZIP built in 1; for example,
./configure --disable-shared --with-zlib=/path_to_universal_zlib --with-szlib=/path_to_universal_szip --prefix=/<install_path>
make
make check
make install
3. Bring utilities installed under <install_path>/bin on each of the machines to a directory on a machine of your choice; I usually just tar them, ftp, then untar.
4. Use a script to combine utilities; put universal binaries in a separate directory. Here is an example of a rudimentary script to combine HDF5 utilities built for 32 and 64 Mac Intel
#/bin/sh
lipo _tmp64/hdf5/bin/h5copy _tmp32/hdf5/bin/h5copy -output _univ/bin/h5copy -create
lipo _tmp64/hdf5/bin/h5debug _tmp32/hdf5/bin/h5debug -output _univ/bin/h5debug -create
lipo _tmp64/hdf5/bin/h5diff _tmp32/hdf5/bin/h5diff -output _univ/bin/h5diff -create
lipo _tmp64/hdf5/bin/h5dump _tmp32/hdf5/bin/h5dump -output _univ/bin/h5dump -create
lipo _tmp64/hdf5/bin/h5import _tmp32/hdf5/bin/h5import -output _univ/bin/h5import -create
lipo _tmp64/hdf5/bin/h5jam _tmp32/hdf5/bin/h5jam -output _univ/bin/h5jam -create
lipo _tmp64/hdf5/bin/h5ls _tmp32/hdf5/bin/h5ls -output _univ/bin/h5ls -create
lipo _tmp64/hdf5/bin/h5mkgrp _tmp32/hdf5/bin/h5mkgrp -output _univ/bin/h5mkgrp -create
lipo _tmp64/hdf5/bin/h5perf_serial _tmp32/hdf5/bin/h5perf_serial -output _univ/bin/h5perf_serial -create
lipo _tmp64/hdf5/bin/h5repack _tmp32/hdf5/bin/h5repack -output _univ/bin/h5repack -create
lipo _tmp64/hdf5/bin/h5repart _tmp32/hdf5/bin/h5repart -output _univ/bin/h5repart -create
lipo _tmp64/hdf5/bin/h5stat _tmp32/hdf5/bin/h5stat -output _univ/bin/h5stat -create
lipo _tmp64/hdf5/bin/h5unjam _tmp32/hdf5/bin/h5unjam -output _univ/bin/h5unjam -create
lipo _tmp64/hdf5/bin/h52gif _tmp32/hdf5/bin/h52gif -output _univ/bin/h52gif -create
lipo _tmp64/hdf5/bin/gif2h5 _tmp32/hdf5/bin/gif2h5 -output _univ/bin/gif2h5 -create
5. Fun part: How to test? 
After building universal binaries, I tested them manually on each architecture. I also tried to replace binaries built in 2 with the new ones and run "make check" - tedious but it worked.
It is possible to create libraries too, but libraries require much more manual work to create proper public header files and testing is hard.
We stopped building universal binaries for the HDF5 utilities because we noticed that no one was using them and none of our funding agencies requested them. If there is enough interest we may reconsider our decision.
Thank you!
Elena
On Jun 29, 2011, at 3:49 PM, Nicholas Yue wrote:
Thanks in advanced.
Regards
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Is the CMake support now in working order? Because with CMake, you just have to set CMAKE_OSX_ARCHITECTURES and it will build universal. (Assuming results of 'try runs' are not used to decide things like endianess and pointer size.)
On Tue, 5 Jul 2011 12:39:57 +0400, Elena Pourmal said:
In the past I only built universal binaries for HDF5 and HDF4 command
line utilities (not libraries). I used different machines and didn't not
use compiler flags on the same machine (but tried it both ways;
unfortunately I couldn't make binaries built on 64-bit Intel to work on
32-bit Intel). Here is the procedure I followed:
--
____________________________________________________________
Sean McBride, B. Eng sean@rogue-research.com
Rogue Research www.rogue-research.com
Mac Software Developer Montréal, Québec, Canada
Not that easy. There is a program that is generated that is then run to get very specific details about floating point conversions and things like that during the build. The output from that program generates a source file that is included in the build of the library. For my own HDF5 1.6.9 repo I did all the necessary additions to actually make the source file that was produced universal binary compatible. I offered it for inclusion into the latest HDF5 sources but was declined. So about the best you can do is come up with a script that uses cmake to create 4 build trees for HDF5 and then lipo everything together at the end.
On Jul 5, 2011, at 12:09 PM, Sean McBride wrote:
On Tue, 5 Jul 2011 12:39:57 +0400, Elena Pourmal said:
In the past I only built universal binaries for HDF5 and HDF4 command
line utilities (not libraries). I used different machines and didn't not
use compiler flags on the same machine (but tried it both ways;
unfortunately I couldn't make binaries built on 64-bit Intel to work on
32-bit Intel). Here is the procedure I followed:Is the CMake support now in working order? Because with CMake, you just have to set CMAKE_OSX_ARCHITECTURES and it will build universal. (Assuming results of 'try runs' are not used to decide things like endianess and pointer size.)
--
____________________________________________________________
Sean McBride, B. Eng sean@rogue-research.com
Rogue Research www.rogue-research.com
Mac Software Developer Montréal, Québec, Canada
___________________________________________________________
Mike Jackson www.bluequartz.net
Principal Software Engineer mike.jackson@bluequartz.net
BlueQuartz Software Dayton, Ohio
Oh. Forget what I said then.
On Tue, 5 Jul 2011 12:24:59 -0400, Michael Jackson said:
Not that easy. There is a program that is generated that is then run to
get very specific details about floating point conversions and things
like that during the build. The output from that program generates a
source file that is included in the build of the library. For my own
HDF5 1.6.9 repo I did all the necessary additions to actually make the
source file that was produced universal binary compatible. I offered it
for inclusion into the latest HDF5 sources but was declined. So about
the best you can do is come up with a script that uses cmake to create 4
build trees for HDF5 and then lipo everything together at the end.
--
____________________________________________________________
Sean McBride, B. Eng sean@rogue-research.com
Rogue Research www.rogue-research.com
Mac Software Developer Montréal, Québec, Canada
