Hello,
The HDF5 FAQ states that parallel HDF5 does not support shared
libraries. Currently (as of 1.8.2) the HDF5 configure script does not
enforce this restriction. Does this mean that a version of HDF5 built
with --enable-parallel --enable-shared will not function correctly?
This becomes important for high-level languages like Python (which my
own project h5py uses). Python packages consist of a number of shared
libraries dynamically loaded by the interpreter; in this case, all of
them plug into libhdf5.so.
I ask because now that a stable Python MPI interface exists (mpi4py),
it looks like it would otherwise be trivial to support parallel HDF5
in h5py.
Thanks,
Andrew Collette
Hi Andrew,
In parallel case configure disables shared libraries unless the user explicitly enables them with the --enable-shared option. We will need to fix our FAQ.
But we do not support shared parallel library (i.e., we do not test this option), so please use the option with caution (in other words, please make sure that all tests passed before using the library).
BTW, on which system you will be using it?
Elena
···
On Aug 18, 2009, at 7:22 PM, Andrew Collette wrote:
Hello,
The HDF5 FAQ states that parallel HDF5 does not support shared
libraries. Currently (as of 1.8.2) the HDF5 configure script does not
enforce this restriction. Does this mean that a version of HDF5 built
with --enable-parallel --enable-shared will not function correctly?
This becomes important for high-level languages like Python (which my
own project h5py uses). Python packages consist of a number of shared
libraries dynamically loaded by the interpreter; in this case, all of
them plug into libhdf5.so.
I ask because now that a stable Python MPI interface exists (mpi4py),
it looks like it would otherwise be trivial to support parallel HDF5
in h5py.
Thanks,
Andrew Collette
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Hi Elena,
But we do not support shared parallel library (i.e., we do not test this
option), so please use the option with caution (in other words, please make
sure that all tests passed before using the library).
All tests pass on Linux, for 1.8.3 on 64-bit x86 and 32-bit PPC (the
machines I have immediate access to). It also looks like the default
Debian/Ubuntu package for libhdf5-openmpi (HDF5 1.6.6) ships with
shared libraries. Presumably the package managers have done their own
testing, although I don't know what patchset they've applied. 
BTW, on which system you will be using it?
I would be running it on 64-bit x86 Linux. However, my personal work
has limited benefit from MPI, so this is more of a general
investigation to see if it is feasible across UNIX-ish platforms. A
few people have asked me about using parallel HDF5 from Python, and
now that mpi4py is stable I thought I'd have a go.
Andrew
Hi Andrew,
Hi Elena,
But we do not support shared parallel library (i.e., we do not test this
option), so please use the option with caution (in other words, please make
sure that all tests passed before using the library).
All tests pass on Linux, for 1.8.3 on 64-bit x86 and 32-bit PPC (the
machines I have immediate access to). It also looks like the default
Debian/Ubuntu package for libhdf5-openmpi (HDF5 1.6.6) ships with
shared libraries. Presumably the package managers have done their own
testing, although I don't know what patchset they've applied.
--enable-shared with --enable-parallel goes back to at least to 1.8.2, so probably they just enabled shared
BTW, on which system you will be using it?
I would be running it on 64-bit x86 Linux. However, my personal work
has limited benefit from MPI, so this is more of a general
investigation to see if it is feasible across UNIX-ish platforms. A
few people have asked me about using parallel HDF5 from Python, and
now that mpi4py is stable I thought I'd have a go.
I see. Thank you!
Elena
···
On Aug 19, 2009, at 1:07 AM, Andrew Collette wrote:
Andrew
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
I'd like to note, I am interested in this as well (I use Andrew's
h5py). Is it possible that the hdf group will support this
configuration in the future?
Darren
···
On Wed, Aug 19, 2009 at 9:14 AM, Elena Pourmal <epourmal@hdfgroup.org> wrote:
Hi Andrew,
On Aug 19, 2009, at 1:07 AM, Andrew Collette wrote:
Hi Elena,
But we do not support shared parallel library (i.e., we do not test this
option), so please use the option with caution (in other words, please
make
sure that all tests passed before using the library).
All tests pass on Linux, for 1.8.3 on 64-bit x86 and 32-bit PPC (the
machines I have immediate access to). It also looks like the default
Debian/Ubuntu package for libhdf5-openmpi (HDF5 1.6.6) ships with
shared libraries. Presumably the package managers have done their own
testing, although I don't know what patchset they've applied.
--enable-shared with --enable-parallel goes back to at least to 1.8.2, so
probably they just enabled shared
BTW, on which system you will be using it?
I would be running it on 64-bit x86 Linux. However, my personal work
has limited benefit from MPI, so this is more of a general
investigation to see if it is feasible across UNIX-ish platforms. A
few people have asked me about using parallel HDF5 from Python, and
now that mpi4py is stable I thought I'd have a go.
I see. Thank you!
I forgot to mention, after enabling parallel and shared, hdf5's tests
passed for me as well an 64-bit gentoo (gcc-4.4.1).
Darren
···
On Wed, Sep 30, 2009 at 10:22 AM, Darren Dale <dsdale24@gmail.com> wrote:
On Wed, Aug 19, 2009 at 9:14 AM, Elena Pourmal <epourmal@hdfgroup.org> wrote:
Hi Andrew,
On Aug 19, 2009, at 1:07 AM, Andrew Collette wrote:
Hi Elena,
But we do not support shared parallel library (i.e., we do not test this
option), so please use the option with caution (in other words, please
make
sure that all tests passed before using the library).
All tests pass on Linux, for 1.8.3 on 64-bit x86 and 32-bit PPC (the
machines I have immediate access to). It also looks like the default
Debian/Ubuntu package for libhdf5-openmpi (HDF5 1.6.6) ships with
shared libraries. Presumably the package managers have done their own
testing, although I don't know what patchset they've applied.
--enable-shared with --enable-parallel goes back to at least to 1.8.2, so
probably they just enabled shared
BTW, on which system you will be using it?
I would be running it on 64-bit x86 Linux. However, my personal work
has limited benefit from MPI, so this is more of a general
investigation to see if it is feasible across UNIX-ish platforms. A
few people have asked me about using parallel HDF5 from Python, and
now that mpi4py is stable I thought I'd have a go.
I see. Thank you!
I'd like to note, I am interested in this as well (I use Andrew's
h5py). Is it possible that the hdf group will support this
configuration in the future?
