hdf5-1.8.13 API/ABI change without updating soname

I realize this is old news, but in removing H5P{set,get}_fapl_mpiposix,
1.8.13 broke source and binary compatibility without bumping the soname
and (it appears) without any warning of deprecation. This prevents many
packagers from upgrading hdf5 until dependent packages catch up and
causes unexpected link errors for projects upgrading their stack.

  http://mail.lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2014-May/007827.html

Perhaps in the future, HDF5 could use this tool to check whether soname
needs to be bumped.

  http://ispras.linuxbase.org/index.php/ABI_compliance_checker

Hi Jed,

We are well aware of the ABI_compliance_checker tool. It is a part of our release procedure.

As you know, the H5P{set,get}_fapl_mpiposix functions belonged to the HDF5 parallel library, and we do not support shared parallel HDF5 library. One has to specifically enable “shared" at configure time to build it.

For the sequential HDF5 library compatibility was not violated. Said this, I would like to bring to your attention the following dilemma we face.

The HDF5 library can be built using many configurations; for example, one can disable deprecated symbols like H5Dopen1, and build another version of the sequential shared HDF5 library. The library in that configuration is not compatible with the HDF5 library built from the same distribution. I.e., if we follow the rules, we will need to maintain sonames for every possible build configuration of HDF5. My examples above show three non-compatible versions:

“default” - build with ./configure
“light” - built with ./configure —disable-depracted-symbols
“parallel” - built with ./configure —enable-parallel —enable-shared

Our current approach is to follow the soname rules only for the “default” configuration. We will appreciate any suggestions on how to solve the problem.

Thank you!

Elena

···

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Elena Pourmal The HDF Group http://hdfgroup.org
1800 So. Oak St., Suite 203, Champaign IL 61820
217.531.6112
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

On Sep 9, 2014, at 11:45 AM, Jed Brown <jed@jedbrown.org> wrote:

I realize this is old news, but in removing H5P{set,get}_fapl_mpiposix,
1.8.13 broke source and binary compatibility without bumping the soname
and (it appears) without any warning of deprecation. This prevents many
packagers from upgrading hdf5 until dependent packages catch up and
causes unexpected link errors for projects upgrading their stack.

http://mail.lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2014-May/007827.html

Perhaps in the future, HDF5 could use this tool to check whether soname
needs to be bumped.

http://ispras.linuxbase.org/index.php/ABI_compliance_checker

Aron Ahmadia <aron@ahmadia.net> writes:

My instinct is that you should provide a consistent set of
externally-facing symbols for the entire HDF5 library (including shared and
parallel builds), and in situations where the function is not available,
calls to that function should raise an HDF5 not implemented error.

I agree with this, though most "parallel" functions become either very
simple or no-ops in the sequential case. I'm in favor of defining the
symbols and implementing the serial case when it makes sense.

Jed, how does PETSc handle this?

MPI types leak into many PETSc public interfaces, so PETSc uses stubs in
the serial case. If MPI could get their act together to define an ABI,
we could distribute binaries with weak symbols that would work in
parallel if a real MPI became available.

The broader issue of different configurations is something that PETSc
suffers from in an even more critical way with scalar types. It is a
major problem for packaging and binary distribution, but none of the
solutions are great.

I don't know how maintainers deal with "light" versions of their
libraries. If the symbols are deprecated but not removed from this version
of HDF5, then the soname version should probably be bumped to reflect
this.

Agreed, and this would be a developer option "test that your stack has
been upgraded" before the symbols are permanently removed.

That said, I think it's more important that the default/parallel
public APIs to libhdf5.so are identical than that the "light" version
be properly versioned.

Yes.

As you know, the H5P{set,get}_fapl_mpiposix functions belonged to the HDF5
parallel library, and we do not support shared parallel HDF5 library. One
has to specifically enable “shared" at configure time to build it.

Just to be clear, parallel is supported, and shared is supported, but not
parallel and shared? Is the rationale for this explained somewhere?

For the sequential HDF5 library compatibility was not violated. Said this,

I would like to bring to your attention the following dilemma we face.

The HDF5 library can be built using many configurations; for example, one
can disable deprecated symbols like H5Dopen1, and build another version of
the sequential shared HDF5 library. The library in that configuration is
not compatible with the HDF5 library built from the same distribution.
I.e., if we follow the rules, we will need to maintain sonames for every
possible build configuration of HDF5. My examples above show three
non-compatible versions:

“default” - build with ./configure
“light” - built with ./configure —disable-depracted-symbols
“parallel” - built with ./configure —enable-parallel —enable-shared

Our current approach is to follow the soname rules only for the “default”
configuration. We will appreciate any suggestions on how to solve the
problem.

My instinct is that you should provide a consistent set of
externally-facing symbols for the entire HDF5 library (including shared and
parallel builds), and in situations where the function is not available,
calls to that function should raise an HDF5 not implemented error. Jed,
how does PETSc handle this?

I don't know how maintainers deal with "light" versions of their
libraries. If the symbols are deprecated but not removed from this version
of HDF5, then the soname version should probably be bumped to reflect
this. That said, I think it's more important that the default/parallel
public APIs to libhdf5.so are identical than that the "light" version be
properly versioned.

A