2.0: Missing API compatibility macros

These five API functions were removed in 2.0.0, without providing compatibility macros. In my opinion, compatibility macros are needed to ease upgrades, etc. Can you please add the missing macros?

H5FDperform_init
H5VLpeek_connector_id_by_name
H5VLpeek_connector_id_by_value
H5VLfinish_lib_state
H5VLstart_lib_state

For example, missing H5FDperform_init caused this downstream problem. Go to comment #4.

Thanks for the question. For these types of “developer” routines we have not in general provided API compatibility routines since the way these things work can change in fundamental ways between major releases and it can be difficult to attempt to implement the new behavior in terms of the old API in general terms. And the amount of changes can be much greater than for the normal “user” API routines. These developer routines can be found in H5*develop.h and H5VLconnector*.h headers. We are however considering options for implementing some compatibility for these in the future to ease the burden on VOL developers.

For H5FDperform_init(), all the original routine did was initialize the library (which can be done with H5open()) then call the provided callback function and return the return value of that callback, so this can be easily implemented this way without calling that function and this code should work with either version of HDF5. H5VLfinish_lib_state() and H5VLstart_lib_state() have been renamed to H5VLclose_lib_context() and H5VLopen_lib_context(). And the peek functions should not be necessary with the way the VOL works in HDF5 2.0. These H5VL* functions may require #ifdefs in VOL connector code to support both versions of HDF5.

1 Like

@nfortne2 Are you suggesting that these five functions were never part of the HDF5 public API? If so, looking at the history of netcdf-C, this starts to make a little more sense to me.

They were part of the public API but were considered to be of use only to plugin or HDF5 library developers, and these functions have generally not been versioned with API compatibility macros.

Okay. Since they were previously public, I think a good strategy would be to add versioned compatibility macros, and put out a quick patch release. Then make occasional deprecation announcements, and keep the compatibility macros in the code base for an appropriate waiting period. This will benefit adoption of HDF5 2.0, which is now blocked in some places, because of this and perhaps other incompatibilities. I have noticed this strategy in other projects that needed to make breaking changes in their API’s.

The only function that I care about is H5FDperform_init because it is important for the netcdf-C core library. I do not know whether the H5VL* functions were adopted widely enough to cause a similar dependency.

Hey, netCDF developer here; this is causing problems for us in our currently release, re: the lack of a compatibility macros/unannounced, breaking changes to the public API. I’d love to establish a channel of communication (such as the HDF Group and NSF Unidata have previously enjoyed in the past), so that we aren’t caught flat footed by changes like this :). Is there any chance the HDF team might adopt @dave.allured’s suggestion to introduce a compatibility macro for this? I would +1 that suggestion, and also don’t want to invest time in fixing this in our codebase (on top of everything else we’re working on) if it’s going to be addressed on your end anyways :).

Thanks, congrats on the 2.0.0 milestone and release! :fireworks:

-Ward

1 Like

We can certainly let you know in the future when we introduce changes like this, though it’s possible things could be missed. It might be a good idea to try building netCDF against the develop branch of HDF5 occasionally so things like this can be caught earlier.

In this case, it should be possible to avoid calling H5FDperform_init() for any version of HDF5 by using something like H5OPEN callback_func() - I’ve posted a suggestion in the linked github issue. If that doesn’t work you can simply add macros that perform the initialization differently depending on the state of the H5_VERS_MAJOR, H5_VERS_MINOR, and H5_VERS_RELEASE macros. I don’t think we’ll add a compatibility macro for this function specifically, but we’re thinking about adding some sort of compatibility for these “developer” routines and will address this function as appropriate if we decide to do this. To reiterate, there would be a lot of work and extra code in the public API necessary to support multiple versions of these functions in general.