Hi John,
Quincey
Thanks for the corrections. I'm working on getting a "real" policy
for accepting patches into place internally and then I'll get the code into
the repository. Along those lines, can you check if there will be any
difficulties on your side getting an intellectual property release for
assigning ownership of your changes to the HDF Group?
There is no difficulty transferring IP for the patches. Please consider them yours to use as you wish.
OK, when we've got a draft of the IP transfer process ready I'll send you the form to sign & return.
====
With regard to the VFD interface. We are using a custom driver (Originally from Jerry Clarke) implementing a DSM interface, all is well except that we must compile the driver into the HDF source code due to the MPI hooks etc. The patch you already have removes most of the need for this, but there is one place left, which is in
int H5FD_term_interface(void) {
....
#ifdef H5_HAVE_DSM
H5FD_dsm_term();
#endif
...
}
where we have added the 3 lines above.
In fact the only thing that the H5FD_dsm_term call does is set
// The driver identification number, initialized at runtime
static hid_t H5FD_DSM_g = 0;
to zero so that any future attempts to reference it will trigger the init code once again if necessary and not try to use a handle that has been destroyed.
As far as I can determine, the only time that the H5fd_term_interface function is called is when H5close is called. At this point, all the handles to file drivers are closed, and each driver has it's terminate function called, which sets this variable back to zero.
If I could remove this one function from H5FD.c, then it would be possible to supply a completely user defined file driver, without need to modify the core sources at all.
That's the goal with the VFD interface, so we want to help you move in that direction.
The trouble is the obvious thing to do is ...
Add a function to the file driver class struct, which would be called during H5FD_term_interface to set the static H5FD_DSM_g to zero - but we can't do this, because when we reach the point in H5FD_term_interface where H5FD_dsm_term() is called, the handle to the driver has already been closed, and we've invalidate the memory, so the function pointer to callback is not valid.
That's the right solution, yes. We can change the order that the interfaces are shut down at library termination so this problem doesn't happen.
One idea that occurred to me is that since each driver maintains a static handle of the type
static hid_t H5FD_DSM_g = 0; or
static hid_t H5FD_MPIO_g = 0; or
then the address of this global file driver storage handle could be passed as a member of the main driver class structure, and it could be set to zero by the H5FD code itself. This would then remove the need for the cleanup which is at present in H5FD.c.
#ifdef H5_HAVE_DIRECT
H5FD_direct_term();
#endif
H5FD_log_term();
H5FD_stdio_term();
#ifdef H5_HAVE_WINDOWS
H5FD_windows_term();
#endif
H5FD_family_term();
H5FD_core_term();
H5FD_multi_term();
#ifdef H5_HAVE_PARALLEL
H5FD_mpio_term();
H5FD_mpiposix_term();
#ifdef H5_HAVE_DSM
H5FD_dsm_term(); <- our extra one
#endif
#endif /* H5_HAVE_PARALLEL */
I don't know if there is ever a need for the library to maintain more than one handle to a particular file driver. I have a suspicion that the use of static vars like H5_DSM_g and H5_MPIO_g means that only one is ever maintained at a time. If this is the case, then separating these global/static handles out and passing in the global handle would work alright - though quite when to zero it out is not yet obvious to me, (the handles seem to be freed in a call to H5I_clear_type, but this handles handles of al types and not just file drivers, so how we know to zero out the var requires thought).
Anyway. I am looking for a way to cleanly clear up the driver interface, and open/close repeatedly the handles etc, without causing problems or requiring the driver to be linked in. If you have suggestions, please make them and I'll see if I can create a patch.
As you say, this way is prone to difficulty.
I'd lean toward adding a new VFD 'term' callback to the H5FD_class_t (in the 1.10.0 release) and adjusting the way the library shuts down, so that it gets invoked at the correct time. If you'd like to look into providing a patch for this, you can look at the H5_term_library() code in src/H5.c and decide if the order of the interface shutdowns needs to change (or maybe a new shutdown call needs to be inserted).
Thanks,
Quincey
···
On Dec 4, 2009, at 10:56 AM, Biddiscombe, John A. wrote:
on "How can I help with the HDF5 software testing and porting?", "How to submit a patch?", "How to report a bug?", "Software engineering at The HDF Group", "HDF5 coding standards", "HDF5 configure explained", etc,. will help to address the issues discussed on this forum and will help The HDF Group to be more efficient in its work with the community.
I think we'll make it before the end of 2010, but it _could_ slip to '11... We _have_ to get some more backward/forward features in place before the next major release and I'm not certain exactly how long those will take to implement.