External links and forward compatibility

Hdf-forum archives
1

Hi,

I've just implemented support for links in PyTables and HDF5 1.8.x series (via
H5L API) and I'm trying now to see if the generated files can be accessed via
PyTables + HDF5 1.6.x. The idea is not to support links with the latter
scenario, but just recognize links and assign them to a special
`UnImplementedLink` class.

This approach works well for soft links, i.e. I can open files with soft links
and map them to the `UnImplementedLink` class. However, when I try to do the
same with external links, the HDF5 (using 1.6.10 here) throws the next error:

{{{
HDF5-DIAG: Error detected in HDF5 library version: 1.6.10 thread 0. Back
trace follows.
  #000: H5G.c line 258 in H5Gopen(): not a group
    major(01): Function arguments
    minor(03): Inappropriate type
  #001: H5G.c line 2095 in H5G_get_type(): unable to determine object type
    major(10): Symbol table layer
    minor(29): Unable to initialize object
}}}

when trying to open the root group (which has a child that is a external
link).

I've tried to use the native HDF5 tools in order to see if they work better,
but h5ls from HDF5 1.6.10 does not print anything:

{{{
$ h5ls elink.h5
$
}}}

while h5ls from HDF5 1.8.4 does:

{{{
$ h5ls elink.h5
pep Group
pep2 External Link {elink2.h5//pep}
$
}}}

Then, I've tried with h5dump (1.6.10) and got:

{{{
$ h5dump elink.h5
h5dump error: internal error (file h5dump.c:line 3798)
}}}

while h5dump (1.8.4) seems to work well:

{{{
$ h5dump elink.h5
[clip]
   EXTERNAL_LINK "pep2" {
      TARGETFILE "elink2.h5"
      TARGETPATH "/pep"
[clip]
$
}}}

These results drives me to think that HDF5 1.6.10 is really unable to open
1.8.x files with external links (I'm attaching an example). Can you confirm
this point please? In case this is true, do you plan to fix that in the 1.6.x
branch or that incompatibility will be there forever?

Thanks,

elink.h5 (2.29 KB)

elink2.h5 (2.18 KB)

···

--
Francesc Alted

2

Hi Francesc,

···

On Dec 9, 2009, at 11:34 AM, Francesc Alted wrote:

Hi,

I've just implemented support for links in PyTables and HDF5 1.8.x series (via
H5L API) and I'm trying now to see if the generated files can be accessed via
PyTables + HDF5 1.6.x. The idea is not to support links with the latter
scenario, but just recognize links and assign them to a special
`UnImplementedLink` class.

This approach works well for soft links, i.e. I can open files with soft links
and map them to the `UnImplementedLink` class. However, when I try to do the
same with external links, the HDF5 (using 1.6.10 here) throws the next error:

{{{
HDF5-DIAG: Error detected in HDF5 library version: 1.6.10 thread 0. Back
trace follows.
#000: H5G.c line 258 in H5Gopen(): not a group
   major(01): Function arguments
   minor(03): Inappropriate type
#001: H5G.c line 2095 in H5G_get_type(): unable to determine object type
   major(10): Symbol table layer
   minor(29): Unable to initialize object
}}}

when trying to open the root group (which has a child that is a external
link).

I've tried to use the native HDF5 tools in order to see if they work better,
but h5ls from HDF5 1.6.10 does not print anything:

{{{
$ h5ls elink.h5
$
}}}

while h5ls from HDF5 1.8.4 does:

{{{
$ h5ls elink.h5
pep Group
pep2 External Link {elink2.h5//pep}
$
}}}

Then, I've tried with h5dump (1.6.10) and got:

{{{
$ h5dump elink.h5
h5dump error: internal error (file h5dump.c:line 3798)
}}}

while h5dump (1.8.4) seems to work well:

{{{
$ h5dump elink.h5
[clip]
  EXTERNAL_LINK "pep2" {
     TARGETFILE "elink2.h5"
     TARGETPATH "/pep"
[clip]
$
}}}

These results drives me to think that HDF5 1.6.10 is really unable to open
1.8.x files with external links (I'm attaching an example). Can you confirm
this point please? In case this is true, do you plan to fix that in the 1.6.x
branch or that incompatibility will be there forever?

  The problem you are seeing is due to the fact that the format of the group itself changed in order to store external links in the group. So, the 1.6.x branch is unable to open groups with external links in them (and we aren't planning on back-porting support for it doing so). It can open _files_ with external links in a group, but you've put the external links in the root group and therefore the 1.6.x library can't "get past" the root group to see anything else in the file.

  Quincey

3

A Wednesday 09 December 2009 20:11:44 Quincey Koziol escrigué:

> These results drives me to think that HDF5 1.6.10 is really unable to
> open 1.8.x files with external links (I'm attaching an example). Can you
> confirm this point please? In case this is true, do you plan to fix that
> in the 1.6.x branch or that incompatibility will be there forever?

  The problem you are seeing is due to the fact that the format of the group
itself changed in order to store external links in the group. So, the
1.6.x branch is unable to open groups with external links in them (and we
aren't planning on back-porting support for it doing so). It can open
_files_ with external links in a group, but you've put the external links
in the root group and therefore the 1.6.x library can't "get past" the
root group to see anything else in the file.

Okay. So I'll warn people that is creating external links that neither the
external link *nor* the parent group (and hence, all its children) will be
readable when using HDF5 1.6.x. HDF5 1.6 reports a group containing an
external link as H5G_UNKNOWN, and that can be catch without HDF5 issuing
errors. It is a pity that if the group is the root group HDF5 1.6 will raise
the error, rendering the complete file unavailable for the user, but well,
that's life.

Thanks,

···

--
Francesc Alted