Hi all,
I've just found a weird construct in one of our CGNS/HDF5 files (part of h5dump below).
We have an external link to another file and inside the external link itself there is a 'local' group.
The group inside actually masks a group of same name in the target file (see bc_1_8 below).
We did obtain such a file because of a bug in our application, it uses to replace a group with an external
link having the same name but without deleting the original group.
I wonder if:
1. yes this is the expected behavior, the link is always read first so that you can use this feature to partly mask the target sub-tree
2. no, this file shows a bad construct, should be corrected and HDF5 should forbid such
3. side-effect, it is not a feature but you can build such a file, as long as we do not change implementation the side effect is... a feature (oups!)
-MP-
EXTERNAL_LINK " link" {
TARGETFILE "mesh.cgns"
TARGETPATH "/Base#1/Zone#1/ZoneGridConnectivity"
GROUP "/Base#1/Zone#1/ZoneGridConnectivity" {
ATTRIBUTE "flags" {
DATATYPE H5T_STD_I32LE
DATASPACE SIMPLE { ( 1 ) / ( 1 ) }
DATA {
(0): 0
}
}
ATTRIBUTE "label" {
DATATYPE H5T_STRING {
STRSIZE 33;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "ZoneGridConnectivity_t"
}
}
ATTRIBUTE "name" {
DATATYPE H5T_STRING {
STRSIZE 33;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "ZoneGridConnectivity"
}
}
ATTRIBUTE "type" {
DATATYPE H5T_STRING {
STRSIZE 3;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "MT"
}
}
GROUP "bc_1_8" {
ATTRIBUTE "flags" {
DATATYPE H5T_STD_I32LE
DATASPACE SIMPLE { ( 1 ) / ( 1 ) }
DATA {
(0): 0
}
}
}
···
--
-- ------------------------------------------------------------
-- Marc POINOT [ONERA/DMFN] Tel:+33.1.46.73.42.84
-- Avertissement/disclaimer http://www.onera.fr/en/emails-terms
-- ------------------------------------------------------------
Hi Marc,
Hi all,
I've just found a weird construct in one of our CGNS/HDF5 files (part of h5dump below).
We have an external link to another file and inside the external link itself there is a 'local' group.
The group inside actually masks a group of same name in the target file (see bc_1_8 below).
We did obtain such a file because of a bug in our application, it uses to replace a group with an external
link having the same name but without deleting the original group.
I wonder if:
1. yes this is the expected behavior, the link is always read first so that you can use this feature to partly mask the target sub-tree
2. no, this file shows a bad construct, should be corrected and HDF5 should forbid such
3. side-effect, it is not a feature but you can build such a file, as long as we do not change implementation the side effect is... a feature (oups!)
Hmm, I think 1 & 3 are basically the same, and that’s what the HDF5 library is supposed to do. Basically, the library will attempt to traverse the external link’s path, and if it encounters soft or external links in the destination file’s group hierarchy, it’ll just keep resolving those as it encounters them. Since the external links are stored as “plain” strings in the source HDF5 file, there’s no reference to the “actual” object in the destination file, just a path in that file to traverse to reach the object.
Make sense? 
Quincey
···
On Sep 16, 2015, at 2:54 AM, marc.poinot@onera.fr wrote:
-MP-
EXTERNAL_LINK " link" {
TARGETFILE "mesh.cgns"
TARGETPATH "/Base#1/Zone#1/ZoneGridConnectivity"
GROUP "/Base#1/Zone#1/ZoneGridConnectivity" {
ATTRIBUTE "flags" {
DATATYPE H5T_STD_I32LE
DATASPACE SIMPLE { ( 1 ) / ( 1 ) }
DATA {
(0): 0
}
}
ATTRIBUTE "label" {
DATATYPE H5T_STRING {
STRSIZE 33;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "ZoneGridConnectivity_t"
}
}
ATTRIBUTE "name" {
DATATYPE H5T_STRING {
STRSIZE 33;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "ZoneGridConnectivity"
}
}
ATTRIBUTE "type" {
DATATYPE H5T_STRING {
STRSIZE 3;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
DATA {
(0): "MT"
}
}
GROUP "bc_1_8" {
ATTRIBUTE "flags" {
DATATYPE H5T_STD_I32LE
DATASPACE SIMPLE { ( 1 ) / ( 1 ) }
DATA {
(0): 0
}
}
}
--
-- ------------------------------------------------------------
-- Marc POINOT [ONERA/DMFN] Tel:+33.1.46.73.42.84
-- Avertissement/disclaimer http://www.onera.fr/en/emails-terms
-- ------------------------------------------------------------
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
Thanks Quincey,
so in our example we have a name collision, the winner is the local group.
if the link traversal is always before the local group read,
we can use this as a feature to allow masking parts of the external sub-tree we retrieve.
would this be stable, is the answer more 1 than 3?
if you say it is more 1 than we can add interesting new features to our CGNS/HDF5 lib, if it's more 3 we probably won't change.
this can also be used as a 'default' mecanism, the link refers to a set of default sub-trees as long as an actual value is not written in the actual file.
-MP-
···
On 09/16/15 18:05, Quincey Koziol wrote:
On Sep 16, 2015, at 2:54 AM, marc.poinot@onera.fr wrote:
Hi all,
I've just found a weird construct in one of our CGNS/HDF5 files (part of h5dump below).
We have an external link to another file and inside the external link itself there is a 'local' group.
The group inside actually masks a group of same name in the target file (see bc_1_8 below).
We did obtain such a file because of a bug in our application, it uses to replace a group with an external
link having the same name but without deleting the original group.
I wonder if:
1. yes this is the expected behavior, the link is always read first so that you can use this feature to partly mask the target sub-tree
2. no, this file shows a bad construct, should be corrected and HDF5 should forbid such
3. side-effect, it is not a feature but you can build such a file, as long as we do not change implementation the side effect is... a feature (oups!)
Hmm, I think 1 & 3 are basically the same, and that’s what the HDF5 library is supposed to do. Basically, the library will attempt to traverse the external link’s path, and if it encounters soft or external links in the destination file’s group hierarchy, it’ll just keep resolving those as it encounters them. Since the external links are stored as “plain” strings in the source HDF5 file, there’s no reference to the “actual” object in the destination file, just a path in that file to traverse to reach the object.
Make sense?
--
-- ------------------------------------------------------------
-- Marc POINOT [ONERA/DMFN] Tel:+33.1.46.73.42.84
-- Avertissement/disclaimer http://www.onera.fr/en/emails-terms
-- ------------------------------------------------------------
Hi Marc,
Thanks Quincey,
so in our example we have a name collision, the winner is the local group.
if the link traversal is always before the local group read,
we can use this as a feature to allow masking parts of the external sub-tree we retrieve.
would this be stable, is the answer more 1 than 3?
Yes, this is stable (option 1).
if you say it is more 1 than we can add interesting new features to our CGNS/HDF5 lib, if it's more 3 we probably won't change.
You should be fine.
this can also be used as a 'default' mechanism, the link refers to a set of default sub-trees as long as an actual value is not written in the actual file.
Sounds like an interesting and possibly useful way to do things.
Quincey
···
On Sep 17, 2015, at 2:43 AM, Marc Poinot <marc.poinot@onera.fr> wrote:
-MP-
On 09/16/15 18:05, Quincey Koziol wrote:
On Sep 16, 2015, at 2:54 AM, marc.poinot@onera.fr wrote:
Hi all,
I've just found a weird construct in one of our CGNS/HDF5 files (part of h5dump below).
We have an external link to another file and inside the external link itself there is a 'local' group.
The group inside actually masks a group of same name in the target file (see bc_1_8 below).
We did obtain such a file because of a bug in our application, it uses to replace a group with an external
link having the same name but without deleting the original group.
I wonder if:
1. yes this is the expected behavior, the link is always read first so that you can use this feature to partly mask the target sub-tree
2. no, this file shows a bad construct, should be corrected and HDF5 should forbid such
3. side-effect, it is not a feature but you can build such a file, as long as we do not change implementation the side effect is... a feature (oups!)
Hmm, I think 1 & 3 are basically the same, and that’s what the HDF5 library is supposed to do. Basically, the library will attempt to traverse the external link’s path, and if it encounters soft or external links in the destination file’s group hierarchy, it’ll just keep resolving those as it encounters them. Since the external links are stored as “plain” strings in the source HDF5 file, there’s no reference to the “actual” object in the destination file, just a path in that file to traverse to reach the object.
Make sense?
--
-- ------------------------------------------------------------
-- Marc POINOT [ONERA/DMFN] Tel:+33.1.46.73.42.84
-- Avertissement/disclaimer http://www.onera.fr/en/emails-terms
-- ------------------------------------------------------------
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5
