(Apologies if this gets posted twice)
Someone reported a bug at the h5py issue tracker:
···
---
import h5py
# test setup
fid = h5py.File('test.hdf5', 'w')
g = fid.create_group('old_loc')
g2 = g.create_group('group')
d = g.create_dataset('dataset', data=0)
print "before move:"
print g2.name
print d.name
# now rename toplevel group
g.parent.id.move('old_loc', 'new_loc')
print "after move:"
# old parent remains in dataset name, group is ok
print g2.name
print d.name
# parent is accessed by name 'g' which does not exist any more
d.parent
fid.close()
---
That script produces the following output:
---
before move:
/old_loc/group
/old_loc/dataset
after move:
/new_loc/group
/old_loc/dataset
Traceback (most recent call last):
File "move_error.py", line 24, in <module>
d.parent
File "/Users/darren/Library/Python/2.7/lib/python/site-packages/h5py/_hl/base.py",
line 144, in parent
return self.file[posixpath.dirname(self.name)]
File "/Users/darren/Library/Python/2.7/lib/python/site-packages/h5py/_hl/group.py",
line 128, in __getitem__
oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
File "h5o.pyx", line 176, in h5py.h5o.open (h5py/h5o.c:2814)
KeyError: "unable to open object (Symbol table: Can't open object)"
---
g.name and d.name simply return the result of h5i.get_name.
d.parent just splits d.name at the last "/" and returns the the first
part of the split.
g.parent.id.move calls H5Gmove2. I've read the warnings about
corrupting data using H5Gmove at
http://www.hdfgroup.org/HDF5/doc1.6/Groups.html#H5GUnlinkToCorrupt ,
but the situation described there does not appear to be relevant to
the problem we are seeing. Is h5py not performing the move properly,
or could this be a bug in HDF5?
Thanks,
Darren
I think there is a misunderstanding. g2's name after the move *is*
"/new_loc/group". I probably should have been more explicit: the
example illustrates that the problem is isolated to Datasets, Groups
are not effected.
H5py does not do any explicit name tracking. It delegates that task
completely to hdf5: for both Datasets and Groups, h5py simply calls
H5Iget_name. It seems to me that open dataset ids are not getting
their information updated, but open group ids are. Now that I think
about it, aren't there other issues with copying open datasets in
<hdf5-1.8.9? Perhaps the issues extend to moving open datasets as
well.
Darren
···
On Wed, Dec 14, 2011 at 8:03 AM, Quincey Koziol <koziol@hdfgroup.org> wrote:
Hi Darren,
On Dec 12, 2011, at 11:29 AM, Darren Dale wrote:
(Apologies if this gets posted twice)
Someone reported a bug at the h5py issue tracker:
---
import h5py
# test setup
fid = h5py.File('test.hdf5', 'w')
g = fid.create_group('old_loc')
g2 = g.create_group('group')
d = g.create_dataset('dataset', data=0)
print "before move:"
print g2.name
print d.name
# now rename toplevel group
g.parent.id.move('old_loc', 'new_loc')
print "after move:"
# old parent remains in dataset name, group is ok
print g2.name
print d.name
# parent is accessed by name 'g' which does not exist any more
d.parent
fid.close()
---
That script produces the following output:
---
before move:
/old_loc/group
/old_loc/dataset
after move:
/new_loc/group
/old_loc/dataset
Traceback (most recent call last):
File "move_error.py", line 24, in <module>
d.parent
File "/Users/darren/Library/Python/2.7/lib/python/site-packages/h5py/_hl/base.py",
line 144, in parent
return self.file[posixpath.dirname(self.name)]
File "/Users/darren/Library/Python/2.7/lib/python/site-packages/h5py/_hl/group.py",
line 128, in __getitem__
oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
File "h5o.pyx", line 176, in h5py.h5o.open (h5py/h5o.c:2814)
KeyError: "unable to open object (Symbol table: Can't open object)"
---
g.name and d.name simply return the result of h5i.get_name.
d.parent just splits d.name at the last "/" and returns the the first
part of the split.
g.parent.id.move calls H5Gmove2. I've read the warnings about
corrupting data using H5Gmove at
http://www.hdfgroup.org/HDF5/doc1.6/Groups.html#H5GUnlinkToCorrupt ,
but the situation described there does not appear to be relevant to
the problem we are seeing. Is h5py not performing the move properly,
or could this be a bug in HDF5?
I think that the move is probably working correctly, but maybe h5py's name tracking isn't\. I would say that g2's name after the move should be "/new\_loc/group", not "/old\_loc/group"\.
Hi Darren,
Hi Darren,
(Apologies if this gets posted twice)
Someone reported a bug at the h5py issue tracker:
---
import h5py
# test setup
fid = h5py.File('test.hdf5', 'w')
g = fid.create_group('old_loc')
g2 = g.create_group('group')
d = g.create_dataset('dataset', data=0)
print "before move:"
print g2.name
print d.name
# now rename toplevel group
g.parent.id.move('old_loc', 'new_loc')
print "after move:"
# old parent remains in dataset name, group is ok
print g2.name
print d.name
# parent is accessed by name 'g' which does not exist any more
d.parent
fid.close()
---
That script produces the following output:
---
before move:
/old_loc/group
/old_loc/dataset
after move:
/new_loc/group
/old_loc/dataset
Traceback (most recent call last):
File "move_error.py", line 24, in <module>
d.parent
File "/Users/darren/Library/Python/2.7/lib/python/site-packages/h5py/_hl/base.py",
line 144, in parent
return self.file[posixpath.dirname(self.name)]
File "/Users/darren/Library/Python/2.7/lib/python/site-packages/h5py/_hl/group.py",
line 128, in __getitem__
oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
File "h5o.pyx", line 176, in h5py.h5o.open (h5py/h5o.c:2814)
KeyError: "unable to open object (Symbol table: Can't open object)"
---
g.name and d.name simply return the result of h5i.get_name.
d.parent just splits d.name at the last "/" and returns the the first
part of the split.
g.parent.id.move calls H5Gmove2. I've read the warnings about
corrupting data using H5Gmove at
http://www.hdfgroup.org/HDF5/doc1.6/Groups.html#H5GUnlinkToCorrupt ,
but the situation described there does not appear to be relevant to
the problem we are seeing. Is h5py not performing the move properly,
or could this be a bug in HDF5?
I think that the move is probably working correctly, but maybe h5py's name tracking isn't. I would say that g2's name after the move should be "/new_loc/group", not "/old_loc/group".
I think there is a misunderstanding. g2's name after the move *is*
"/new_loc/group". I probably should have been more explicit: the
example illustrates that the problem is isolated to Datasets, Groups
are not effected.
Whoops, sorry, yes, you are right.
H5py does not do any explicit name tracking. It delegates that task
completely to hdf5: for both Datasets and Groups, h5py simply calls
H5Iget_name. It seems to me that open dataset ids are not getting
their information updated, but open group ids are. Now that I think
about it, aren't there other issues with copying open datasets in
<hdf5-1.8.9? Perhaps the issues extend to moving open datasets as
well.
That's pretty weird (that it would only affect datasets), since the H5Iget_name code doesn't depend on the object type. Can you replicate this with a short C program?
Quincey
···
On Dec 14, 2011, at 9:46 AM, Darren Dale wrote:
On Wed, Dec 14, 2011 at 8:03 AM, Quincey Koziol <koziol@hdfgroup.org> wrote:
On Dec 12, 2011, at 11:29 AM, Darren Dale wrote:
By the way, I see that H5Gmove2 has been deprecated in favor of
H5Lmove. I see the same problem if I use H5Lmove.
Darren
···
On Thu, Dec 15, 2011 at 11:11 AM, Darren Dale <dsdale24@gmail.com> wrote:
Hi Quincey,
On Wed, Dec 14, 2011 at 3:56 PM, Quincey Koziol <qkoziol@gmail.com> wrote:
On Dec 14, 2011, at 9:46 AM, Darren Dale wrote:
I probably should have been more explicit: the
example illustrates that the problem is isolated to Datasets, Groups
are not effected.
Whoops, sorry, yes, you are right\.
H5py does not do any explicit name tracking. It delegates that task
completely to hdf5: for both Datasets and Groups, h5py simply calls
H5Iget_name. It seems to me that open dataset ids are not getting
their information updated, but open group ids are. Now that I think
about it, aren't there other issues with copying open datasets in
<hdf5-1.8.9? Perhaps the issues extend to moving open datasets as
well.
That's pretty weird \(that it would only affect datasets\), since the
H5Iget_name code doesn't depend on the object type. Can you
replicate this with a short C program?
Yes. See the attached modified version of one of the hdf5 examples.
After I move group1 from "old_loc" to "new_loc", the names of the
nodes contained in that group werereported by hdf5-1.8.8 to be:
/new_loc/group
/old_loc/dataset
