Dataset resizing

Hi all,

I think this may be of interest to many, so here is my question: If I want
to resize a contiguous dataset, I have to check the following:

1) If the new size is smaller than the old size, I can safely use
H5Dset_extent. Hopefully this is an atomic operation.
2) If the new size is larger than the old size, I cannot use H5Dset_extent.
Instead, what I do is to create the new dataset("new") into the existing
group where the dataset("desired name") resides. What I would like to do
now, is to swap the links to the two datasets in an atomic manner, so that I
at no single step there is no proper dataset in the group. The way I do this
now, is to create a link to the old dataset ("old"), delete the link to the
dataset and then move the link of the new dataset ("new"->"desired name");
then I delete the old dataset. However, somewhere in the middle, I have no
link "desired name"; only "old" and "new".

Any ideas how this could be done more safely and elegandly?

thanks!

-- dimitris

Hi Dimitris,

···

On Sep 1, 2008, at 9:33 AM, Dimitris Servis wrote:

Hi all,

I think this may be of interest to many, so here is my question: If I want to resize a contiguous dataset, I have to check the following:

1) If the new size is smaller than the old size, I can safely use H5Dset_extent. Hopefully this is an atomic operation.
2) If the new size is larger than the old size, I cannot use H5Dset_extent. Instead, what I do is to create the new dataset("new") into the existing group where the dataset("desired name") resides. What I would like to do now, is to swap the links to the two datasets in an atomic manner, so that I at no single step there is no proper dataset in the group. The way I do this now, is to create a link to the old dataset ("old"), delete the link to the dataset and then move the link of the new dataset ("new"->"desired name"); then I delete the old dataset. However, somewhere in the middle, I have no link "desired name"; only "old" and "new".

Any ideas how this could be done more safely and elegandly?

  Hmm, H5Lmove() comes closest, but doesn't currently do what you want (it will fail if the destination link name exists, currently). That behavior could be changed with a new link creation property for that call though...

  Quincey

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.

Hi Quincey,

thanks for the reply.

Hi Dimitris,

Hi all,

I think this may be of interest to many, so here is my question: If I want
to resize a contiguous dataset, I have to check the following:

1) If the new size is smaller than the old size, I can safely use
H5Dset_extent. Hopefully this is an atomic operation.
2) If the new size is larger than the old size, I cannot use
H5Dset_extent. Instead, what I do is to create the new dataset("new") into
the existing group where the dataset("desired name") resides. What I would
like to do now, is to swap the links to the two datasets in an atomic
manner, so that I at no single step there is no proper dataset in the group.
The way I do this now, is to create a link to the old dataset ("old"),
delete the link to the dataset and then move the link of the new dataset
("new"->"desired name"); then I delete the old dataset. However, somewhere
in the middle, I have no link "desired name"; only "old" and "new".

Any ideas how this could be done more safely and elegandly?

       Hmm, H5Lmove() comes closest, but doesn't currently do what you want
(it will fail if the destination link name exists, currently). That
behavior could be changed with a new link creation property for that call
though...

I am not sure I understand your last comment. I assume you mean it could be
implemented, AFAIK there is no such property currently. Also I am not sure
how this could be done safely and under (weak) ACID assumptions, so that
H5Lmove() would actually replace the existing link by switching it from one
group or dataset to the other.

Thanks!

-- dimitris

···

2008/9/4 Quincey Koziol <koziol@hdfgroup.org>

On Sep 1, 2008, at 9:33 AM, Dimitris Servis wrote:

Hi Dimitris,

···

On Sep 4, 2008, at 4:07 AM, Dimitris Servis wrote:

Hi Quincey,

thanks for the reply.

2008/9/4 Quincey Koziol <koziol@hdfgroup.org>

Hi Dimitris,

On Sep 1, 2008, at 9:33 AM, Dimitris Servis wrote:

Hi all,

I think this may be of interest to many, so here is my question: If I want to resize a contiguous dataset, I have to check the following:

1) If the new size is smaller than the old size, I can safely use H5Dset_extent. Hopefully this is an atomic operation.
2) If the new size is larger than the old size, I cannot use H5Dset_extent. Instead, what I do is to create the new dataset("new") into the existing group where the dataset("desired name") resides. What I would like to do now, is to swap the links to the two datasets in an atomic manner, so that I at no single step there is no proper dataset in the group. The way I do this now, is to create a link to the old dataset ("old"), delete the link to the dataset and then move the link of the new dataset ("new"->"desired name"); then I delete the old dataset. However, somewhere in the middle, I have no link "desired name"; only "old" and "new".

Any ideas how this could be done more safely and elegandly?

       Hmm, H5Lmove() comes closest, but doesn't currently do what you want (it will fail if the destination link name exists, currently). That behavior could be changed with a new link creation property for that call though...

I am not sure I understand your last comment. I assume you mean it could be implemented, AFAIK there is no such property currently. Also I am not sure how this could be done safely and under (weak) ACID assumptions, so that H5Lmove() would actually replace the existing link by switching it from one group or dataset to the other.

  Yes, that's what I meant - H5Lmove() doesn't do this currently, but it makes sense to add that capability through a new link creation property. Regarding the "ACID" property - I think that as long as it occurred in a single HDF5 API call, that would give you the safety you need.

  Quincey

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.