Modifying dataspace dimensions (extent) after creating?

Hdf-forum archives
1

Is modifying dataspace dimensions (extent) after creating them
possible? Suppose I originally created a 2D dataset with dimensions [2
x 307200]. Then later on I want to change the dimensions to [4 x
153600], same number of elements, different dimension size. Or even
possibly change the rank from 2 to 3, but keep then number of elements
same, i.e. dimensions of [1200 x 2 x 256].

I tried using 'set_extent_simple' since the user guide said it can be
used to set, or reset, the size of an existing dataspace.

I attached example Matlab code that attempts to open, read, and re-
write the dataspace size. But it isn't persistent, as in once I close
and re-open, my changes aren't saved.

Thanks!

···

------------------------------------------------------------------------------------------

Code:

% Open the dataset/dataspace
fid = H5F.open('test1.h5','H5F_ACC_RDWR', 'H5P_DEFAULT');
gid = H5G.open(fid,'test1');
did = H5D.open(gid, 'Range');
sid = H5D.get_space(did);

% Read current dimensions
[ndims dims maxdims] = H5S.get_simple_extent_dims(sid)

% Convert dimensions from 2D to 3D
H5S.set_extent_simple(sid, 3, [1200 2 256], [])

% Read back
[ndims dims maxdims] = H5S.get_simple_extent_dims(sid)

% Close
H5S.close(sid);
H5D.close(did);
H5G.close(gid);
H5F.close(fid);

% Re-open and re-read to check
fid = H5F.open('test1.h5','H5F_ACC_RDWR', 'H5P_DEFAULT');
gid = H5G.open(fid,'test1');
did = H5D.open(gid, 'Range');
sid = H5D.get_space(did);

[ndims dims maxdims] = H5S.get_simple_extent_dims(sid)

H5S.close(sid);
H5D.close(did);
H5G.close(gid);
H5F.close(fid);

------------------------------------------------------------------------
% OUTPUT

ndims = 2
dims = 2 307200
maxdims = 2 307200

% After change
ndims = 3
dims = 1200 2 256
maxdims = 1200 2 256

% After closed and re-opened
ndims = 2
dims = 2 307200
maxdims = 2 307200

2

I verified that Tom's code doesn't error out in MATLAB versions 2009a and
earlier, and I can reproduce the same situation in C code as well. But I'd
always been under the impression that H5Dset_extent was the function that one
should use for resizing a dataset?

There is a list of restrictions for H5Dset_extent, one of which is that fixed
size dimensions can only be decreased.

···

On Saturday 25 July 2009 10:48:05 Tom wrote:

Is modifying dataspace dimensions (extent) after creating them
possible? Suppose I originally created a 2D dataset with dimensions [2
x 307200]. Then later on I want to change the dimensions to [4 x
153600], same number of elements, different dimension size. Or even
possibly change the rank from 2 to 3, but keep then number of elements
same, i.e. dimensions of [1200 x 2 x 256].

I tried using 'set_extent_simple' since the user guide said it can be
used to set, or reset, the size of an existing dataspace.

I attached example Matlab code that attempts to open, read, and re-
write the dataspace size. But it isn't persistent, as in once I close
and re-open, my changes aren't saved.

Thanks!

---------------------------------------------------------------------------
---------------

Code:

% Open the dataset/dataspace
fid = H5F.open('test1.h5','H5F_ACC_RDWR', 'H5P_DEFAULT');
gid = H5G.open(fid,'test1');
did = H5D.open(gid, 'Range');
sid = H5D.get_space(did);

% Read current dimensions
[ndims dims maxdims] = H5S.get_simple_extent_dims(sid)

% Convert dimensions from 2D to 3D
H5S.set_extent_simple(sid, 3, [1200 2 256], [])

% Read back
[ndims dims maxdims] = H5S.get_simple_extent_dims(sid)

% Close
H5S.close(sid);
H5D.close(did);
H5G.close(gid);
H5F.close(fid);

% Re-open and re-read to check
fid = H5F.open('test1.h5','H5F_ACC_RDWR', 'H5P_DEFAULT');
gid = H5G.open(fid,'test1');
did = H5D.open(gid, 'Range');
sid = H5D.get_space(did);

[ndims dims maxdims] = H5S.get_simple_extent_dims(sid)

H5S.close(sid);
H5D.close(did);
H5G.close(gid);
H5F.close(fid);

------------------------------------------------------------------------
% OUTPUT

ndims = 2
dims = 2 307200
maxdims = 2 307200

% After change
ndims = 3
dims = 1200 2 256
maxdims = 1200 2 256

% After closed and re-opened
ndims = 2
dims = 2 307200
maxdims = 2 307200

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

3

Thank you for the C and Matlab verification.

For H5Dset_extent, does that mean you can only extend/reduce the size of the elements held in a each dimension, but cannot actually change the dimensions themselves? I'm interested in keeping the size (# of elements) in a dataset the same, and only just reshaping them. ie a [2x6] to a [3x4] or a [2x2x3].

···

On Jul 27, 2009 8:46am, John Evans <john.evans@mathworks.com> wrote:

I verified that Tom's code doesn't error out in MATLAB versions 2009a and

earlier, and I can reproduce the same situation in C code as well. But I'd

always been under the impression that H5Dset_extent was the function that one

should use for resizing a dataset?

There is a list of restrictions for H5Dset_extent, one of which is that fixed

size dimensions can only be decreased.

On Saturday 25 July 2009 10:48:05 Tom wrote:

> Is modifying dataspace dimensions (extent) after creating them

> possible? Suppose I originally created a 2D dataset with dimensions [2

> x 307200]. Then later on I want to change the dimensions to [4 x

> 153600], same number of elements, different dimension size. Or even

> possibly change the rank from 2 to 3, but keep then number of elements

> same, ie dimensions of [1200 x 2 x 256].

>

> I tried using 'set_extent_simple' since the user guide said it can be

> used to set, or reset, the size of an existing dataspace.

>

>

> I attached example Matlab code that attempts to open, read, and re-

> write the dataspace size. But it isn't persistent, as in once I close

> and re-open, my changes aren't saved.

>

> Thanks!

>

> ---------------------------------------------------------------------------

>---------------

>

>

> Code:

>

> % Open the dataset/dataspace

> fid = H5F.open('test1.h5','H5F_ACC_RDWR', 'H5P_DEFAULT');

> gid = H5G.open(fid,'test1');

> did = H5D.open(gid, 'Range');

> sid = H5D.get_space(did);

>

> % Read current dimensions

> [ndims dims maxdims] = H5S.get_simple_extent_dims(sid)

>

> % Convert dimensions from 2D to 3D

> H5S.set_extent_simple(sid, 3, [1200 2 256], [])

>

> % Read back

> [ndims dims maxdims] = H5S.get_simple_extent_dims(sid)

>

> % Close

> H5S.close(sid);

> H5D.close(did);

> H5G.close(gid);

> H5F.close(fid);

>

> % Re-open and re-read to check

> fid = H5F.open('test1.h5','H5F_ACC_RDWR', 'H5P_DEFAULT');

> gid = H5G.open(fid,'test1');

> did = H5D.open(gid, 'Range');

> sid = H5D.get_space(did);

>

> [ndims dims maxdims] = H5S.get_simple_extent_dims(sid)

>

> H5S.close(sid);

> H5D.close(did);

> H5G.close(gid);

> H5F.close(fid);

>

>

> ------------------------------------------------------------------------

> % OUTPUT

>

> ndims = 2

> dims = 2 307200

> maxdims = 2 307200

>

>

> % After change

> ndims = 3

> dims = 1200 2 256

> maxdims = 1200 2 256

>

>

> % After closed and re-opened

> ndims = 2

> dims = 2 307200

> maxdims = 2 307200

>

> _______________________________________________

> Hdf-forum is for HDF software users discussion.

> Hdf-forum@hdfgroup.org

> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

4

Thank you for the C and Matlab verification.

For H5Dset_extent, does that mean you can only extend/reduce the size of the elements held in a each dimension, but cannot actually change the dimensions themselves? I'm interested in keeping the size (# of elements) in a dataset the same, and only just reshaping them. i.e. a [2x6] to a [3x4] or a [2x2x3].

  The HDF5 library doesn't currently support reshaping a dataspace, just extending/shrinking dimensions.

    Quincey

···

On Jul 27, 2009, at 9:23 AM, tszumowski@gmail.com wrote:

On Jul 27, 2009 8:46am, John Evans <john.evans@mathworks.com> wrote:
> I verified that Tom's code doesn't error out in MATLAB versions 2009a and
>
> earlier, and I can reproduce the same situation in C code as well. But I'd
>
> always been under the impression that H5Dset_extent was the function that one
>
> should use for resizing a dataset?
>
> There is a list of restrictions for H5Dset_extent, one of which is that fixed
>
> size dimensions can only be decreased.
>
> On Saturday 25 July 2009 10:48:05 Tom wrote:
>
> > Is modifying dataspace dimensions (extent) after creating them
>
> > possible? Suppose I originally created a 2D dataset with dimensions [2
>
> > x 307200]. Then later on I want to change the dimensions to [4 x
>
> > 153600], same number of elements, different dimension size. Or even
>
> > possibly change the rank from 2 to 3, but keep then number of elements
>
> > same, i.e. dimensions of [1200 x 2 x 256].
>
> >
>
> > I tried using 'set_extent_simple' since the user guide said it can be
>
> > used to set, or reset, the size of an existing dataspace.
>
> >
>
> >
>
> > I attached example Matlab code that attempts to open, read, and re-
>
> > write the dataspace size. But it isn't persistent, as in once I close
>
> > and re-open, my changes aren't saved.
>
> >
>
> > Thanks!
>
> >
>
> > ---------------------------------------------------------------------------
>
> >---------------
>
> >
>
> >
>
> > Code:
>
> >
>
> > % Open the dataset/dataspace
>
> > fid = H5F.open('test1.h5','H5F_ACC_RDWR', 'H5P_DEFAULT');
>
> > gid = H5G.open(fid,'test1');
>
> > did = H5D.open(gid, 'Range');
>
> > sid = H5D.get_space(did);
>
> >
>
> > % Read current dimensions
>
> > [ndims dims maxdims] = H5S.get_simple_extent_dims(sid)
>
> >
>
> > % Convert dimensions from 2D to 3D
>
> > H5S.set_extent_simple(sid, 3, [1200 2 256], [])
>
> >
>
> > % Read back
>
> > [ndims dims maxdims] = H5S.get_simple_extent_dims(sid)
>
> >
>
> > % Close
>
> > H5S.close(sid);
>
> > H5D.close(did);
>
> > H5G.close(gid);
>
> > H5F.close(fid);
>
> >
>
> > % Re-open and re-read to check
>
> > fid = H5F.open('test1.h5','H5F_ACC_RDWR', 'H5P_DEFAULT');
>
> > gid = H5G.open(fid,'test1');
>
> > did = H5D.open(gid, 'Range');
>
> > sid = H5D.get_space(did);
>
> >
>
> > [ndims dims maxdims] = H5S.get_simple_extent_dims(sid)
>
> >
>
> > H5S.close(sid);
>
> > H5D.close(did);
>
> > H5G.close(gid);
>
> > H5F.close(fid);
>
> >
>
> >
>
> > ------------------------------------------------------------------------
>
> > % OUTPUT
>
> >
>
> > ndims = 2
>
> > dims = 2 307200
>
> > maxdims = 2 307200
>
> >
>
> >
>
> > % After change
>
> > ndims = 3
>
> > dims = 1200 2 256
>
> > maxdims = 1200 2 256
>
> >
>
> >
>
> > % After closed and re-opened
>
> > ndims = 2
>
> > dims = 2 307200
>
> > maxdims = 2 307200
>
> >
>
> > _______________________________________________
>
> > Hdf-forum is for HDF software users discussion.
>
> > Hdf-forum@hdfgroup.org
>
> > http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>
>_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org