HDF5 1.8.5 - writing data elements one-by-one

Hdf-forum archives
1

I'm trying to write data to a dataset consisting of a 1-by-4 array of
integers, but I want to write the integers one-by-one, appending an
integer to the existing dataset each time through the loop. I'm using a
hyperslab to select where I am writing, and then writing the integer
(four writes total). This simple example simulates the way I will
(eventually) be writing more complex real-time data that is streaming
in.

Everything I've read so far indicates that the following code should
work. Am I missing an obvious step? Why is only the first element being
written correctly?

Thanks in advance - I appreciate any feedback and am looking to learn

  // Create file descriptor for HDF output file
  hid_t file = H5Fcreate("output.h5", H5F_ACC_TRUNC, H5P_DEFAULT,
H5P_DEFAULT);

  hid_t dataset, datatype, dataspace;

  // Initial hyperslab selection will be the first element (0,0)
  hsize_t start[] = {0,0};
  hsize_t count[] = {1,1};

  // Make the dataset dimensions 1-by-4
  hsize_t dims[] = {1, 4};

  // Create the file dataspace
  dataspace = H5Screate_simple(2, dims, NULL);
    
  // Create the dataset
  dataset = H5Dcreate(file, "/dset", H5T_STD_I32LE, dataspace,
H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);

  // Create buffer
  int *ii = new int;

  for (int i = 0; i < 4; i++) {
    // Set value of buffer (first time is 15, then 16, etc)
    *ii = i+15;
    
    // Select 1-by-1 hyperslab to write to, starting at the
first element (0,0)
    start[1] = i;
    H5Sselect_hyperslab(dataspace, H5S_SELECT_SET, start,
NULL, count, NULL);
    
    // Write data
    H5Dwrite(dataset, H5T_STD_I32LE, H5S_ALL, dataspace,
H5P_DEFAULT, ii);
  }

  // Clean up
  H5Dclose(dataset);
  H5Sclose(dataspace);
  H5Fclose(file);

  // I expect this to create the following 1-by-4 dataset of
integer elements:
  //
  // | 15 | 16 | 17 | 18 |
  //
  // Instead, it only seems to write to the first element:
  //
  // | 15 | -1219428236 | 0 | 17 |
  //
  // In fact, even if skip the first element, and initially set
start to {0, 1}, it still won't
  // let me write to any of the other elements:
  //
  // | 0 | -1219428236 | 0 | 17 |
  //
  // What is going on? FYI: I mostly followed the code written in
the HDF User Guide

···

from my mistakes.

2

I think your problem might be the H5S_ALL in your H5DWrite. The docs say
the following for specifying H5S_ALL for memspace and a valid dataspace id
for file space:

The file dataset's dataspace is used for the memory dataspace and the
selection specified with file_space_id specifies the selection within it.
The combination of the file dataset's dataspace and the selection from
file_space_id is used for memory also.

Which means that you're selecting elements at an offset of more than one
into your one element array ii.

Cheers,
Ethan

···

On Mon, Jul 19, 2010 at 4:32 PM, Schoolov, Mihail Y CIV SPAWARSYSCEN-PACIFIC, 56540 <mihail.schoolov@navy.mil> wrote:

I'm trying to write data to a dataset consisting of a 1-by-4 array of
integers, but I want to write the integers one-by-one, appending an integer
to the existing dataset each time through the loop. I'm using a hyperslab to
select where I am writing, and then writing the integer (four writes total).
This simple example simulates the way I will (eventually) be writing more
complex real-time data that is streaming in.

Everything I've read so far indicates that the following code should work.
Am I missing an obvious step? Why is only the first element being written
correctly?

Thanks in advance - I appreciate any feedback and am looking to learn from
my mistakes.

        // Create file descriptor for HDF output file
        hid_t file = H5Fcreate("output.h5", H5F_ACC_TRUNC, H5P_DEFAULT,
H5P_DEFAULT);

        hid_t dataset, datatype, dataspace;

        // Initial hyperslab selection will be the first element (0,0)
        hsize_t start[] = {0,0};
        hsize_t count[] = {1,1};

        // Make the dataset dimensions 1-by-4
        hsize_t dims[] = {1, 4};

        // Create the file dataspace
        dataspace = H5Screate_simple(2, dims, NULL);

        // Create the dataset
        dataset = H5Dcreate(file, "/dset", H5T_STD_I32LE, dataspace,
H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);

        // Create buffer
        int *ii = new int;

        for (int i = 0; i < 4; i++) {
                // Set value of buffer (first time is 15, then 16, etc)
                *ii = i+15;

                // Select 1-by-1 hyperslab to write to, starting at the
first element (0,0)
                start[1] = i;
                H5Sselect_hyperslab(dataspace, H5S_SELECT_SET, start,
NULL, count, NULL);

                // Write data
                H5Dwrite(dataset, H5T_STD_I32LE, H5S_ALL, dataspace,
H5P_DEFAULT, ii);
        }

        // Clean up
        H5Dclose(dataset);
        H5Sclose(dataspace);
        H5Fclose(file);

        // I expect this to create the following 1-by-4 dataset of integer
elements:
        //
        // | 15 | 16 | 17 | 18 |
        //
        // Instead, it only seems to write to the first element:
        //
        // | 15 | -1219428236 | 0 | 17 |
        //
        // In fact, even if skip the first element, and initially set
start to {0, 1}, it still won't
        // let me write to any of the other elements:
        //
        // | 0 | -1219428236 | 0 | 17 |
        //
        // What is going on? FYI: I mostly followed the code written in
the HDF User Guide

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

3

I think your problem might be the H5S_ALL in your H5DWrite.

  Yes, I think that's part of the issue. I also think Mihail wants to use H5Dset_extent() to make the dataset's dataspace larger on each pass through the loop.

  Quincey

···

On Jul 19, 2010, at 8:56 PM, Ethan Dreyfuss wrote:

The docs say the following for specifying H5S_ALL for memspace and a valid dataspace id for file space:

The file dataset's dataspace is used for the memory dataspace and the selection specified with file_space_id specifies the selection within it. The combination of the file dataset's dataspace and the selection from file_space_id is used for memory also.

Which means that you're selecting elements at an offset of more than one into your one element array ii.

Cheers,
Ethan

On Mon, Jul 19, 2010 at 4:32 PM, Schoolov, Mihail Y CIV SPAWARSYSCEN-PACIFIC, 56540 <mihail.schoolov@navy.mil> wrote:
I'm trying to write data to a dataset consisting of a 1-by-4 array of integers, but I want to write the integers one-by-one, appending an integer to the existing dataset each time through the loop. I'm using a hyperslab to select where I am writing, and then writing the integer (four writes total). This simple example simulates the way I will (eventually) be writing more complex real-time data that is streaming in.

Everything I've read so far indicates that the following code should work. Am I missing an obvious step? Why is only the first element being written correctly?

Thanks in advance - I appreciate any feedback and am looking to learn from my mistakes.

        // Create file descriptor for HDF output file
        hid_t file = H5Fcreate("output.h5", H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT);

        hid_t dataset, datatype, dataspace;

        // Initial hyperslab selection will be the first element (0,0)
        hsize_t start[] = {0,0};
        hsize_t count[] = {1,1};

        // Make the dataset dimensions 1-by-4
        hsize_t dims[] = {1, 4};

        // Create the file dataspace
        dataspace = H5Screate_simple(2, dims, NULL);
                
        // Create the dataset
        dataset = H5Dcreate(file, "/dset", H5T_STD_I32LE, dataspace, H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);

        // Create buffer
        int *ii = new int;

        for (int i = 0; i < 4; i++) {
                // Set value of buffer (first time is 15, then 16, etc)
                *ii = i+15;
                
                // Select 1-by-1 hyperslab to write to, starting at the first element (0,0)
                start[1] = i;
                H5Sselect_hyperslab(dataspace, H5S_SELECT_SET, start, NULL, count, NULL);
                
                // Write data
                H5Dwrite(dataset, H5T_STD_I32LE, H5S_ALL, dataspace, H5P_DEFAULT, ii);
        }

        // Clean up
        H5Dclose(dataset);
        H5Sclose(dataspace);
        H5Fclose(file);

        // I expect this to create the following 1-by-4 dataset of integer elements:
        //
        // | 15 | 16 | 17 | 18 |
        //
        // Instead, it only seems to write to the first element:
        //
        // | 15 | -1219428236 | 0 | 17 |
        //
        // In fact, even if skip the first element, and initially set start to {0, 1}, it still won't
        // let me write to any of the other elements:
        //
        // | 0 | -1219428236 | 0 | 17 |
        //
        // What is going on? FYI: I mostly followed the code written in the HDF User Guide

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org