Read to a vector, read a single element

Hdf-forum archives
1

Hello programmers :-),

I'm still new to this HDF file format, and what I could do is run the C++
examples given in the tutorial to read a pressures data file given to me
by someone.

My first question is, is it possible to read the values of my H5 file to a
vector? all I've seen is array works, which is not convenient when working
generally with different sizes of data files (the files I'm using are 3D
matrices).

So what I would like to do, is having a program in this scheme:

1-Read the file and determine the data type and size
2-create a 3D vector for reading the data (like vector< vector< vector<
double > > > data;)
3-copy the data from the file to this vector.

is this possible with the current library?

My second question, is there a function in the library to read a single
element? I just found a function for reading a whole set into an array in
the tutorial:

status = H5Dread(dataset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT,
dset_data);

If there's a tutorial that is more detailed than this, or let's say a
manual, I'd be so happy to know about its existence.

Thank you, any efforts are highly appreciated.

Best regards,
Samer

2

It might be possible to read data directly into a std::vector<std::vector<T> >
using the variable length data type API of HDF5 by defining appropriate
memory allocation/release callback functions, but it would be very
inefficient.

The question is do you really have a 3D structure that can have a
variable number of elements in each dimensions, such as provided
by std::vector<std::vector<std::vector<> > > >, or can you simplify
the problem by knowing that each 2D slice in this 3D structure has
the same number of elements, and each 2D slice has the same number
of columns?

Usually for the CFD data that I had been working with data are of
equal size in each dimension, so they actually are 3D arrays rather
than nested vectors of different lengths.

  Werner

···

On Wed, 30 Jun 2010 05:11:18 -0500, <samer@icp.uni-stuttgart.de> wrote:

Hello programmers :-),

I'm still new to this HDF file format, and what I could do is run the C++
examples given in the tutorial to read a pressures data file given to me
by someone.

My first question is, is it possible to read the values of my H5 file to a
vector? all I've seen is array works, which is not convenient when working
generally with different sizes of data files (the files I'm using are 3D
matrices).

So what I would like to do, is having a program in this scheme:

1-Read the file and determine the data type and size
2-create a 3D vector for reading the data (like vector< vector< vector<
double > > > data;)
3-copy the data from the file to this vector.

is this possible with the current library?

My second question, is there a function in the library to read a single
element? I just found a function for reading a whole set into an array in
the tutorial:

status = H5Dread(dataset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT,
dset_data);

If there's a tutorial that is more detailed than this, or let's say a
manual, I'd be so happy to know about its existence.

Thank you, any efforts are highly appreciated.

Best regards,
Samer

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

3

Hi Samer,

I think that the HDF5 C API documentation contains enough information
to find your way.

I have made my own C++ interface on top of the C API of HDF5.
It uses H5Dget_type to get the data type of a data set.
The shape can be obtained like:

    int rank = H5Sget_simple_extent_ndims(itsDSid);
    vector<hsize_t> shp(rank);
    if (rank > 0) {
      rank = H5Sget_simple_extent_dims(itsDSid, &(shp[0]), NULL);
    }

where itsDSId is the data space id of the data set.

You cannot simply get the data into a vector<vector<vector>>> because
such a structure has non-contiguous memory. You can only do it hyperslab
by hyperslab which can be slowish. But why don't use you an array
package like boost::multi_array (or Blitz or casacore Arrays) to hold
your array data?

You can get a single element using the appropriate hyperslab, but
iterating in that way through your data set will be very slow.

Cheers,
Ger

<samer@icp.uni-stuttgart.de> 6/30/2010 12:11 PM >>>

Hello programmers :-),

I'm still new to this HDF file format, and what I could do is run the
C++
examples given in the tutorial to read a pressures data file given to
me
by someone.

My first question is, is it possible to read the values of my H5 file
to a
vector? all I've seen is array works, which is not convenient when
working
generally with different sizes of data files (the files I'm using are
3D
matrices).

So what I would like to do, is having a program in this scheme:

1-Read the file and determine the data type and size
2-create a 3D vector for reading the data (like vector< vector<
vector<
double > > > data;)
3-copy the data from the file to this vector.

is this possible with the current library?

My second question, is there a function in the library to read a
single
element? I just found a function for reading a whole set into an array
in
the tutorial:

status = H5Dread(dataset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL,
H5P_DEFAULT,
dset_data);

If there's a tutorial that is more detailed than this, or let's say a
manual, I'd be so happy to know about its existence.

Thank you, any efforts are highly appreciated.

Best regards,
Samer

···

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

4

I suspect you'd be better off abandoning the nested

  std::vector< std::vector< std::vector< double > > >

construct in favor of either

(A) A single

  std::valarray< double >

manipulated through a 3D std::slice

or (B) A single

  boost::multi_array< double, 3 >

Either way should enable you to read from an HDF file into a 3D array of
type double.

Regards.
Greg

···

-----Original Message-----
From: hdf-forum-bounces@hdfgroup.org
[mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of
samer@icp.uni-stuttgart.de
Sent: Wednesday, June 30, 2010 3:11 AM
To: hdf-forum@hdfgroup.org
Subject: [Hdf-forum] Read to a vector, read a single element

Hello programmers :-),

I'm still new to this HDF file format, and what I could do is run the
C++
examples given in the tutorial to read a pressures data file given to me
by someone.

My first question is, is it possible to read the values of my H5 file to
a
vector? all I've seen is array works, which is not convenient when
working
generally with different sizes of data files (the files I'm using are 3D
matrices).

So what I would like to do, is having a program in this scheme:

1-Read the file and determine the data type and size
2-create a 3D vector for reading the data (like vector< vector< vector<
double > > > data;)
3-copy the data from the file to this vector.

is this possible with the current library?

My second question, is there a function in the library to read a single
element? I just found a function for reading a whole set into an array
in
the tutorial:

status = H5Dread(dataset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL,
H5P_DEFAULT,
dset_data);

If there's a tutorial that is more detailed than this, or let's say a
manual, I'd be so happy to know about its existence.

Thank you, any efforts are highly appreciated.

Best regards,
Samer

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

5

Thank you all guys for the answers. :slight_smile: I think I got it

···

I suspect you'd be better off abandoning the nested

  std::vector< std::vector< std::vector< double > > >

construct in favor of either

(A) A single

  std::valarray< double >

manipulated through a 3D std::slice

or (B) A single

  boost::multi_array< double, 3 >

Either way should enable you to read from an HDF file into a 3D array of
type double.

Regards.
Greg

-----Original Message-----
From: hdf-forum-bounces@hdfgroup.org
[mailto:hdf-forum-bounces@hdfgroup.org] On Behalf Of
samer@icp.uni-stuttgart.de
Sent: Wednesday, June 30, 2010 3:11 AM
To: hdf-forum@hdfgroup.org
Subject: [Hdf-forum] Read to a vector, read a single element

Hello programmers :-),

I'm still new to this HDF file format, and what I could do is run the
C++
examples given in the tutorial to read a pressures data file given to me
by someone.

My first question is, is it possible to read the values of my H5 file to
a
vector? all I've seen is array works, which is not convenient when
working
generally with different sizes of data files (the files I'm using are 3D
matrices).

So what I would like to do, is having a program in this scheme:

1-Read the file and determine the data type and size
2-create a 3D vector for reading the data (like vector< vector< vector<
double > > > data;)
3-copy the data from the file to this vector.

is this possible with the current library?

My second question, is there a function in the library to read a single
element? I just found a function for reading a whole set into an array
in
the tutorial:

status = H5Dread(dataset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL,
H5P_DEFAULT,
dset_data);

If there's a tutorial that is more detailed than this, or let's say a
manual, I'd be so happy to know about its existence.

Thank you, any efforts are highly appreciated.

Best regards,
Samer

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org