Reading Compound Datatype using Hyperslab Concept

HDF5 Library
1

Hi Greetings,

I am having a compound datatype dataset, I need to read the data from it using hyper slab concept. But I am getting only [0][0] is correct value. what is datatype I should for out put buffer in H5Dread function, I am having compound datatypes with types unsigned 64 bit. Unsigned 32 bit, unsigned 8 bit …,Can you help me on this.

I having function H5Dget_type(dataset name) ,How I can find datatype of it like Int/float/…

2

Have a look at https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5-examples/browse/1_10/C/H5T/h5ex_t_cmpd.c. There are also examples for other languages.

3

If c++ is an option for you check out H5CPP a project that supports compiler assisted reflection and seamless persistence for arbitrary deep POD structs and popular linear algebra systems.
Here is an excerpt from github example: 

#include <cstdint>
#include "struct.h"
#include <h5cpp/core> 
	#include "generated.h"
#include <h5cpp/io>
#include "utils.hpp"

int main(){

   h5::create<sn::example::Record>(fd, "/orm/chunked_2D", 
      h5::current_dims{NROWS,NCOLS}, h5::chunk{1,CHUNK_SIZE} | h5::gzip{8} );
   // read dataset back
   auto data = h5::read<T>(fd,"/orm/chunked_2D");
}

In make file invoce h5cpp the LLVM based reflection tool / compiler to generate generated.h which will contain all HDF5 CAPI related shim codes. The compiler is freely available on the project webpage for debian/ubuntu and redhat based systems.

generated.h: struct.h
	h5cpp  struct.cpp -- $(CXXFLAGS) $(INCLUDES) -I/usr/include/h5cpp-llvm -Dgenerated.h

Detailed examples provided with the project, download and install from here
best wishes: steven

1 Like
4

Hi gheber,

Thanks for the example, but I am trying to read the data with out defining structures and I am getting entire data if all the member names of same datatype, I using Hyper slab concept for this. I am unable to get the data of dataset with different datatypes . Can u suggest alternative way to get values without defining structures.

auto *rdataslap = new unsigned long long[ROWS]COLUMN;

	 start[0] = 0; //start for row
	 start[1] = 1; // start for column
	 stride[0] = 1;
	 stride[1] = 1;
	 count[0] = 1;
	 count[1] = 1;
	 block[0] = 1;
     block[1] = 1;
	herr_t statusslap  = H5Sselect_hyperslab(dataspace, H5S_SELECT_SET, start, stride, count, block); // (13)
	cout << "hyperslap status" << statusslap << endl;

	herr_t status = H5Dread(datasetname, H5Dget_type(datasetname), H5S_ALL, dataspace, H5P_DEFAULT,
		rdataslap); // (14)
5

In the end, it’s all bytes: The last argument to H5Dread is opaque (void*). Since you know the file and memory types, and the extent of the selection, if you want to be clever, you can just calculate a minimum buffer size (in bytes) you need and go for it. (Beware of alignment constraints!) Good luck w/ the reinterpret_casts.

6

Hi Gheber Greetings,

I am trying to read as separate column of every member data using hyper slab, But I am getting the data in horizontal manner( getting row data) means it is not reading the column data(vertical manner) .

              hsize_t count[2];
	 hsize_t col_dims[1];
	  col_dims[0]= 5;
	  hsize_t	chunk_dims[2] = { 5, 1 };
	//unsigned  long *column = new unsigned  long[300];
	  unsigned long column[300];
	hsize_t offset[2];
	hid_t filespace = H5Dget_space(datasetname);	
	hid_t memspace = H5Screate_simple(1, col_dims,NULL);

	 /*
	 * Define the column (hyperslab) to read.
	 */
	 offset[0] = 0; // start point of row
	 offset[1] = 0; // start point of column
	 count[0] = 5; // row count
	 count[1] = 1; // column count
	 herr_t statusslab = H5Sselect_hyperslab(filespace, H5S_SELECT_SET, offset, NULL,
		 count, NULL);
	cout << "hyperslab status  " << statusslab << endl;

	herr_t statusread = H5Dread(datasetname, H5Dget_type(datasetname), memspace, filespace,
		 H5P_DEFAULT,column);
	cout << "read status  " << statusread << endl;

	 printf("\n");
	 printf(" column: \n");
	 for (int i = 0; i < 5; i++) {
		 printf("%lu \n", column[i]);
	 }
7

Any one help me on this.

8

Hello!

Attached is an example program that shows how to edit a specific field in a dataset with a compound datatype. Is this what you wish to do?

cmpfstred.c (4.2 KB)

This example edits a fixed length string field.
It does not use hyperslab selection, as that can only be performed on a dataspace.

-Barbara

9

Hi Greetings,

Thanks for example, but this is not exactly what I needed. I having dataset of compound data type. My aim is read each column of it using hyper slab, I am not writing/modifying any new data. I need to read each member name column separate.

10

Hello bljones,

Currently I am able to get all data using hyper slab selection(all member names having same datatype),However for member names with different datatypes I have to get data separating each member name as individual column. I have to achieve it without defining structures.

11

hi @chandrasekhar.kolapa, I am now reading compound dataset by defining the struct but I have encountered a dilemma where I am now unable to read a large dataset into the memory space.
I have discovered that I have to chunk the big dataset using hyper slab concept. But I am unable to find a CPP example that allows me to do chunking of compound datatype of a large dataset using hypeslab concept.

if you could post your example on how you read compound datatype here means will be of great help.