Read hdf5 string dataset with C#

Hdf-forum archives
1

Dear all,

I am developing a .Net software MeteoInfo to view and analysis
multi-source meteorological datasets, it is freely available at
http://www.meteothinker.com . The data formats of NetCDF, GRIB 1 & 2
and so on have been supported, but HDF format is still a big
challenge. I want to try HDF5-EOS dataset firstly. I am using C# and
the .Net wrapper library from HDF website. For a simple special
dataset, I can get the Longitude, Latitude, Pressure dimentions and
plot a contour layer (attached figure). In my understanding, the
projection and dimension information should be got from
'StructMetadata.x' dataset under 'HDFEOS INFORMATION' group.
'StructMetadata.x' is a 'HDF5 Scalar Dataset' and the data type is
'String'.

But I can't get 'StructMetadata.x' dataset string value by my
following code. outdata[0] is null. Can anybody help me? Thanks a lot!

            string aFile = "E:\\temp\\TES-Aura_L3-H2O_r0000006641_F01_04.he5";
            _fileID = H5F.open(aFile, H5F.OpenMode.ACC_RDONLY);

            //Open meta data group
            H5GroupId infoGId = H5G.open(_fileID, "HDFEOS INFORMATION");
            ulong num_meta = H5G.getNumObjects(infoGId);
            for (int i = 0; i < (int)num_meta; i++)
            {
                string metaName = H5G.getObjectNameByIndex(infoGId, i);
                if (metaName.Contains("StructMetadata"))
                {
                    ObjectInfo metaInfo = H5G.getObjectInfo(infoGId,
metaName, false);
                    H5DataSetId dsId = H5D.open(infoGId, metaName);
                    H5DataSpaceId spaId = H5D.getSpace(dsId);
                    H5DataTypeId dtId = H5D.getType(dsId);
                    H5T.H5TClass tcls = H5T.getClass(dtId);
                    int rank = H5S.getSimpleExtentNDims(spaId);
                    ulong[] dims = H5S.getSimpleExtentDims(spaId);
                    string[] outdata = new string[1];
                    H5DataTypeId dtypeId = H5T.copy(H5T.H5Type.C_S1);
                    H5D.read(dsId, dtypeId, new H5Array<string>(outdata));
                    _metaStr = outdata[0];

                    H5D.close(dsId);
                    H5S.close(spaId);
                    H5T.close(dtId);

                    break;
                }
            }

Image00781.png1000×650 91.7 KB

···

--
*************************************************
Dr. Yaqiang Wang
Chinese Academy of Meteorological Sciences (CAMS)
46, Zhong-Guan-Cun South Avenue
Beijing, 100081
China

yaqiang.wang@gmail.com

**************************************************

2

Yaqiang, how are you? Your code is pretty close, but misses a technical
detail.
The 'StructMetadata.x' scalar datasets of fixed-length ASCII encoded
strings.
Strings in .NET are Unicode-based and use 2-byte character representations
(as opposed to 1-byte for ASCII). It'd be nice to directly do a

H5D.read(dsId, dtypeId, new H5Array<string>(outdata));

but that doesn't work currently. Please have a look at the attached
IronPython script
that shows what to do and adapt it to C#. The main change is to read into an
array of
System.Byte and then use System.Text.ASCIIEncoding to get an ASCII
representation
of the string, i.e.,

...
mtype = H5T.copy(H5T.H5Type.C_S1)
H5T.setSize(mtype, size)
buffer = Array.CreateInstance(System.Byte, size)
H5D.read(dset, mtype, H5Array[Byte](buffer))
print System.Text.ASCIIEncoding.ASCII.GetString(buffer) # <- that's
'StructMetadata.0
....

Best, G.

StructMetadata.0.py (1.09 KB)

···

-----Original Message-----
From: hdf-forum-bounces@hdfgroup.org [mailto:hdf-forum-bounces@hdfgroup.org]
On Behalf Of Yaqiang Wang
Sent: Wednesday, November 23, 2011 3:19 AM
To: hdf-forum@hdfgroup.org
Subject: [Hdf-forum] Read hdf5 string dataset with C#

Dear all,

I am developing a .Net software MeteoInfo to view and analysis multi-source
meteorological datasets, it is freely available at
http://www.meteothinker.com . The data formats of NetCDF, GRIB 1 & 2 and so
on have been supported, but HDF format is still a big challenge. I want to
try HDF5-EOS dataset firstly. I am using C# and the .Net wrapper library
from HDF website. For a simple special dataset, I can get the Longitude,
Latitude, Pressure dimentions and plot a contour layer (attached figure). In
my understanding, the projection and dimension information should be got
from 'StructMetadata.x' dataset under 'HDFEOS INFORMATION' group.
'StructMetadata.x' is a 'HDF5 Scalar Dataset' and the data type is 'String'.

But I can't get 'StructMetadata.x' dataset string value by my following
code. outdata[0] is null. Can anybody help me? Thanks a lot!

            string aFile =
"E:\\temp\\TES-Aura_L3-H2O_r0000006641_F01_04.he5";
            _fileID = H5F.open(aFile, H5F.OpenMode.ACC_RDONLY);

            //Open meta data group
            H5GroupId infoGId = H5G.open(_fileID, "HDFEOS INFORMATION");
            ulong num_meta = H5G.getNumObjects(infoGId);
            for (int i = 0; i < (int)num_meta; i++)
            {
                string metaName = H5G.getObjectNameByIndex(infoGId, i);
                if (metaName.Contains("StructMetadata"))
                {
                    ObjectInfo metaInfo = H5G.getObjectInfo(infoGId,
metaName, false);
                    H5DataSetId dsId = H5D.open(infoGId, metaName);
                    H5DataSpaceId spaId = H5D.getSpace(dsId);
                    H5DataTypeId dtId = H5D.getType(dsId);
                    H5T.H5TClass tcls = H5T.getClass(dtId);
                    int rank = H5S.getSimpleExtentNDims(spaId);
                    ulong[] dims = H5S.getSimpleExtentDims(spaId);
                    string[] outdata = new string[1];
                    H5DataTypeId dtypeId = H5T.copy(H5T.H5Type.C_S1);
                    H5D.read(dsId, dtypeId, new H5Array<string>(outdata));
                    _metaStr = outdata[0];

                    H5D.close(dsId);
                    H5S.close(spaId);
                    H5T.close(dtId);

                    break;
                }
            }

--
*************************************************
Dr. Yaqiang Wang
Chinese Academy of Meteorological Sciences (CAMS) 46, Zhong-Guan-Cun South
Avenue Beijing, 100081 China

yaqiang.wang@gmail.com

**************************************************

3

Dear Gerd,

It works now following your code. Thanks!

Best regards
Yaqiang

···

On Thu, Nov 24, 2011 at 2:11 AM, Gerd Heber <gheber@hdfgroup.org> wrote:

Yaqiang, how are you? Your code is pretty close, but misses a technical
detail.
The 'StructMetadata.x' scalar datasets of fixed-length ASCII encoded
strings.
Strings in .NET are Unicode-based and use 2-byte character representations
(as opposed to 1-byte for ASCII). It'd be nice to directly do a

H5D.read(dsId, dtypeId, new H5Array<string>(outdata));

but that doesn't work currently. Please have a look at the attached
IronPython script
that shows what to do and adapt it to C#. The main change is to read into an
array of
System.Byte and then use System.Text.ASCIIEncoding to get an ASCII
representation
of the string, i.e.,

...
mtype = H5T.copy(H5T.H5Type.C_S1)
H5T.setSize(mtype, size)
buffer = Array.CreateInstance(System.Byte, size)
H5D.read(dset, mtype, H5Array[Byte](buffer))
print System.Text.ASCIIEncoding.ASCII.GetString(buffer) # <- that's
'StructMetadata.0
....

Best, G.

-----Original Message-----
From: hdf-forum-bounces@hdfgroup.org [mailto:hdf-forum-bounces@hdfgroup.org]
On Behalf Of Yaqiang Wang
Sent: Wednesday, November 23, 2011 3:19 AM
To: hdf-forum@hdfgroup.org
Subject: [Hdf-forum] Read hdf5 string dataset with C#

Dear all,

I am developing a .Net software MeteoInfo to view and analysis multi-source
meteorological datasets, it is freely available at
http://www.meteothinker.com . The data formats of NetCDF, GRIB 1 & 2 and so
on have been supported, but HDF format is still a big challenge. I want to
try HDF5-EOS dataset firstly. I am using C# and the .Net wrapper library
from HDF website. For a simple special dataset, I can get the Longitude,
Latitude, Pressure dimentions and plot a contour layer (attached figure). In
my understanding, the projection and dimension information should be got
from 'StructMetadata.x' dataset under 'HDFEOS INFORMATION' group.
'StructMetadata.x' is a 'HDF5 Scalar Dataset' and the data type is 'String'.

But I can't get 'StructMetadata.x' dataset string value by my following
code. outdata[0] is null. Can anybody help me? Thanks a lot!

       string aFile =

"E:\\temp\\TES-Aura_L3-H2O_r0000006641_F01_04.he5";
_fileID = H5F.open(aFile, H5F.OpenMode.ACC_RDONLY);

       //Open meta data group
       H5GroupId infoGId = H5G\.open\(\_fileID, &quot;HDFEOS INFORMATION&quot;\);
       ulong num\_meta = H5G\.getNumObjects\(infoGId\);
       for \(int i = 0; i &lt; \(int\)num\_meta; i\+\+\)
       \{
           string metaName = H5G\.getObjectNameByIndex\(infoGId, i\);
           if \(metaName\.Contains\(&quot;StructMetadata&quot;\)\)
           \{
               ObjectInfo metaInfo = H5G\.getObjectInfo\(infoGId,

metaName, false);
H5DataSetId dsId = H5D.open(infoGId, metaName);
H5DataSpaceId spaId = H5D.getSpace(dsId);
H5DataTypeId dtId = H5D.getType(dsId);
H5T.H5TClass tcls = H5T.getClass(dtId);
int rank = H5S.getSimpleExtentNDims(spaId);
ulong[] dims = H5S.getSimpleExtentDims(spaId);
string[] outdata = new string[1];
H5DataTypeId dtypeId = H5T.copy(H5T.H5Type.C_S1);
H5D.read(dsId, dtypeId, new H5Array<string>(outdata));
_metaStr = outdata[0];

               H5D\.close\(dsId\);
               H5S\.close\(spaId\);
               H5T\.close\(dtId\);

               break;
           \}
       \}

--
*************************************************
Dr. Yaqiang Wang
Chinese Academy of Meteorological Sciences (CAMS) 46, Zhong-Guan-Cun South
Avenue Beijing, 100081 China

yaqiang.wang@gmail.com

http://www.meteothinker.com
**************************************************

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

--
*************************************************
Dr. Yaqiang Wang
Chinese Academy of Meteorological Sciences (CAMS)
46, Zhong-Guan-Cun South Avenue
Beijing, 100081
China

yaqiang.wang@gmail.com

http://www.meteothinker.com
**************************************************