Reading VLEN string attribute values.

Matt_Wood · March 30, 2017, 3:24pm

The output from h5dump is:

ATTRIBUTE "ORIGIN" {
    DATATYPE H5T_STRING {
       STRSIZE H5T_VARIABLE;
       STRPAD H5T_STR_NULLTERM;
       CSET H5T_CSET_ASCII;
       CTYPE H5T_C_S1;
    }
    DATASPACE SIMPLE { ( 1 ) / ( 1 ) }
    DATA {
    (0): "BASIC"
    }
}

And I tried:

int byteLength = Marshal.SizeOf(typeof(IntPtr));
IntPtr intPtr = Marshal.AllocHGlobal(byteLength);
H5A.read(aid, typeId, intPtr);
string result = Marshal.PtrToStringAnsi(intPtr);

And result contains "\u0001\u0002\b\n\n".

Matt

gheber · March 30, 2017, 3:46pm

The dataspace is simple, i.e., it's an array which in this case
happens to have only one element. (This is NOT the same as a scalar attribute!)
Have a look at H5DreadTest2 in

github.com

HDFGroup/HDF.PInvoke/blob/master/UnitTests/H5DTest/H5Dread.cs

/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 * Copyright by The HDF Group.                                               *
 * Copyright by the Board of Trustees of the University of Illinois.         *
 * All rights reserved.                                                      *
 *                                                                           *
 * This file is part of HDF5.  The full HDF5 copyright notice, including     *
 * terms governing use, modification, and redistribution, is contained in    *
 * the files COPYING and Copyright.html.  COPYING can be found at the root   *
 * of the source code distribution tree; Copyright.html can be found at the  *
 * root level of an installed copy of the electronic HDF5 document set and   *
 * is linked from the top-level documents page.  It can also be found at     *
 * http://hdfgroup.org/HDF5/doc/Copyright.html.  If you do not have          *
 * access to either file, you may request a copy from help@hdfgroup.org.     *
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */

using System;
using System.Collections;
using System.IO;
using System.Runtime.InteropServices;
using System.Text;

This file has been truncated. show original

        [TestMethod]
        public void H5DreadTest2()
        {
            hid_t mem_type = H5T.create(H5T.class_t.STRING, H5T.VARIABLE);
            Assert.IsTrue(H5T.set_cset(mem_type, H5T.cset_t.UTF8) >= 0);
            Assert.IsTrue(H5T.set_strpad(mem_type, H5T.str_t.NULLTERM) >= 0);

hid_t fspace = H5D.get_space(m_v0_utf8_dset);
Assert.IsTrue(fspace >= 0);

            hssize_t count = H5S.get_simple_extent_npoints(fspace);
            Assert.IsTrue(count > 0);
            Assert.IsTrue(H5S.close(fspace) >= 0);

            IntPtr[] rdata = new IntPtr[count];
            GCHandle hnd = GCHandle.Alloc(rdata, GCHandleType.Pinned);
            Assert.IsTrue(H5D.read(m_v0_utf8_dset, mem_type, H5S.ALL, H5S.ALL,
                H5P.DEFAULT, hnd.AddrOfPinnedObject()) >= 0);

            for (int i = 0; i < rdata.Length; ++i)
            {
                int len = 0;
                while (Marshal.ReadByte(rdata[i], len) != 0) { ++len; }
                byte[] buffer = new byte[len];
                Marshal.Copy(rdata[i], buffer, 0, buffer.Length);
                string s = Encoding.UTF8.GetString(buffer);

Assert.IsTrue(s == ((string)m_utf8strings[i]));

Assert.IsTrue(H5.free_memory(rdata[i]) >= 0);
}

...

            hnd.Free();
        }
    }

(The example is for datasets, but applies the same way for attributes.)

Your snippet is almost correct (i.e., allocate an IntPtr array of size 1).
The part that's wrong is that data needs no initialization or byte array allocation.
Reading a simple dataset of variable-length strings returns a pointer array.
(You must free the strings later or leak memory!)

G.

···

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Matt Wood
Sent: Thursday, March 30, 2017 10:24 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] Reading VLEN string attribute values.

The output from h5dump is:

ATTRIBUTE "ORIGIN" {
    DATATYPE H5T_STRING {
       STRSIZE H5T_VARIABLE;
       STRPAD H5T_STR_NULLTERM;
       CSET H5T_CSET_ASCII;
       CTYPE H5T_C_S1;
    }
    DATASPACE SIMPLE { ( 1 ) / ( 1 ) }
    DATA {
    (0): "BASIC"
    }
}

And I tried:

int byteLength = Marshal.SizeOf(typeof(IntPtr)); IntPtr intPtr = Marshal.AllocHGlobal(byteLength); H5A.read(aid, typeId, intPtr); string result = Marshal.PtrToStringAnsi(intPtr);

And result contains "\u0001\u0002\b\n\n".

Matt

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

Matt_Wood · March 30, 2017, 4:23pm

Many thanks, that worked a treat. I have attached my working method below.

Matt

/// <summary>
/// Helper method that extracts a string array value from a variable length attribute.
/// See https://github.com/HDFGroup/HDF.PInvoke/blob/master/UnitTests/H5DTest/H5Dread.cs
/// </summary>
/// <param name="aid">The id of the attribute.</param>
/// <param name="tid">The type id of the attribute.</param>
/// <returns>An array of strings.</returns>
public string[] GetStrings(long aid, long tid)
{
      long space = H5A.get_space(aid);
      // TODO close space
      long count = H5S.get_simple_extent_npoints(space);
      // Create pointer [] to receive the addresses.
      IntPtr[] rdata = new IntPtr[count];
      // Create array for result.
      string[] result = new string[count];
      GCHandle hnd = GCHandle.Alloc(rdata, GCHandleType.Pinned);
      // Call the function with address of pointer [].
      H5A.read(aid, tid, hnd.AddrOfPinnedObject());
      hnd.Free();
      // For each pointer extract string.
      for (int ii = 0; ii < rdata.Length; ++ii)
      {
      int len = 0;
      // Find the end of the string (\0)
     while (Marshal.ReadByte(rdata[ii], len) != 0) { ++len; }
           byte[] buffer = new byte[len];
           // Copy to buffer
Marshal.Copy(rdata[ii], buffer, 0, buffer.Length);
           // Encode the string.
           result[ii] = Encoding.UTF8.GetString(buffer);
      }
      return result;
}

gheber · March 30, 2017, 4:29pm

Glad to hear that it works.

Don’t forget to free the buffer(s) that the HDF5 library
allocated for those strings or you’ll have created a memory leak:

Assert.IsTrue(H5.free_memory(rdata[i]) >= 0);

G.

···

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Matt Wood
Sent: Thursday, March 30, 2017 11:23 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] Reading VLEN string attribute values.

Many thanks, that worked a treat. I have attached my working method below.

Matt

/// <summary>
/// Helper method that extracts a string array value from a variable length attribute.
/// See https://github.com/HDFGroup/HDF.PInvoke/blob/master/UnitTests/H5DTest/H5Dread.cs
/// </summary>
/// <param name="aid">The id of the attribute.</param>
/// <param name="tid">The type id of the attribute.</param>
/// <returns>An array of strings.</returns>
public string[] GetStrings(long aid, long tid)
{
     long space = H5A.get_space(aid);
     // TODO close space
     long count = H5S.get_simple_extent_npoints(space);
     // Create pointer [] to receive the addresses.
     IntPtr[] rdata = new IntPtr[count];
     // Create array for result.
     string[] result = new string[count];
     GCHandle hnd = GCHandle.Alloc(rdata, GCHandleType.Pinned);
     // Call the function with address of pointer [].
     H5A.read(aid, tid, hnd.AddrOfPinnedObject());
     hnd.Free();
     // For each pointer extract string.
     for (int ii = 0; ii < rdata.Length; ++ii)
     {
          int len = 0;
          // Find the end of the string (\0)
          while (Marshal.ReadByte(rdata[ii], len) != 0) { ++len; }
          byte[] buffer = new byte[len];
          // Copy to buffer
          Marshal.Copy(rdata[ii], buffer, 0, buffer.Length);
          // Encode the string.
          result[ii] = Encoding.UTF8.GetString(buffer);
     }
     return result;
}