Reading VLEN string attribute values.

Hdf-forum archives
1

The output from h5dump is:

ATTRIBUTE "ORIGIN" {
    DATATYPE H5T_STRING {
       STRSIZE H5T_VARIABLE;
       STRPAD H5T_STR_NULLTERM;
       CSET H5T_CSET_ASCII;
       CTYPE H5T_C_S1;
    }
    DATASPACE SIMPLE { ( 1 ) / ( 1 ) }
    DATA {
    (0): "BASIC"
    }
}

And I tried:

int byteLength = Marshal.SizeOf(typeof(IntPtr));
IntPtr intPtr = Marshal.AllocHGlobal(byteLength);
H5A.read(aid, typeId, intPtr);
string result = Marshal.PtrToStringAnsi(intPtr);

And result contains "\u0001\u0002\b\n\n".

Matt

2

The dataspace is simple, i.e., it's an array which in this case
happens to have only one element. (This is NOT the same as a scalar attribute!)
Have a look at H5DreadTest2 in

        [TestMethod]
        public void H5DreadTest2()
        {
            hid_t mem_type = H5T.create(H5T.class_t.STRING, H5T.VARIABLE);
            Assert.IsTrue(H5T.set_cset(mem_type, H5T.cset_t.UTF8) >= 0);
            Assert.IsTrue(H5T.set_strpad(mem_type, H5T.str_t.NULLTERM) >= 0);

            hid_t fspace = H5D.get_space(m_v0_utf8_dset);
            Assert.IsTrue(fspace >= 0);

            hssize_t count = H5S.get_simple_extent_npoints(fspace);
            Assert.IsTrue(count > 0);
            Assert.IsTrue(H5S.close(fspace) >= 0);

            IntPtr[] rdata = new IntPtr[count];
            GCHandle hnd = GCHandle.Alloc(rdata, GCHandleType.Pinned);
            Assert.IsTrue(H5D.read(m_v0_utf8_dset, mem_type, H5S.ALL, H5S.ALL,
                H5P.DEFAULT, hnd.AddrOfPinnedObject()) >= 0);
            
            for (int i = 0; i < rdata.Length; ++i)
            {
                int len = 0;
                while (Marshal.ReadByte(rdata[i], len) != 0) { ++len; }
                byte[] buffer = new byte[len];
                Marshal.Copy(rdata[i], buffer, 0, buffer.Length);
                string s = Encoding.UTF8.GetString(buffer);

                Assert.IsTrue(s == ((string)m_utf8strings[i]));

                Assert.IsTrue(H5.free_memory(rdata[i]) >= 0);
            }

            ...

            hnd.Free();
        }
    }

(The example is for datasets, but applies the same way for attributes.)

Your snippet is almost correct (i.e., allocate an IntPtr array of size 1).
The part that's wrong is that data needs no initialization or byte array allocation.
Reading a simple dataset of variable-length strings returns a pointer array.
(You must free the strings later or leak memory!)

G.

···

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Matt Wood
Sent: Thursday, March 30, 2017 10:24 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] Reading VLEN string attribute values.

The output from h5dump is:

ATTRIBUTE "ORIGIN" {
    DATATYPE H5T_STRING {
       STRSIZE H5T_VARIABLE;
       STRPAD H5T_STR_NULLTERM;
       CSET H5T_CSET_ASCII;
       CTYPE H5T_C_S1;
    }
    DATASPACE SIMPLE { ( 1 ) / ( 1 ) }
    DATA {
    (0): "BASIC"
    }
}

And I tried:

int byteLength = Marshal.SizeOf(typeof(IntPtr)); IntPtr intPtr = Marshal.AllocHGlobal(byteLength); H5A.read(aid, typeId, intPtr); string result = Marshal.PtrToStringAnsi(intPtr);

And result contains "\u0001\u0002\b\n\n".

Matt

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

3

Many thanks, that worked a treat. I have attached my working method below.

Matt

/// <summary>
/// Helper method that extracts a string array value from a variable length attribute.
/// See https://github.com/HDFGroup/HDF.PInvoke/blob/master/UnitTests/H5DTest/H5Dread.cs
/// </summary>
/// <param name="aid">The id of the attribute.</param>
/// <param name="tid">The type id of the attribute.</param>
/// <returns>An array of strings.</returns>
public string[] GetStrings(long aid, long tid)
{
      long space = H5A.get_space(aid);
      // TODO close space
      long count = H5S.get_simple_extent_npoints(space);
      // Create pointer [] to receive the addresses.
      IntPtr[] rdata = new IntPtr[count];
      // Create array for result.
      string[] result = new string[count];
      GCHandle hnd = GCHandle.Alloc(rdata, GCHandleType.Pinned);
      // Call the function with address of pointer [].
      H5A.read(aid, tid, hnd.AddrOfPinnedObject());
      hnd.Free();
      // For each pointer extract string.
      for (int ii = 0; ii < rdata.Length; ++ii)
      {
      int len = 0;
      // Find the end of the string (\0)
     while (Marshal.ReadByte(rdata[ii], len) != 0) { ++len; }
           byte[] buffer = new byte[len];
           // Copy to buffer
Marshal.Copy(rdata[ii], buffer, 0, buffer.Length);
           // Encode the string.
           result[ii] = Encoding.UTF8.GetString(buffer);
      }
      return result;
}

4

Glad to hear that it works.

Don’t forget to free the buffer(s) that the HDF5 library
allocated for those strings or you’ll have created a memory leak:

Assert.IsTrue(H5.free_memory(rdata[i]) >= 0);

G.

···

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Matt Wood
Sent: Thursday, March 30, 2017 11:23 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] Reading VLEN string attribute values.

Many thanks, that worked a treat. I have attached my working method below.

Matt

/// <summary>
/// Helper method that extracts a string array value from a variable length attribute.
/// See https://github.com/HDFGroup/HDF.PInvoke/blob/master/UnitTests/H5DTest/H5Dread.cs
/// </summary>
/// <param name="aid">The id of the attribute.</param>
/// <param name="tid">The type id of the attribute.</param>
/// <returns>An array of strings.</returns>
public string[] GetStrings(long aid, long tid)
{
     long space = H5A.get_space(aid);
     // TODO close space
     long count = H5S.get_simple_extent_npoints(space);
     // Create pointer [] to receive the addresses.
     IntPtr[] rdata = new IntPtr[count];
     // Create array for result.
     string[] result = new string[count];
     GCHandle hnd = GCHandle.Alloc(rdata, GCHandleType.Pinned);
     // Call the function with address of pointer [].
     H5A.read(aid, tid, hnd.AddrOfPinnedObject());
     hnd.Free();
     // For each pointer extract string.
     for (int ii = 0; ii < rdata.Length; ++ii)
     {
          int len = 0;
          // Find the end of the string (\0)
          while (Marshal.ReadByte(rdata[ii], len) != 0) { ++len; }
          byte[] buffer = new byte[len];
          // Copy to buffer
          Marshal.Copy(rdata[ii], buffer, 0, buffer.Length);
          // Encode the string.
          result[ii] = Encoding.UTF8.GetString(buffer);
     }
     return result;
}