Problem with creating string attributes from Java

Could someone please look at the code below and tell me what's wrong with it? It creates a string dataset and attaches a string array attribute. But when I read the attribute value back, I get only garbage. I also see garbage with HDFView, so the bug is in writing, not reading. When I use integer data instead of strings it works fine, BTW.

Thanks in advance,
  Konrad.

···

--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Moléculaire, CNRS Orléans
Synchrotron Soleil - Division Expériences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: research AT khinsen DOT fastmail DOT net
---------------------------------------------------------------------

import ncsa.hdf.object.*; // the common object package
import ncsa.hdf.object.h5.*; // the HDF5 implementation
import java.util.List;

public class StringAttributeTest
{
    private static String fname = "StringAttributeTest.h5";

    public static void main( String args[] ) throws Exception
    {
        FileFormat fileFormat = FileFormat.getFileFormat(FileFormat.FILE_TYPE_HDF5);
        if (fileFormat == null)
        {
            System.err.println("Cannot find HDF5 FileFormat.");
            return;
        }

        H5File testFile = (H5File)fileFormat.create(fname);
        if (testFile == null)
        {
            System.err.println("Failed to create file:"+fname);
            return;
        }

        testFile.open();
        Group root = (Group)((javax.swing.tree.DefaultMutableTreeNode)testFile.getRootNode()).getUserObject();

        String[] data = {"abc de fghi jk lmn op qrst uvw xyz\n0 12 345 6 789"};
        long[] data_dims = {1};
        Datatype dtype = testFile.createDatatype(
            Datatype.CLASS_STRING, data[0].length(),
            Datatype.NATIVE, Datatype.NATIVE);
        Dataset dataset = testFile.createScalarDS
            ("text", root, dtype, data_dims, null, null, 0, data);

        long[] attr_dims = {3};
        String[] attr_value = {"a", "ab", "abc"};
        Datatype attr_dtype = testFile.createDatatype(
            Datatype.CLASS_STRING, 5,
            Datatype.NATIVE, Datatype.NATIVE);
        Attribute attr = new Attribute("foo", attr_dtype, attr_dims);
        attr.setValue(attr_value);
        dataset.writeMetadata(attr);

        testFile.close();

        // read attributes back
        testFile = (H5File)fileFormat.open(fname, FileFormat.READ);
        if (testFile == null)
        {
            System.err.println("Failed to open file:"+fname);
            return;
        }
        testFile.open();
        root = (Group)((javax.swing.tree.DefaultMutableTreeNode)testFile.getRootNode()).getUserObject();
        dataset = (Dataset)root.getMemberList().get(0);
        List attrList = dataset.getMetadata();
        attr = (Attribute)attrList.get(0);
        String[] value = (String [])attr.getValue();
        System.out.println( attr.toString() );
        System.out.println( value[0] );
        System.out.println( value[1] );
        System.out.println( value[2] );
        testFile.close();
    }

}

Directly writing attribute of array of strings is not supported. We may add this feature in future
release. The workaround is to convert it to array of bytes, e.g.
         byte[] bvalue = Dataset.stringToByte(attr_value, 5);
         attr.setValue(bvalue);

Thanks
--pc

···

On 3/25/2011 6:58 AM, Konrad Hinsen wrote:

Could someone please look at the code below and tell me what's wrong with it? It creates a string dataset and attaches a string array attribute. But when I read the attribute value back, I get only garbage. I also see garbage with HDFView, so the bug is in writing, not reading. When I use integer data instead of strings it works fine, BTW.

Thanks in advance,
   Konrad.
--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Moléculaire, CNRS Orléans
Synchrotron Soleil - Division Expériences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: research AT khinsen DOT fastmail DOT net
---------------------------------------------------------------------

import ncsa.hdf.object.*; // the common object package
import ncsa.hdf.object.h5.*; // the HDF5 implementation
import java.util.List;

public class StringAttributeTest
{
     private static String fname = "StringAttributeTest.h5";

     public static void main( String args[] ) throws Exception
     {
         FileFormat fileFormat = FileFormat.getFileFormat(FileFormat.FILE_TYPE_HDF5);
         if (fileFormat == null)
         {
             System.err.println("Cannot find HDF5 FileFormat.");
             return;
         }

         H5File testFile = (H5File)fileFormat.create(fname);
         if (testFile == null)
         {
             System.err.println("Failed to create file:"+fname);
             return;
         }

         testFile.open();
         Group root = (Group)((javax.swing.tree.DefaultMutableTreeNode)testFile.getRootNode()).getUserObject();

         String[] data = {"abc de fghi jk lmn op qrst uvw xyz\n0 12 345 6 789"};
         long[] data_dims = {1};
         Datatype dtype = testFile.createDatatype(
             Datatype.CLASS_STRING, data[0].length(),
             Datatype.NATIVE, Datatype.NATIVE);
         Dataset dataset = testFile.createScalarDS
             ("text", root, dtype, data_dims, null, null, 0, data);

         long[] attr_dims = {3};
         String[] attr_value = {"a", "ab", "abc"};
         Datatype attr_dtype = testFile.createDatatype(
             Datatype.CLASS_STRING, 5,
             Datatype.NATIVE, Datatype.NATIVE);
         Attribute attr = new Attribute("foo", attr_dtype, attr_dims);
         attr.setValue(attr_value);
         dataset.writeMetadata(attr);

         testFile.close();

         // read attributes back
         testFile = (H5File)fileFormat.open(fname, FileFormat.READ);
         if (testFile == null)
         {
             System.err.println("Failed to open file:"+fname);
             return;
         }
         testFile.open();
         root = (Group)((javax.swing.tree.DefaultMutableTreeNode)testFile.getRootNode()).getUserObject();
         dataset = (Dataset)root.getMemberList().get(0);
         List attrList = dataset.getMetadata();
         attr = (Attribute)attrList.get(0);
         String[] value = (String [])attr.getValue();
         System.out.println( attr.toString() );
         System.out.println( value[0] );
         System.out.println( value[1] );
         System.out.println( value[2] );
         testFile.close();
     }

}

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Is there a similar trick for variable-length string data? If I set the string length in my example to -1, it crashes the JVM.

Konrad.

···

On 25 Mar, 2011, at 15:17 , Peter Cao wrote:

Directly writing attribute of array of strings is not supported. We may add this feature in future
release. The workaround is to convert it to array of bytes, e.g.
       byte[] bvalue = Dataset.stringToByte(attr_value, 5);
       attr.setValue(bvalue);

--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Moléculaire, CNRS Orléans
Synchrotron Soleil - Division Expériences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: research AT khinsen DOT fastmail DOT net
---------------------------------------------------------------------

Writing attribute of array of variable strings is not supported. The trick will not work.

Writing array of variable length data (strings, floats, ints, etc) is in our wish-list. We
need more resources to complete the work.

Thanks
--pc

···

On 3/25/2011 9:33 AM, Konrad Hinsen wrote:

On 25 Mar, 2011, at 15:17 , Peter Cao wrote:

Directly writing attribute of array of strings is not supported. We may add this feature in future
release. The workaround is to convert it to array of bytes, e.g.
        byte[] bvalue = Dataset.stringToByte(attr_value, 5);
        attr.setValue(bvalue);

Is there a similar trick for variable-length string data? If I set the string length in my example to -1, it crashes the JVM.

Konrad.
--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Moléculaire, CNRS Orléans
Synchrotron Soleil - Division Expériences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: research AT khinsen DOT fastmail DOT net
---------------------------------------------------------------------

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

I certainly understand that! In the meantime, it would be nice to get clear error messages ("not implemented yet") for missing features, or at least have a list of them somewhere in the documentation. Since I started my current project, I spent more than half of my time on analyzing strange behavior that was ultimately due to unimplemented features. I am seriously considering giving up on the HDF Object layer and use the low-level libraries directly, just to get rid of the bad surprises.

Konrad.

···

On 25 Mar, 2011, at 15:46 , Peter Cao wrote:

Writing attribute of array of variable strings is not supported. The trick will not work.

Writing array of variable length data (strings, floats, ints, etc) is in our wish-list. We
need more resources to complete the work.

--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Moléculaire, CNRS Orléans
Synchrotron Soleil - Division Expériences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: research AT khinsen DOT fastmail DOT net
---------------------------------------------------------------------

Konrad, you are absolutely right. It should not crash. We will look into that. thanks. --pc

···

On 3/25/2011 10:17 AM, Konrad Hinsen wrote:

On 25 Mar, 2011, at 15:46 , Peter Cao wrote:

Writing attribute of array of variable strings is not supported. The trick will not work.

Writing array of variable length data (strings, floats, ints, etc) is in our wish-list. We
need more resources to complete the work.

I certainly understand that! In the meantime, it would be nice to get clear error messages ("not implemented yet") for missing features, or at least have a list of them somewhere in the documentation. Since I started my current project, I spent more than half of my time on analyzing strange behavior that was ultimately due to unimplemented features. I am seriously considering giving up on the HDF Object layer and use the low-level libraries directly, just to get rid of the bad surprises.

Konrad.
--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Moléculaire, CNRS Orléans
Synchrotron Soleil - Division Expériences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: research AT khinsen DOT fastmail DOT net
---------------------------------------------------------------------

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org