Issues with basic Java HDF5 code

Hello all

I'm trying to use HDF5 in Java. I'm running the Sun JDK 1.6.0_01 on Windows XP SP2 and I'm using the library I downloaded here:

ftp://ftp.hdfgroup.org/HDF5/hdf-java/bin/win/hdf-java-2.3.jar

Following the examples here:

http://www.hdfgroup.org/hdf-java-html/hdf-object/use.html#examples

I wrote the following code:

package frags;

import javax.swing.tree.DefaultMutableTreeNode;

import ncsa.hdf.object.Dataset;
import ncsa.hdf.object.Datatype;
import ncsa.hdf.object.FileFormat;
import ncsa.hdf.object.Group;
import ncsa.hdf.object.h5.H5File;

public final class Frags {
   public static void main(final String[] args) throws Exception {
       FileFormat fileFormat =
FileFormat.getFileFormat(FileFormat.FILE_TYPE_HDF5);
       H5File file = (H5File) fileFormat.create("test.h5");
       file.open();
       Group root = (Group) ((DefaultMutableTreeNode)
file.getRootNode()).getUserObject();
       Group group1 = file.createGroup("a", root);
       Group group2 = file.createGroup("b", group1);
       Datatype dtype = file.createDatatype(Datatype.CLASS_FLOAT, 4,
Datatype.NATIVE, Datatype.NATIVE);
       long[] dims = {1, 1};
       Dataset dataset = file.createScalarDS("c", group2, dtype,
dims, null, null, 0, null);
       for (long i = 0; i < dims[1]; i++) {
           float[] data = new float[(int) dims[0]];
           for (int j = 0; j < data.length; j++) {
               data[j] = 1.0f + i * j + j;
           }
           dataset.write(data);
       }
       file.close();
   }
}

Depending on dims, this code either works, writes garbage, crashes the JVM or throws an exception. I read the files from MATLAB R2006b using the following code:

hdf5read('test.h5', 'a/b/c')

With dims={1, 1}, the Java code writes a valid file.

With dims={3, 1}, it writes a valid file containing [1 2 3].

With dims={1, 3}, it writes an invalid file containing [1; 0; 0] instead of [1; 2; 3].

With dims={2, 2}, it writes an invalid file containing [1 0; 3 0] instead of [1 2; 3 4].

With dims={100,100} the JVM crashes due to an access violation somewhere in the code contained in the jhdf5.dll library:

···

#
# An unexpected error has been detected by Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x383d0a4a, pid=2296, tid=3052
#
# Java VM: Java HotSpot(TM) Server VM (1.6.0_01-b06 mixed mode)
# Problematic frame:
# C [jhdf5.dll+0x90a4a]
#
# An error report file with more information is saved as hs_err_pid2296.log
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#

With dims={1000,1000} I get the following exception:

Exception in thread "main"
ncsa.hdf.hdf5lib.exceptions.HDF5LowLevelIOException: Write failed
       at ncsa.hdf.hdf5lib.H5.H5Dwrite(Native Method)
       at ncsa.hdf.hdf5lib.H5.H5Dwrite(H5.java:939)
       at ncsa.hdf.object.h5.H5ScalarDS.write(H5ScalarDS.java:436)
       at frags.Frags.main(Frags.java:27)

Am I doing something wrong? The obvious culprit is the write to the dataset, but what I'm doing seems to match what is done in this example:

http://www.hdfgroup.org/hdf-java-html/hdf-object/javaExample/H5DatasetRead.java.txt

Any help would be appreciated.

Regards,

Albert Strasheim

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.

Albert,

We think the problem is in your code, You are trying to write a whole dataset
with a smaller memory buffer. To correct the problem, you can either select
a subset and write the subset or write the whole dataset. The following code
demonstrates how to do so.

     public static void TestH5WriteFloats( final String filename) throws Exception
     {
         FileFormat fileFormat = FileFormat.getFileFormat(FileFormat. FILE_TYPE_HDF5 );
         H5File file = (H5File) fileFormat.create(filename);
         file.open();
         Group root = (Group) ((javax.swing.tree.DefaultMutableTreeNode) file.getRootNode()).getUserObject();
         Group group1 = file.createGroup( "a", root);
         Group group2 = file.createGroup( "b", group1);
         Datatype dtype = file.createDatatype(Datatype. CLASS_FLOAT , 4, Datatype. NATIVE, Datatype. NATIVE);

         // write a subset of the dataset
         long [] dims = {2, 3};
         Dataset dataset = file.createScalarDS( "c" , group2, dtype, dims, null , null , 0, null );
                 dataset.init();
         long [] count = dataset.getSelectedDims();
         float [] data = new float [( int ) (dims[0])];

         count[0] = dims[0];
         count[1] =1;
                 for (long i = 0; i < dims[1]; i++) {
             for (int j = 0; j < data. length; j++) {
                 data[j] = 1.0f + i * j + j;
             }
             dataset.write(data);
         }

         /* write the whole dataset
         long[] dims = {2, 3};
         float[] data = new float[(int) (dims[0]*dims[1])];
         Dataset dataset = file.createScalarDS("c", group2, dtype, dims, null, null, 0, null);

         for (int i = 0; i < data.length; i++) {
             data[i] = 1.0f + i * 10;
         }
         dataset.write(data);
         */
                 file.close();
     }

···

At 1:40 AM +0200 8/21/07, Albert Strasheim wrote:

Hello all

I'm trying to use HDF5 in Java. I'm running the Sun JDK 1.6.0_01 on Windows XP SP2 and I'm using the library I downloaded here:

ftp://ftp.hdfgroup.org/HDF5/hdf-java/bin/win/hdf-java-2.3.jar

Following the examples here:

http://www.hdfgroup.org/hdf-java-html/hdf-object/use.html#examples

I wrote the following code:

package frags;

import javax.swing.tree.DefaultMutableTreeNode;

import ncsa.hdf.object.Dataset;
import ncsa.hdf.object.Datatype;
import ncsa.hdf.object.FileFormat;
import ncsa.hdf.object.Group;
import ncsa.hdf.object.h5.H5File;

public final class Frags {
  public static void main(final String[] args) throws Exception {
      FileFormat fileFormat =
FileFormat.getFileFormat(FileFormat.FILE_TYPE_HDF5);
      H5File file = (H5File) fileFormat.create("test.h5");
      file.open();
      Group root = (Group) ((DefaultMutableTreeNode)
file.getRootNode()).getUserObject();
      Group group1 = file.createGroup("a", root);
      Group group2 = file.createGroup("b", group1);
      Datatype dtype = file.createDatatype(Datatype.CLASS_FLOAT, 4,
Datatype.NATIVE, Datatype.NATIVE);
      long[] dims = {1, 1};
      Dataset dataset = file.createScalarDS("c", group2, dtype,
dims, null, null, 0, null);
      for (long i = 0; i < dims[1]; i++) {
          float[] data = new float[(int) dims[0]];
          for (int j = 0; j < data.length; j++) {
              data[j] = 1.0f + i * j + j;
          }
          dataset.write(data);
      }
      file.close();
  }
}

Depending on dims, this code either works, writes garbage, crashes the JVM or throws an exception. I read the files from MATLAB R2006b using the following code:

hdf5read('test.h5', 'a/b/c')

With dims={1, 1}, the Java code writes a valid file.

With dims={3, 1}, it writes a valid file containing [1 2 3].

With dims={1, 3}, it writes an invalid file containing [1; 0; 0] instead of [1; 2; 3].

With dims={2, 2}, it writes an invalid file containing [1 0; 3 0] instead of [1 2; 3 4].

With dims={100,100} the JVM crashes due to an access violation somewhere in the code contained in the jhdf5.dll library:

#
# An unexpected error has been detected by Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x383d0a4a, pid=2296, tid=3052
#
# Java VM: Java HotSpot(TM) Server VM (1.6.0_01-b06 mixed mode)
# Problematic frame:
# C [jhdf5.dll+0x90a4a]
#
# An error report file with more information is saved as hs_err_pid2296.log
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#

With dims={1000,1000} I get the following exception:

Exception in thread "main"
ncsa.hdf.hdf5lib.exceptions.HDF5LowLevelIOException: Write failed
      at ncsa.hdf.hdf5lib.H5.H5Dwrite(Native Method)
      at ncsa.hdf.hdf5lib.H5.H5Dwrite(H5.java:939)
      at ncsa.hdf.object.h5.H5ScalarDS.write(H5ScalarDS.java:436)
      at frags.Frags.main(Frags.java:27)

Am I doing something wrong? The obvious culprit is the write to the dataset, but what I'm doing seems to match what is done in this example:

http://www.hdfgroup.org/hdf-java-html/hdf-object/javaExample/H5DatasetRead.java.txt

Any help would be appreciated.

Regards,

Albert Strasheim

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.

--

------------------------------------------------------------
Elena Pourmal
The HDF Group
1901 So First ST.
Suite C-2
Champaign, IL 61820

epourmal@hdfgroup.org
(217)333-0238 (office)
(217)333-9049 (fax)
------------------------------------------------------------