Missing Constant in HDF5Constants in HDFView 3.0.0?


#1

Based on the this answer in a different thread by @dave.allured I wanted to try setting the library version boundaries in my Java HDF5 writer using

((H5File)file).setLibBounds(
    HDF5Constants.H5F_LIBVER_EARLIEST
   ,HDF5Constants.H5F_LIBVER_V18
);

However, H5F_LIBVER_V18 is not defined in the version of HDF5Constants.java that comes with the binaries of HDFView3.0.0, despite being in HDF5Constants.java of HDF5\1.10.4\src\java\src\hdf\hdf5lib\.

Am I missing something here? Shouldn’t HDF5Constants.H5F_LIBVER_V18 and HDF5Constants.H5F_LIBVER_V110 be accessible from the HDFView.jar library that comes with HDFView3.0.0?

By the way, there is a constant H5File.LIBVER_V18 according to the API.


Edit:

Also, when I set

((H5File)file).setLibBounds(
    HDF5Constants.H5F_LIBVER_EARLIEST
   ,HDF5Constants.H5F_LIBVER_EARLIEST
);

I still get a HDF5 1.10.X file:

System.out.println("test: " + file.getLibversion());

gives:

test: HDF5 1.10.1

I would expect that to be 1.8.X.


#2

HDF5Constants.H5F_LIBVER_* are from the hdf5 java library, which would need to be imported.
HDFView imports those constants into H5File as H5File.LIBVER_*.

Allen


#3

Hello @byrn, could you tell me explicitely what I have to import? My current setting is import:

  • HDFView.jar
  • slf4j-simple-1.7.5.jar

and setting -Djava.library.path="[MYPATH]\HDF_Group\HDFView\3.0.0\lib\". This way, I do find

HDF5Constants.H5F_LIBVER_EARLIEST
HDF5Constants.H5F_LIBVER_LATEST

but not

HDF5Constants.H5F_LIBVER_V18
HDF5Constants.H5F_LIBVER_V110

For the latter two, I get Cannot find symbol.


#4

The constants are imported into the hdf/object/H5/H5File class (you could just use the object library if you don’t need the visual classes from HDFView) as follows:
public static final int LIBVER_LATEST = HDF5Constants.H5F_LIBVER_LATEST;
public static final int LIBVER_EARLIEST = HDF5Constants.H5F_LIBVER_EARLIEST;
public static final int LIBVER_V18 = HDF5Constants.H5F_LIBVER_V18;
public static final int LIBVER_V110 = HDF5Constants.H5F_LIBVER_V110;

NOTE that all four constants are available in HDF5Constants from the hdf5 library jar file:
public static final int H5F_LIBVER_ERROR = H5F_LIBVER_ERROR();
public static final int H5F_LIBVER_EARLIEST = H5F_LIBVER_EARLIEST();
public static final int H5F_LIBVER_V18 = H5F_LIBVER_V18();
public static final int H5F_LIBVER_V110 = H5F_LIBVER_V110();
public static final int H5F_LIBVER_NBOUNDS = H5F_LIBVER_NBOUNDS();
public static final int H5F_LIBVER_LATEST = H5F_LIBVER_LATEST();

These lines are from the released versions of HDFView3.0 (hdfoject.jar or HDFView.jar) and 1.10.4 (jarhdf5-1.10.4.jar).

I would double check paths and env settings, because everything I checked here indicates the constants should be available for you.

Allen


#5

Ok, I gave it another shot with HDFView 3.1.0. I figured some stuff out. I guess I mixed up something between the object package from HDFView and the Java stuff inside HDF5 itself. Unfortunately, most of the documentation is on the “old” JNI interface and not for the object package. It is really unfortunate that there are no examples in HDFView repository how to use the object package. Instead, the Java examples in the HDF5 repository are for the non-object package way.

My problem with HDF5Constants is resolved if you know that the HDF5Constants.java from HDF5 can be found in H5File, FileFormat.java and so on.

However, I still get an error. Here is what I did. I installed HDFView 3.1.0 to D:\Programme\HDF_Group\HDFView\3.1.0. From the lib folder in there, I than installed jarhdf5-1.10.5.jar and hdfobject.jar to my local Maven repository:

mvn install:install-file -Dfile=jarhdf5-1.10.5.jar -DgroupId=org.hdfgroup -DartifactId=jarhdf5 -Dversion=1.10.5 -Dpackaging=jar
mvn install:install-file -Dfile=hdfobject.jar -DgroupId=org.hdfgroup -DartifactId=hdfobject -Dversion=1.10.5 -Dpackaging=jar

I than created a small test project just to create a HDF5 file

package org.test.hdf5objecttest;

import hdf.object.FileFormat;
import hdf.object.h5.H5File;

public class HDF5ObjectTest {
    
    public static void main(String args[]) {
        
        H5File file = new H5File(
            "test.h5"
           ,FileFormat.WRITE
        );
    }
}

with my pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.test</groupId>
    <artifactId>HDF5ObjectTest</artifactId>
    <version>0.0.0.1</version>
    <packaging>jar</packaging>
    <dependencies>
        <dependency>
            <groupId>org.hdfgroup</groupId>
            <artifactId>hdfobject</artifactId>
            <version>1.10.5</version>
            <type>jar</type>
        </dependency>
        <dependency>
            <groupId>org.hdfgroup</groupId>
            <artifactId>jarhdf5</artifactId>
            <version>1.10.5</version>
            <type>jar</type>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>1.7.25</version>
        </dependency>
    </dependencies>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>12</maven.compiler.source>
        <maven.compiler.target>12</maven.compiler.target>
    </properties>
</project>

I also added -Djava.library.path=D:\Programme\HDF_Group\HDFView\3.1.0\lib to the VM options just to be on the safe side. I can build the test program, However, on execution I get

Exception in thread "main" java.lang.NoClassDefFoundError: hdf/hdflib/HDFException
    at java.base/java.lang.Class.forName0(Native Method)
    at java.base/java.lang.Class.forName(Class.java:332)
    at hdf.object.FileFormat.<clinit>(FileFormat.java:208)
at org.test.hdf5objecttest.HDF5ObjectTest.main(HDF5ObjectTest.java:10)
Caused by: java.lang.ClassNotFoundException: hdf.hdflib.HDFException
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    ... 5 more
Command execution failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
    ...

and I simply can not find a HDFException class in any of the repositories. HDF5Exception yes, HDFExpection no. I’ll attach the sample problem. I really am on the verge of giving up. maybe someone has a short hint and can point out what I am doing wring. Any clue would be much appreciated.

HDF5ObjectTest.rar (1.8 KB)


#6

Hi Martin,

There are object package examples accessible from the support portal, here:

https://portal.hdfgroup.org/display/HDF5/Examples+by+API

-Barbara


#7

Hello and thanks @bljones. Despite the header information This file is intended for use with HDF5 Library version 1.6 are these still valid examples?

I took the file creation part from this example, but still get the same error java.lang.ClassNotFoundException: hdf.hdflib.HDFException.


#8

Hi Martin,

I tried a couple of examples that had that comment (not all of them do) and I was able to build it with no problems. (I had to comment out the “package examples.datasets” at the top.)

I’ll enter a bug report to clean up and test these examples and include them with the built HDFView binaries.

Thanks!
-Barbara


#9

Hi @bljones, I took this example , and just ran the code having the configuration from my previous post. I still get

Exception in thread "main" java.lang.NoClassDefFoundError: hdf/hdflib/HDFException
    at java.base/java.lang.Class.forName0(Native Method)
    at java.base/java.lang.Class.forName(Class.java:332)
    at hdf.object.FileFormat.<clinit>(FileFormat.java:208)
    at org.test.hdf5objecttest.H5ObjectEx_G_Create.CreateGroup(H5ObjectEx_G_Create.java:25)
    at org.test.hdf5objecttest.HDF5ObjectTest.main(HDF5ObjectTest.java:31)
Caused by: java.lang.ClassNotFoundException: hdf.hdflib.HDFException
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    ... 5 more
Command execution failed.

So something must be wrong with my configuration if the code runs for you with the latest HDFView binary. Could you maybe check if there is something obviously wrong with my config from this post. I loose the ideas, what I can do differently.

I also tried importing the HDFView.jar instead of hdfobject.jar and installed it to maven, as before with

mvn install:install-file -Dfile=HDFView.jar -DgroupId=org.hdfgroup -DartifactId=HDFView -Dversion=1.10.5 -Dpackaging=jar

and replaced the reference to hdfobject.jar in the pom.xml. So now I have

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>org.test</groupId>
    <artifactId>HDF5ObjectTest</artifactId>
    <version>0.0.0.1</version>
    <packaging>jar</packaging>
    <dependencies>
        <dependency>
            <groupId>org.hdfgroup</groupId>
            <!--<artifactId>hdfobject</artifactId>-->
            <artifactId>HDFView</artifactId>
            <version>1.10.5</version>
            <type>jar</type>
        </dependency>
        <dependency>
            <groupId>org.hdfgroup</groupId>
            <artifactId>jarhdf5</artifactId>
            <version>1.10.5</version>
            <type>jar</type>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>1.7.25</version>
        </dependency>
    </dependencies>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>12</maven.compiler.source>
        <maven.compiler.target>12</maven.compiler.target>
    </properties>
</project>

All other preferences stay the same. This way, I get another error:

java.lang.UnsatisfiedLinkError: D:\Programme\HDF_Group\HDFView\3.1.0\lib\hdf5_java.dll: Die angegebene Prozedur wurde nicht gefunden
    at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
    at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2430)
    at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2487)
    at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2684)
    at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2649)
    at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:827)
    at java.base/java.lang.System.loadLibrary(System.java:1902)
    at hdf.hdf5lib.H5.loadH5Lib(H5.java:313)
    at hdf.hdf5lib.H5.<clinit>(H5.java:253)
    at hdf.hdf5lib.HDF5Constants.<clinit>(HDF5Constants.java:29)
    at hdf.object.h5.H5File.<init>(H5File.java:94)
    at hdf.object.h5.H5File.<init>(H5File.java:127)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
    at java.base/java.lang.reflect.ReflectAccess.newInstance(ReflectAccess.java:166)
    at java.base/jdk.internal.reflect.ReflectionFactory.newInstance(ReflectionFactory.java:404)
    at java.base/java.lang.Class.newInstance(Class.java:590)
    at hdf.object.FileFormat.<clinit>(FileFormat.java:225)
    at org.test.hdf5objecttest.H5ObjectEx_G_Create.CreateGroup(H5ObjectEx_G_Create.java:25)
    at org.test.hdf5objecttest.HDF5ObjectTest.main(HDF5ObjectTest.java:31)

The first statement basically says “The specified procedure was not found.”. So I guess there is still something wrong with the way I include the dll’s


#10

Hi Martin,

It looks like it can’t find the HDF5 library… (jarhdf5-1.10.5.jar).
Here is what I did:

  • Installed HDFView
  • Moved slf4j-simple-1.7.25.jar out of the lib/ directory (due to duplicate variables found)
  • Created a script (runlinobj.sh) that sets the Java path and classpath.
  • Edited H5ObjectEx_G_Create.java and commented out the line “package examples.groups;”
    near the top.
  • Typed “./runlinobj.sh H5ObjectEx_G_Create”

Attached are the runlinobj.sh script and the modified H5ObjectEx_G_Create.java files.
If you have HDFView installed, then edit the script to point to the HDFView home location (HDFJAVA_HOME) and edit JAVAPATH for the location of java.

runlinobj.sh (387 Bytes)
H5ObjectEx_G_Create.java (1.9 KB)


#11

@bljones thanks. I am on a windows machine (Windows 7), but I basically do the same. I

  • installed the binaries of HDFView from the HDFGroup page. to D:\Programme\HDF_Group\HDFView\3.1.0\

  • I addad D:\Programme\HDF_Group\HDFView\3.1.0\lib\ to the windows PATH variable

  • I manually installed hdfobject.jar, jarhdf5-1.10.5.jar to my local maven repository, so I can add the dependency in the pom.xml as stated above

  • I added the dependencies to hdfobject.jar, jarhdf5-1.10.5.jar to the pom.xml, so that the code is able to find and resolve

    import hdf.object.FileFormat;
    import hdf.object.h5.H5File;
    
  • Build/Compiled without error

  • When I run the application with -Djava.library.path=D:\Programme\HDF_Group\HDFView\3.1.0\lib, the exception occurs

The funny thing is, if I do exactly the same as above but for HDFView 2.13.0, everything works and I do not get an error on execution. In that case I use HDFView.jar and jarhdf5-3.2.1.jar. But I would really appreciate using a current version.


#12

@bljones Next step in my quest to get a current version of HDF5 working in my tool. One question: I currently do not have any dependency to a HDF5 installation, just like you don’t in the example you provided. It is all based on the HDFView installation.

However, when looking at the error message I get, the problem, as you mentioned, is loading the library files. The error cause basically is this line from the error message:

java.lang.UnsatisfiedLinkError: D:\Programme\HDF_Group\HDFView\3.1.0\lib\hdf5_java.dll: Die angegebene Prozedur wurde nicht gefunden
...
at hdf.hdf5lib.H5.loadH5Lib(H5.java:313)

So I had a look at line 313 of H5.java from here. For the class to reach line 313, the boolean variable !isLibraryLoaded must be false after the following steps:

// first try loading library by name from user supplied library path
...

// else try loading library via full path
...

So this means, that neither System.getProperty(H5_LIBRARY_NAME_PROPERTY_KEY, null); nor System.getProperty(H5PATH_PROPERTY_KEY, null);, which are basically System.getProperty("hdf.hdf5lib.H5.loadLibraryName", null); and System.getProperty("hdf.hdf5lib.H5.hdf5lib", null); gives a valid result.

When I test in my tool for

System.getProperty("hdf.hdf5lib.H5.loadLibraryName", null);
System.getProperty("hdf.hdf5lib.H5.hdf5lib", null);

I get null for both.

Should these be set during the installation of HDFView?

If I do the same for HDFView 2.13.0, I also get null. However, I can still create the HDF5 file.


#13

HI Martin,

Attached is a batch file, compile.bat, that we managed to get working to compile a java object application on Windows. It expects to be run from the top HDFView installation directory. Can you try it to see if you can get it to work for you?

-Barbara

compile.bat (1.9 KB)


#14

@bljones thanks, I took the batch script and put it in my HDFView install folder D:\Programme\HDF_Group\HDFView\3.1.0\. I copied the H5ObjectEx_G_Create.java example from your answer to the same folder and simply executed the batch script

compile.bat H5ObjectEx_G_Create

This gives me the following error during execution of the line "%JAVABIN%\\java.exe" -cp "%INSTALLDIR%\\lib\\*; ...:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

This is usually resolved by adding the dependency to slf4j-simple or slf4j-api to my java project. However, it seems to not find it in the library folder. But it is there, I did not make any changes to the installation folder of HDFView. -cp should set the classpath. When I echo "%INSTALLDIR%\\lib\\*;%INSTALLDIR%" and -Djava.library.path="%INSTALLDIR%\\lib" I get

"d:\Programme\HDF_Group\HDFView\3.1.0\\lib\\*;d:\Programme\HDF_Group\HDFView\3.1.0"
-Djava.library.path="d:\Programme\HDF_Group\HDFView\3.1.0\\lib"

When I enter d:\Programme\HDF_Group\HDFView\3.1.0\\lib\\ into windows explorer I get an error that the path does not exist. However, when replacing all \\ with \ in the batch script I still get the same error. But that also should’t matter as windows ignores the second backslash anyways, except in the beginning of the path.

I checked the privileges. I do have read and execution rights on the whole HDFView install folder. I also tried with my admin account. Same result.

I noticed you use OpenJDK. I currently still am on Java SE JDK. But I do not suppose this should be a problem, especially as you deliver the JRE and the script executes your jre tools.


Ok, I got it working. I had to add %INSTALLDIR%\\lib\\extra\\slf4j-simple-1.7.25.jar to the CLASSPATH. The execution command now looks like this:

"%JAVABIN%\\java.exe" -cp "%INSTALLDIR%\\lib\\extra\\slf4j-simple-1.7.25.jar;%INSTALLDIR%\\lib\\*;%INSTALLDIR%" -Djava.library.path="%INSTALLDIR%\\lib"  "%1%" dummy.h5

Ok, I finally came a step forward. The difference between your batch script and executing my code from within the build process in my IDE is setting the classpath -cp.

So I wrote a very simple script

package org.test.hdf5objecttest;

import hdf.object.FileFormat;
import hdf.object.h5.H5File;

public class HDF5ObjectTest {
    
    public static void main(String args[]) throws Exception {
        
        try {
            System.load("D:/Programme/HDF_Group/HDFView/3.1.0/lib/hdf5_java.dll");
        } catch (UnsatisfiedLinkError e) {
            System.err.println("Native code library failed to load.\n" + e);
            System.exit(1);
        }        
        
        FileFormat fileFormat = FileFormat.getFileFormat(FileFormat.FILE_TYPE_HDF5);
        H5File file = (H5File) fileFormat.createFile("test.h5", FileFormat.FILE_CREATE_DELETE);
        System.out.println("file: " + file);
        
    }
}

This gives the same error as above java.lang.UnsatisfiedLinkError: D:\Programme\HDF_Group\HDFView\3.1.0\lib\hdf5_java.dll: The specified procedure could not be found. So I image a procedure or call inside hdf5_java.dll is missing. So, I first checked if there are other versions of the library in the PATH. I had a version of Visit and netCDF that came with an old version of the hdf5 libraries. I removed them from the PATH and got:

>where hdf5.dll
D:\Programme\HDF_Group\HDFView\3.1.0\lib\hdf5.dll

>where hdf5_java.dll
D:\Programme\HDF_Group\HDFView\3.1.0\lib\hdf5_java.dll

As there where no more possible confilicts, I tried loading libraries to find the ones that are missing. And loading hdf5.dll did the trick.

package org.test.hdf5objecttest;

import hdf.object.FileFormat;
import hdf.object.h5.H5File;

public class HDF5ObjectTest {
    
    public static void main(String args[]) throws Exception {
        
        // WITHOUT THIS LOADING hdf_java.dll GIVES AN UnsatisfiedLinkError
        try {
            System.load("D:/Programme/HDF_Group/HDFView/3.1.0/lib/hdf5.dll");
        } catch (UnsatisfiedLinkError e) {
            System.err.println("Native code library failed to load.\n" + e);
            System.exit(1);
        }
        
        try {
            System.load("D:/Programme/HDF_Group/HDFView/3.1.0/lib/hdf5_java.dll");
        } catch (UnsatisfiedLinkError e) {
            System.err.println("Native code library failed to load.\n" + e);
            System.exit(1);
        }        
        
        FileFormat fileFormat = FileFormat.getFileFormat(FileFormat.FILE_TYPE_HDF5);
        H5File file = (H5File) fileFormat.createFile("test.h5", FileFormat.FILE_CREATE_DELETE);
        System.out.println("file: " + file);
        
    }
}

Now I ask myself, isn’t that something that should happen in the H5.java loadH5Lib(); method?


#15

Hi @martin.raedel,

perhaps I’m misunderstanding the issue you’re running into, but loadH5Lib() already has code in place to load the hdf5_java library. It shouldn’t be necessary to explicitly load the hdf5 library. The system properties you previously mentioned (H5_LIBRARY_NAME_PROPERTY_KEY and H5PATH_PROPERTY_KEY) are “advanced” options for allowing a user to directly specify the path/name of the HDF5 library to load, but if they are not set H5.java defaults to our standard convention for the name of the library (hdf5_java) and attempts to find it in java.library.path.

As for the missing symbols issue, it’s hard to guess at what’s going on without seeing what procedure appears to be missing to the linker, but I would first assume that hdf5_java.dll/so/dylib is not correctly linked to the main HDF5 library and so is not pulling in the API.


#16

Hi @jhenderson,

the problem is not hdf5_java.dll. I need to load hdf5.dll manually for my code to work, despite the fact, that both dll are located in the same directory and are now both found in the PATH variable.


#17

Indeed, see

It shouldn’t be necessary to explicitly load the hdf5 library.

from above. I’m going to guess that as part of packaging HDFView the linking to the main HDF5 library for hdf5_java may not have been preserved and is causing the issues that you are running into. We need to investigate this more to see if we encounter the same issues running the Java Object Package examples against the current HDFView binaries.


#18

@jhenderson Great, thx. I am looking forward to the result.