HDF with java, what wrapper is supported/tested on Windows

HDF5 Library
1

Hi,
I’m using HDF5 in C++ and Python projects and my plan is to also switch to HDF in Java. In Python and C++, I spend about 30 minutes making the code work without any prior knowledge. In Java, however, I’ve already spent several hours, and the effort has been fruitless. So my question is: What Java wrapper is tested on Windows? Preferably prebuilt, but not necessarily.

I’ve already tested:

  • List item JHDF: works, but the dataset size limit is prohibitive for my application
  • List item wrappers in the official release: I ended up with hdf5_java.dll: The specified procedure could not be found
  • List item HDFQL: works, but after reading the dataset into a variable, the variable is filled with zeros even though the execute return code is 0.

Also, I’ve seen other Java wrappers:

I do not want to investigate/debug the individual wrappers in this thread. Rather, I’m seeking advice on which wrapper I should focus on cause I do not have the energy to debug multiple Java wrappers simultaneously.

Do you have any experience about which of the Java wrappers is reliable and works on Windows?

2

Hi @david.fido.fiedler,

Would you mind to post the code that you used to test HDFql in Java?

That said, HDFql could be a good candidate for your use-case given that it supports all the programming languages you mentioned (namely: C++, Python and Java).

Hope it helps!

3

We test the Java wrappers in the official release on a number of platforms, on Windows you do need to set the “PATH” to all the dlls in the stack (hdf_java.dll, hdf5.dll) and the interface is through the jarhdf5-1.14.3,jar file.
In the HDF5examples code here is one of the test command lines:
– COMMAND: /usr/bin/java; -Xmx1024M -Dorg.slf4j.simpleLogger.defaultLog=info -Djava.library.path="/HDF_Group/HDF5/1.14.3/lib" -cp ".:/HDF_Group/HDF5/1.14.3/lib/jarhdf5-1.14.3.jar:/HDF_Group/HDF5/1.14.3/lib/slf4j-api-2.0.6.jar:/HDF_Group/HDF5/1.14.3/lib/slf4j-nop-2.0.6.jar:/hdf5-examples/JAVA/H5G/exJ_H5Ex_G_Create.jar" ;-Dhdf.hdf5lib.H5.loadLibraryName=hdf5::hdf5_java; H5Ex_G_Create

4

Ok, let’s start with HDFQL. I created a new topic with the code in the HDFQL category.

BTW, for C++ and Python, I already use the wrappers included in package managers (vcpkg, pip), which work fine with little effort.

5

Blockquote on Windows you do need to set the “PATH” to all the dlls in the stack (hdf_java.dll, hdf5.dll) and the interface is through the jarhdf5-1.14.3,jar file.

I did exactly that, setting the library path to bin folder where both dlls are located and including the jar as a maven dependency. However, there is clearly some problem in the library loading process.

Regarding your suggested test command, there are no jars in the examples folder (/hdf5-examples/JAVA/H5G/exJ_H5Ex_G_Create.jar does not exist). Do I need to build and package the examples first?

6

I figured you had the path set correctly, just stated the obvious for future folks that happen upon this post. ;{)

Right that command was pulled from a build of the examples. (and edited out extra path info)
The important items are the classpath and library options to the java command. (one variation, there are other methods that may be more suitable). The slfj jars may not be needed.

7

Does the linux shellscript, junit.sh.in in the /java/test folder help any (around line 300 and the actual command line around 346)?

It has been awhile since I had been working at this level and with your (and others) help maybe I could get a howto use hdf5 java wrappers document in the repo.

8

Well, I can try to replicate the command and see if it works, but there are some placeholders in the shell script (@H5_JAVAFLAGS@), and I do not know how to get values for them, as I’ve never used such a setup (customizing shell scripts by values provided by build system)

9

I would ignore any external vars things should work with the basic command - I will be working on it locally as well.

10

Here is a command line (edited to shorten paths) from a recent build;
C:/jdk-19.0.2/bin/java.exe -Xmx1024M -Dorg.slf4j.simpleLogger.defaultLog=info -Djava.library.path="C:/bin/RelWithDebInfo" -cp ".;C:/lib/junit.jar;C:/lib/hamcrest-core.jar;C:/lib/jarhdf5-1.15.0.jar;C:/lib/slf4j-api-2.0.6.jar;C:/lib/ext/slf4j-simple-2.0.6.jar;C:/test/jartest5_TestH5D.jar" -Dhdf.hdf5lib.H5.loadLibraryName=hdf5_java;-ea;org.junit.runner.JUnitCore test.TestH5D

11

The problem of the example command is that I do not have the jartest5_TestH5D.jar file. It is not in the precompiled release package, and I was not able to build the project (some LINK error, probably SZIP, but the cmake setup & build documentation is too long to efficiently identify the problem).

12

jartest5_TestH5D.jar file and test.TestH5D would be replaced by your files.

13

I see, you want me to test my program using the command you posted here. This, of course, results in the very same error I reported before.

As it seems that there is no easy solution and we’ve already diverged from the initial question, I’ve started a new thread dedicated to this error.