High Level & Thread Safe

HDF5 Library
1

I’m seeing what it would take to upgrade my HDF implementation and I’m running into issues.

When I run hdf5-1.10.1 through CMake it complains that HL and thread-safety options are not supported. When did this happen? I use both.

I found this discussion from July 2016 on a netcdf group saying that they are mutually exclusive.
https://www.unidata.ucar.edu/mailing_lists/archives/netcdfgroup/2016/msg00223.html

Trying both alone in Cmake seems to work.

I use Packet Tables extensively and Dimension Scales (less so). And I do have multiple threads accessing multiple independent files (on Windows). Can I live without HDF’s thread safety? Or is there a work around?

Are the HL interfaces mostly an afterthought these days? They never were supported in the old HDF5DotNet wrappers and weren’t in the PInvoke implementation when I last looked.

Scott

2

The reason the high-level library is not considered thread-safe is because the global library lock only exists in the C library and is not hoisted up into the HL library. The thread-safe + high-level library combination was not explicitly marked as unsafe and unsupported until a few years ago. You can override this setting in CMake by setting ENABLE_UNSUPPORTED (edit: ALLOW_UNSUPPORTED) to ON or configuring with --enable-unsupported in the autotools.

This may work for you or you may encounter problems; we simply haven’t had the internal resources to investigate this properly. Making thread-safety and the high-level library mutually exclusive came about when we were cleaning up the build systems and that combination (as well as the language wrappers) was flagged as problematic and not supported, not because we know for sure that there are problems.

We’ve talked about fixing this for some time, but unfortunately it’s never made it to the front of our queue. If anyone cares deeply about this and wants to fund the work, that could quickly change.

Also, for what it’s worth, regardless of your file access patterns, the HDF5 C library contains global state that can mutate in even read-only cases. You are far, far better off building with thread-safety and the high-level library (even though it’s technically unsupported) than attempting to use the non-thread-safe library from multiple threads. In the former, you at least have the bulk of the code protected by the global lock.

3

Thanks Dana,

But, is ENABLE_UNSUPPORTED right? That doesn’t change a thing when I add it. I don’t see a reference to it in the files. I see a reference to ALLOW_UNSUPPORTED, but it doesn’t help either.

4

In 1.10.1 the CMake option is ALLOW_UNSUPPORTED.

5

Like I said, that didn’t work for me. And it keeps getting deleted whenever I run the CMake Configure.

6

Okay, can you provide your command line, maybe I might see something that I am not doing to repeat the issue.

Allen

7

I’m using the CMake GUI (3.11.0-0rc2) not the command line version. The following are on (Bold is something I specifically set):

BUILD_SHARED_LIBS, BUILD_TESTING, ALLOW_UNSUPPORTED (added manually), HDF5_BUILD_HL_LIB, HDF5_BUILD_TOOLS, HDF5_ENABLE_ALL_WARNINGS, HDF5_ENABLE_DEPRECATED_SYMBOLS, HDF5_ENABLE_EMBEDDED_LIBINFO, HDF5_ENABLE_SIZET, & HDF5_ENABLE_THREADSAFE.

All the other booleans that autopopulate are off. All the rest are defaults.

8

The GUI has an advanced option, that will display more options, ALLOW_UNSUPPORTED is one of those.
I would think that the manual entry is incorrectly setting the option.

Allen

9

Sorry for the typo, ALLOW_UNSUPPORTED is correct. A quick test on our development trunk (very close to 1.10.1) with ccmake shows that ALLOW_UNSUPPORTED, HDF5_BUILD_HL_LIB, and HDF5_ENABLE_THREADSAFE work correctly.

Allen might have more insight into what is failing.

10

That works, thank you.