thread safety binaries?

Hdf-forum archives
1

Hello- I am trying to use HDF5 in a multi-threaded app and running into many issues.

1) Are winx86/x64 binaries available that have been built with the H5_HAVE_THREADSAFE compile flags?

2) Any gotchas/tips for using hdf5 in a multi-threaded app?

3) Any reason the "threadsafe" binaries would not work with the dot net wrappers?

Thanks in advance!

Warm Regards,
Jim

2

Hi Jim,

···

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Rowe, Jim
Sent: Monday, February 03, 2014 12:06 PM
To: hdf-forum@lists.hdfgroup.org
Subject: [Hdf-forum] thread safety binaries?

Hello- I am trying to use HDF5 in a multi-threaded app and running into many issues.

1) Are winx86/x64 binaries available that have been built with the H5_HAVE_THREADSAFE compile flags?

[Dana Robinson]
We do not supply thread-safe binaries. You'll have to build them yourself, which is not that difficult to do with CMake + Visual Studio.

2) Any gotchas/tips for using hdf5 in a multi-threaded app?

[Dana Robinson]
You may have some performance degradation due to the giant mutex we use to protect the library. There's also a small memory leak in the Windows version due to some thread-local storage not being freed. I can send you a patch for that and the fix will be in the May HDF5 1.8.13 release.

Other than that, the thread-safe library behaves identically to the regular library.

3) Any reason the "threadsafe" binaries would not work with the dot net wrappers?

[Dana Robinson]
I would imagine that it would work, but I have not tried this. The .NET wrappers simply invoke the HDF5 API calls, so they don't do anything complicated, and .NET uses Win32 threads under the hood so I'd imagine that our thread-local storage wouldn't have issues.

Cheers,

Dana

Thanks in advance!

Warm Regards,
Jim

3

Hello Dana, thank you for the response. The patch you mention would be great (j.rowe_AT_questintegrity.com).

Also, I'm not sure if you can help, but I am having problems building hdf5 with zlib. I am using cmake and have added

HDF5_ALLOW_EXTERNAL_SUPPORT = TGZ
TGZPATH = {my path to Zlib.tar.gz}
ZLIB_TGZ_NAME = Zlib.tar.gz
ZLIB_USE_EXTERNAL = true

CMake Error at CMakeFilters.cmake:66 (MESSAGE):
ZLib is Required for ZLib support in HDF5
Call Stack (most recent call first):
CMakeLists.txt:688 (INCLUDE)

Looking at the script it appears to be unable to find my zlib package.

Warm Regards,
Jim

···

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Dana Robinson
Sent: Monday, February 3, 2014 10:40 AM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] thread safety binaries?

Hi Jim,

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Rowe, Jim
Sent: Monday, February 03, 2014 12:06 PM
To: hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>
Subject: [Hdf-forum] thread safety binaries?

Hello- I am trying to use HDF5 in a multi-threaded app and running into many issues.

1) Are winx86/x64 binaries available that have been built with the H5_HAVE_THREADSAFE compile flags?

[Dana Robinson]
We do not supply thread-safe binaries. You'll have to build them yourself, which is not that difficult to do with CMake + Visual Studio.

2) Any gotchas/tips for using hdf5 in a multi-threaded app?

[Dana Robinson]
You may have some performance degradation due to the giant mutex we use to protect the library. There's also a small memory leak in the Windows version due to some thread-local storage not being freed. I can send you a patch for that and the fix will be in the May HDF5 1.8.13 release.

Other than that, the thread-safe library behaves identically to the regular library.

3) Any reason the "threadsafe" binaries would not work with the dot net wrappers?

[Dana Robinson]
I would imagine that it would work, but I have not tried this. The .NET wrappers simply invoke the HDF5 API calls, so they don't do anything complicated, and .NET uses Win32 threads under the hood so I'd imagine that our thread-local storage wouldn't have issues.

Cheers,

Dana

Thanks in advance!

Warm Regards,
Jim

4

Jim,

   Line 66 didn't recognize your HDF5_ALLOW_EXTERNAL_SUPPORT define, try:
-DHDF5_ALLOW_EXTERNAL_SUPPORT:STRING="TGZ"

Allen

···

On Tuesday, February 04, 2014 04:04:37 PM Rowe, Jim wrote:

Hello Dana, thank you for the response. The patch you mention would be
great (j.rowe_AT_questintegrity.com).

Also, I'm not sure if you can help, but I am having problems building hdf5
with zlib. I am using cmake and have added

HDF5_ALLOW_EXTERNAL_SUPPORT = TGZ
TGZPATH = {my path to Zlib.tar.gz}
ZLIB_TGZ_NAME = Zlib.tar.gz
ZLIB_USE_EXTERNAL = true

CMake Error at CMakeFilters.cmake:66 (MESSAGE):
ZLib is Required for ZLib support in HDF5
Call Stack (most recent call first):
CMakeLists.txt:688 (INCLUDE)

Looking at the script it appears to be unable to find my zlib package.

Warm Regards,
Jim

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of
Dana Robinson Sent: Monday, February 3, 2014 10:40 AM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] thread safety binaries?

Hi Jim,

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of
Rowe, Jim Sent: Monday, February 03, 2014 12:06 PM
To: hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>
Subject: [Hdf-forum] thread safety binaries?

Hello- I am trying to use HDF5 in a multi-threaded app and running into many
issues.

1) Are winx86/x64 binaries available that have been built with the
H5_HAVE_THREADSAFE compile flags?

[Dana Robinson]
We do not supply thread-safe binaries. You'll have to build them yourself,
which is not that difficult to do with CMake + Visual Studio.

2) Any gotchas/tips for using hdf5 in a multi-threaded app?

[Dana Robinson]
You may have some performance degradation due to the giant mutex we use to
protect the library. There's also a small memory leak in the Windows
version due to some thread-local storage not being freed. I can send you a
patch for that and the fix will be in the May HDF5 1.8.13 release.

Other than that, the thread-safe library behaves identically to the regular
library.

3) Any reason the "threadsafe" binaries would not work with the dot net
wrappers?

[Dana Robinson]
I would imagine that it would work, but I have not tried this. The .NET
wrappers simply invoke the HDF5 API calls, so they don't do anything
complicated, and .NET uses Win32 threads under the hood so I'd imagine that
our thread-local storage wouldn't have issues.

Cheers,

Dana

Thanks in advance!

Warm Regards,
Jim