Building Static Thread Safe Libs

Hdf-forum archives
1

Is there any way to build a the static libs with thread safety turned on?
I'm only using hdf5 in one binary so I'd rather not have to include an
additional dll in my distribution.

Thanks,
David

2

Hi David,

Since you say "dll" I'm going to assume you are on Windows. Thread-safety is not supported in static builds of HDF5 on Windows. The reason for this is that, unlike with Pthreads, we have to use the dll hooks to set up and tear down thread-local storage when Win32 threads are used. With no dll, there are no hooks.

Cheers,

Dana Robinson
Software Engineer
The HDF Group

···

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of David
Sent: Wednesday, November 4, 2015 3:49 PM
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
Subject: [Hdf-forum] Building Static Thread Safe Libs

Is there any way to build a the static libs with thread safety turned on? I'm only using hdf5 in one binary so I'd rather not have to include an additional dll in my distribution.

Thanks,
David

3

I'm actually building a dll so statically linked hdf5 would still able to
use dll hooks. I just don't know how to get the build procedure to do it.

···

On Wed, Nov 4, 2015 at 12:54 PM, Dana Robinson <derobins@hdfgroup.org> wrote:

Hi David,

Since you say "dll" I'm going to assume you are on Windows. Thread-safety
is not supported in static builds of HDF5 on Windows. The reason for this
is that, unlike with Pthreads, we have to use the dll hooks to set up and
tear down thread-local storage when Win32 threads are used. With no dll,
there are no hooks.

Cheers,

Dana Robinson

Software Engineer

The HDF Group

*From:* Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] *On
Behalf Of *David
*Sent:* Wednesday, November 4, 2015 3:49 PM
*To:* HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
*Subject:* [Hdf-forum] Building Static Thread Safe Libs

Is there any way to build a the static libs with thread safety turned on?
I'm only using hdf5 in one binary so I'd rather not have to include an
additional dll in my distribution.

Thanks,

David

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

4

It's not something that we've ever tested or would be likely to support, but essentially:

1) Convince CMake to build the unsupported thread-safe + static configuration. There should be an "enable unsupported" option for this.
2) Possibly modify the HDF5 source code since our #ifdefs might assume that thread-safety and shared builds are standard on Windows.
3) Call H5TS_win32_thread_enter/exit() in your own DllMain() code like we do in H5.c. Note that those are private functions, so you'll have to pull in our private headers.
4) If you plan on ever updating the version of HDF5 that we use, you'll need to hope we don't change the way we handle TLS in the future since you are using non-public-API functions.

I would definitely recommend against this, even if it can be made to work.

Cheers,

Dana

···

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of David
Sent: Wednesday, November 4, 2015 4:01 PM
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
Subject: Re: [Hdf-forum] Building Static Thread Safe Libs

I'm actually building a dll so statically linked hdf5 would still able to use dll hooks. I just don't know how to get the build procedure to do it.

On Wed, Nov 4, 2015 at 12:54 PM, Dana Robinson <derobins@hdfgroup.org<mailto:derobins@hdfgroup.org>> wrote:
Hi David,

Since you say "dll" I'm going to assume you are on Windows. Thread-safety is not supported in static builds of HDF5 on Windows. The reason for this is that, unlike with Pthreads, we have to use the dll hooks to set up and tear down thread-local storage when Win32 threads are used. With no dll, there are no hooks.

Cheers,

Dana Robinson
Software Engineer
The HDF Group

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org<mailto:hdf-forum-bounces@lists.hdfgroup.org>] On Behalf Of David
Sent: Wednesday, November 4, 2015 3:49 PM
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: [Hdf-forum] Building Static Thread Safe Libs

Is there any way to build a the static libs with thread safety turned on? I'm only using hdf5 in one binary so I'd rather not have to include an additional dll in my distribution.

Thanks,
David

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

5

The CMake code only has commands for shared - you could copy that block of
code over to the static library commands. (Edit as necessary)

Allen

It's not something that we've ever tested or would be likely to support, but
essentially:

1) Convince CMake to build the unsupported thread-safe + static
configuration. There should be an "enable unsupported" option for this.

2)

Possibly modify the HDF5 source code since our #ifdefs might assume that
thread-safety and shared builds are standard on Windows. 3) Call
H5TS_win32_thread_enter/exit() in your own DllMain() code like we do in
H5.c. Note that those are private functions, so you'll have to pull in our
private headers. 4) If you plan on ever updating the version of HDF5 that
we use, you'll need to hope we don't change the way we handle TLS in the
future since you are using non-public-API functions.
I would definitely recommend against this, even if it can be made to work.

Cheers,

Dana

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of
David
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
Subject: Re: [Hdf-forum] Building Static Thread Safe Libs

I'm actually building a dll so statically linked hdf5 would still able to
use dll hooks. I just don't know how to get the build procedure to do it.

Hi David,

···

On Wednesday, November 04, 2015 09:13:16 PM Dana Robinson wrote:
Sent: Wednesday, November 4, 2015 4:01 PM

On Wed, Nov 4, 2015 at 12:54 PM, Dana Robinson > <derobins@hdfgroup.org<mailto:derobins@hdfgroup.org>> wrote:

Since you say "dll" I'm going to assume you are on Windows. Thread-safety is
not supported in static builds of HDF5 on Windows. The reason for this is
that, unlike with Pthreads, we have to use the dll hooks to set up and tear
down thread-local storage when Win32 threads are used. With no dll, there
are no hooks.

Cheers,

Dana Robinson
Software Engineer
The HDF Group

From: Hdf-forum
[mailto:hdf-forum-bounces@lists.hdfgroup.org<mailto:hdf-forum-bounces@lists
.hdfgroup.org>] On Behalf Of David

Sent: Wednesday, November 4, 2015 3:49

PM
To: HDF Users Discussion List
<hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: [Hdf-forum] Building Static Thread Safe Libs

Is there any way to build a the static libs with thread safety turned on?
I'm only using hdf5 in one binary so I'd rather not have to include an
additional dll in my distribution.

Thanks,
David

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

6

Here's what I've done to get static threadsafe libs to build for use with
my own Windows DLL.

In HDF518config.cmake I turned on threadsafe:
set(ADD_BUILD_OPTIONS "${ADD_BUILD_OPTIONS}
-DHDF5_ENABLE_THREADSAFE:BOOL=ON")

Adding HDF5_ENABLE_UNSUPPORTED:BOOL=ON did not work for me (config fails)
so.....

In hdf5-1.8.15-patch1/CMakeLists.txt I commented out the THREADSAFE +
STATIC_LIB check:
    #if (H5_BUILT_AS_STATIC_LIB)
    # message (FATAL_ERROR " **** thread-safety option not supported with
static library **** ")
    #endif (H5_BUILT_AS_STATIC_LIB)

In hdf5-1.8.15-patch1/src/H5.c for THREADSAFE + STATIC_LIB and I enabled
and renamed DllMain to HDF5DllMain

In my own DLL I declare and call HDF5DllMain() from my DllMain()

This seems to be working.

Now I'm trying to formalize my build process which leads me to a few
questions:

1) Does what I did above seem like the a good (and minimal) way to get this
done.

2) To do release vs debug builds, do I need to change
CTEST_BUILD_CONFIGURATION in HDF518config.cmake and the -C option to cmake
(as seen in build-VS2013-32.bat) It would be nice to have a way to do
Release and Debug builds without needing to edit the cmake file. Perhaps
this calls for sed.

3) A bunch of tests fail (H5TEST-dt_arith and H5DUMP-*) but I assume that's
normal as they do with the unmodified package also.

4) The shared lib builds generate named pdb files and put them in the bin
directory and the final package. Static lib builds do not. I can build them
manually but it would be nice to have that automated.

Also I noticed the CPP lib built without complaint. If I enable it
explicitly in HDF518config.cmake with HDF5_BUILD_CPP_LIB:BOOL=ON then it
fails configuration. So for some reason it's not catching that the default
is on in the conflict checks.

···

On Wed, Nov 4, 2015 at 2:46 PM, Allen Byrne <byrn@hdfgroup.org> wrote:

The CMake code only has commands for shared - you could copy that block of
code over to the static library commands. (Edit as necessary)

Allen

On Wednesday, November 04, 2015 09:13:16 PM Dana Robinson wrote:
> It's not something that we've ever tested or would be likely to support,
but
> essentially:

> 1) Convince CMake to build the unsupported thread-safe + static
> configuration. There should be an "enable unsupported" option for this.
2)
> Possibly modify the HDF5 source code since our #ifdefs might assume that
> thread-safety and shared builds are standard on Windows. 3) Call
> H5TS_win32_thread_enter/exit() in your own DllMain() code like we do in
> H5.c. Note that those are private functions, so you'll have to pull in
our
> private headers. 4) If you plan on ever updating the version of HDF5 that
> we use, you'll need to hope we don't change the way we handle TLS in the
> future since you are using non-public-API functions.
> I would definitely recommend against this, even if it can be made to
work.
>
> Cheers,
>
> Dana
>
> From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf
Of
> David
Sent: Wednesday, November 4, 2015 4:01 PM
> To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
> Subject: Re: [Hdf-forum] Building Static Thread Safe Libs
>
> I'm actually building a dll so statically linked hdf5 would still able to
> use dll hooks. I just don't know how to get the build procedure to do it.

> On Wed, Nov 4, 2015 at 12:54 PM, Dana Robinson > > <derobins@hdfgroup.org<mailto:derobins@hdfgroup.org>> wrote:
Hi David,
>
> Since you say "dll" I'm going to assume you are on Windows.
Thread-safety is
> not supported in static builds of HDF5 on Windows. The reason for this is
> that, unlike with Pthreads, we have to use the dll hooks to set up and
tear
> down thread-local storage when Win32 threads are used. With no dll, there
> are no hooks.

> Cheers,
>
> Dana Robinson
> Software Engineer
> The HDF Group
>
> From: Hdf-forum
> [mailto:hdf-forum-bounces@lists.hdfgroup.org<mailto:
hdf-forum-bounces@lists
> .hdfgroup.org>] On Behalf Of David
Sent: Wednesday, November 4, 2015 3:49
> PM
> To: HDF Users Discussion List
> <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
> Subject: [Hdf-forum] Building Static Thread Safe Libs
>
> Is there any way to build a the static libs with thread safety turned on?
> I'm only using hdf5 in one binary so I'd rather not have to include an
> additional dll in my distribution.

> Thanks,
> David
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
> Twitter: https://twitter.com/hdf5
>

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5