On using different runtime libraries ( Re: HDF5-1.8.4 patch1 on Windows7 64bit )

Hi,

Dana wrote:

From what I understand, it boils down to this:

Every library and executable in your application should use the same
instance of the C runtime library.

That's a good rule-of-thumb to avoid problems, but using different runtime libraries
should usually be possible if their interfaces take it into account.
    For example, if you allocate memory in one module and free it in a module with
different runtime library, you can expect a crash... but if you only pass things like
integer handles from one module to the other, it should generally work, provided that
function calling conventions agree.

For example, the C interface of HDF5 typically returns only handles of type hid_t or
basic data types instead of direct pointers. There are some exceptions, like the
hvl_t type of variable length datatypes (which therefore requires that both modules
have compatible understanding of member alignments within that struct) and the custom
filter interface (which has a malloc-free problem when modules have different
runtime libraries), but a fix for the filter interface is in the works as far as
I know.

My experiences are mainly from using DLLs compiled by MS compilers in Borland
projects and using MS and Borland DLLs with Java, using statically-linked runtime
libraries in both modules (for some reason I got into problems when using Borland's
dynamic RTL). So, I'm just assuming that the issues in different versions of
Microsoft compilers are of similar nature.

Regards,
Vesa Paatero

=== Original message follows ===
Message: 1

···

Date: Wed, 17 Mar 2010 11:29:06 -0500
From: Dana Robinson <derobins@hdfgroup.org>
To: HDF Users Discussion List <hdf-forum@hdfgroup.org>
Subject: Re: [Hdf-forum] HDF5-1.8.4 patch1 on Windows7 64bit
Message-ID:
        <878240001003170929t93d23c5v7aebd31ef872cd65@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

I don't know if this solves people's linking problems but I thought I'd
weigh in with what I know about Windows C runtime (CRT) linking.

From what I understand, it boils down to this:

Every library and executable in your application should use the same
instance of the C runtime library.

The reason for this is that separate dlls have a separate internal state and
thus different ideas about things like valid file handles and allocated
memory. For example, if you malloc() some memory in a library linked to
msvcr80.dll and then try to realloc() or free() it in a program linked to
msvcr90.dll, you will have problems since msvcr90 doesn't know about the
first allocation's internal memory handles (writing to memory allocated
using a different C runtime should work fine, though, since the underlying
internal memory handles are not being manipulated). Debug libraries (e.g.
msvcr80d.dll) are also different in this respect since they are different
dlls so using a library compiled in release mode with an executable compiled
in debug mode can cause problems. Keep in mind that many things will appear
to work just fine when you mix C runtime libraries - it's just when you pass
CRT resources around that you'll run into issues. It's also important to
remember that statically linking to a C runtime gives you a private version
of that runtime that is NOT shared with any other components. For this
reason, it is recommended that you only statically link against a library
when you are building an executable and NOT when you are building a library.

You can check your dependencies with dumpbin if you have Visual Studio
installed. Fire up a Visual Studio command prompt and use dumpbin /imports
<path to library or program> to see which dynamic libraries a particular
program or dll requires. The existing 32-bit HDF binaries show a dependency
on msvcr80.dll (Visual C++ 8.0 / Visual Studio 2005), the VS2005 version of
zlib is also linked to msvcr80.dll and szip does not require any C library
functions (it's either just linked to kernel32 and uses Windows functions or
it incorrectly statically links to the CRT - I'll have to check that out).
The tools are an odd case - they are statically linked to HDF5 and have a
private version of the C runtime that is separate from, say, msvcr80.dll but
are also dynamically linked to zlib which is, in turn, linked to
msvcr80.dll. This means that zlib code used by the tools and the rest of
the tool code will have different CRT states.

So this is (pretty much) all well and good as long as you are using Visual
Studio 2005. Problems can arise, however, when you try to use Visual Studio
2008 to build an application that links against VS2005 versions of HDF5.
VS2008 will link your application against msvcr90.dll and this can cause the
issues described above. Unfortunately, we do not distribute VS2008 binaries
on the HDF5 downloads page but I'm sure this will be rectified soon.

We're looking into the library issues here and we'll have more information
in the future, but I think a short term roadmap for us could be:

1) We need to offer Visual Studio 2008 HDF5 binaries that are linked against
msvcr90.dll.

2) Our binaries (HDF5, szip, zlib) need to include debug libraries so that
users can properly link when creating debug builds.

3) We need to educate our Windows users about these issues, probably via a
README document in the distribution.

4) Our static tools should be 100% statically linked.

Also, in general, you should use the official Windows CRT installers to
install the required C runtime, instead of just copying dlls around. You
can find the proper installers on Microsoft's website (links which will
surely go stale soon provided below). I have no idea why the CRTs aren't
just considered a normal part of a Windows installation and maintained
accordingly. You'd think that would make everyone's life easier.

Visual Studio 2005 SP1
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=200b2fd9-ae1a-4a14-984d-389c36f85647

Visual Studio 2008 SP1
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=fbee1648-7106-44a7-9649-6d9f6d58056e

I hope that helps.

Dana Robinson
Bioinformatics Software Engineer
The HDF Group
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.hdfgroup.org/pipermail/hdf-forum_hdfgroup.org/attachments/20100317/cf87ba60/attachment-0001.html>