Make the Cmake Windows build static please !

[Roel_Vanhout]

Hi all,

I have quite a bit of experience in building and deploying both static and dll-based programs, so I figured I'd throw in my $0.02

It seems to me that the crux of this particular part of the argument is here (from one of Dana's previous mails):

"Because a DLL built by linking to a static CRT will have its own CRT state, it is not recommended to link statically to the CRT in a DLL unless the consequences of this are specifically desired and understood."

Note that it says "...to link statically to the CRT _in a DLL_ unless...". Yes, linking the CRT statically into a DLL that is meant for redistribution is a bad idea. HOWEVER, that doesn't mean that static linking per se is bad. Linking a whole program statically (including the CRT) is a convenient way to make self-contained executables that can be distributed by just copying one binary around, that will not interfere with or depend on system-installed runtimes and that are portable across a large range of OS's, without having to wonder whether that OS has the right version of the CRT baked-in.

So, the original question, to have a static build option of HDF, is quite reasonable IMO (I have made my own Visual Studio projects in the past to do just that). The argument that static builds might be misused when they are linked into a DLL which will then go on and be used by programs that use a different version of the CRT doesn't seem like a very convincing argument to me, since (a) when doing builds with dependency chains this complicated, one should be building everything with the same version of the compiler anyway, and (b) there are many scenarios (scenarios that happen more than the dangerous one, IMO) where static building is perfectly safe.

So, the following assertion

"Personally, I'm not keen on making it easy for people to statically link to the CRT since it's a documented bad idea"

is, I'd say, not quite true; it's only not recommended when doing so *in a DLL*, *and* when that DLL is then used by programs that use different versions of the CRT.

Now, as to whether dev time (which is basically money) should be spend on supporting this, I have no opinion about; that is a trade-off that I am in no position to make. Making one's own VS project files and building against that is trivial anyway, and people relying on HDF heavily should always do that, IMO, and it's easy to add static build targets there. But I don't think that technical objections to static linking should tip the scale in the direction of 'don't support'.

cheers,

roel

···

________________________________
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Dana Robinson
Sent: Monday, June 10, 2013 10:11
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] [netcdfgroup] Make the Cmake Windows build staticplease !

Hi Pedro,

Answers interspersed below...

On Sun, Jun 9, 2013 at 11:26 PM, Pedro Vicente <pvicente@uci.edu<mailto:pvicente@uci.edu>> wrote:
Hi Dana

My original thread title

"Make the CMake Windows build static"

is a bit misleading, sorry.

HDF5, in the NCO and netCDF cases, is only one of several input libraries.

You can think of both HDF5 and netCDF in 2 ways

1) A library that will be used as *input* for a program (EXE file) that any HDF/netCDF user wants to build, his/her own program.
In this case he will need your LIB (in case he wants to compile) or the DLL (if he just wants to run the EXE)

2) A final program destination already done (EXE) , like h5dump or ncdump, that can be either statically built (no DLL) or dynamically built (need DLL)

For NCO, only case 2) exists. NCO is just the final program, not an input library to anyone.
We only distribute EXE files, not LIB files, like you do.

If I am taking the HDF5 binaries you provide, and combine with other several other library binaries, and even other libraries source code ,
in a combination of several LIB, DLL, and EXE files, dynamic is the way to go, because of the reasons you mention.

Note: just the binaries. Not the whatever way I have to obtain them, in this case, a Cmake compiler flag.

But, on the other hand, if I take all the source code only from these "libraries", and combine then into an EXE file statically build,
then I can consider all those "libraries" not libraries any more, just pieces of source files that will produce an EXE.

One note, though, and here I would like you to correct me, if I'm wrong.

I can separate all these individual "libraries", like HDF5, into an intermediate input file, a LIB file,
that I use *only* for my compilation into an EXE, *not* to distribute to anyone.

Take the simplest case of 2 source files

a.c that I compile into a.LIB

main.c that I compile into main.EXE with a link input of a.LIB

If I compile both of these Visual Studio projects with the same static flag, then,
there is no DLL dependency, and I think the CRT issues you mention do not happen.

Do you agree with this?

No.

a.lib and main.exe will have separate CRT states. You will not be able to malloc in a.c and free in main.exe without risking heap corruption in your exe.

I have attached a helpful set of slides that explains this.

Note that what you described will "work" in the sense that the OS won't complain. It's not very well documented, but it appears that the Win32 underpinnings are perfectly happy (this really isn't that surprising since everything belongs to the same process and the Win32 layer doesn't care about what's going on above it). It's just the bookkeeping information in the CRTs that gets messed up and this may or may not manifest itself as an error in your application. At any rate, it's not correct behavior.

This is not what happens in Linux and other Unix-like operating systems. The standard C library is a central part of the Linux OS but Windows uses the Win32 API exclusively. The bizarre linking issues on Windows seem to be due to this non-central nature of the library. This is also why you have to install the CRT on Windows, whereas on Linux it's just there.

The executive summary is that you should dynamically link ALL libraries that expose CRT objects via the library API. This obviously includes the Microsoft CRTs and, not so obviously, the HDF5 library.

Also, just to be clear, it is totally ok to write to memory allocated by a different CRT. That doesn't affect internal CRT heap tracking. It's just unsafe to resize or free the memory unless that CRT provided it.

When I started building Windows software in the 90's with the Visual Studio from that time I was using the DLL option.

Why? It seemed like a good idea. Dozens of programs done, thousands of compiler builds, why not just make them share the system DLL?

Many of this software I kept both the source and the final EXE, stored in a backup drive. But I forgot to store also the DLL, just because I never cared about them,
I was not distributing software to anyone, just doing my college things.

Fast forward 20 years, I want to run the EXE from that program. what happens? DLL whatever year 1993 is missing, oh my , I forgot to backup it.
And now where can I find it?, nowhere

The funny thing is that the EXE is still *binary* compatible regarding the Windows year 1993 version and Windows 7 today.
I just cannot run it because I don't have the DLL. If I *had* used the static option, now, today, I could run that EXE.

So, that's all what I was trying to say with my comments, "people, don't use DLLs, they are bad"

Totally different case as compared to distributing HDF5 as a library.

CMake is only an automatic generator of Visual Studio projects.

Like it is now, I can go to your generated projects and change the compiler flags.

But I thought it would be a valuable feature for the community to have this choice already built directly into the Cmake step, like Ward did for netCDF,
and John also did, I think, in the original setup, but you took off the final distribution.

Is that right, John?
Because I think the option -enable-static/shared, is just a given thing with the system, just a matter of adding a one liner to the script ?

So, regarding your comment

"Personally, I'm not keen on making it easy for people to statically link to the CRT since it's a documented bad idea"

I don't think you should be limiting our choices.

I am just asking to let us do your own choices in the way we compile our software.

As for supporting static CRT linking in the HDF5 library, the problem we face is support. If we supply it, people will use it, especially when it's the "easy" thing, and then complain when it doesn't work for them. We don't put it in CMake for the same reason you don't store knives with the pointy ends up or allow smoking at the gas station.

Given what I've demonstrated in the attached slides, you can probably see why I feel this way. I'm not 100% opposed, I'm just strongly opposed, and I'd want such an option flagged so harshly that people would feel personally insulted when they selected it. It's not enough to just document it, because almost nobody reads documentation.

I understand that this sucks. Using standard C on Microsoft platforms has been that way for a very long time. There's no C99 support, no standard place to put libraries (the system directory does not count), you have to rely on the PATH environment variable to find dlls and tools, POSIX support barely exists, etc. AND then you have the CRT linking issue on top of it. It's a complete nightmare, really.

Also, if I'm saying anything incorrect here, I would really appreciate being corrected. I'd be especially keen on seeing any authoritative sources that contradict what I've said. I've spent a lot of time digging around for information on this topic and MS doesn't make it easy.

Cheers,

Dana

[derobins]

Hi Roel,

Note that it says "...to link statically to the CRT _in a DLL_ unless...".

Yes, linking the CRT statically into a DLL that is meant for redistribution
is a bad idea. HOWEVER, that doesn't mean that static linking per se is
bad. Linking a whole program statically (including the CRT) is a convenient
way to make self-contained executables that can be distributed by just
copying one binary around, that will not interfere with or depend on
system-installed runtimes and that are portable across a large range of
OS's, without having to wonder whether that OS has the right version of the
CRT baked-in.

Yes, this is convenient, but it is easy to create incorrect programs when
you link several static libraries together when those libraries expose CRT
objects.

So, the original question, to have a static build option of HDF, is quite
reasonable IMO (I have made my own Visual Studio projects in the past to do
just that). The argument that static builds might be misused when they are
linked into a DLL which will then go on and be used by programs that use a
different version of the CRT doesn't seem like a very convincing argument
to me, since (a) when doing builds with dependency chains this complicated,
one should be building everything with the same version of the compiler
anyway, and (b) there are many scenarios (scenarios that happen more than
the dangerous one, IMO) where static building is perfectly safe.

So, the following assertion

“Personally, I'm not keen on making it easy for people to statically link
to the CRT since it's a documented bad idea”

is, I'd say, not quite true; it's only not recommended when doing so *in a
DLL*, *and* when that DLL is then used by programs that use different
versions of the CRT.

This is not true. Even when each static library is built with the same
compiler at the same time and in the same Visual Studio solution. The
reason for this is that each static library is built as a separate entity,
with its own CRT state. There is no "shared static state" going on unless
you build one thing. Building three libraries and then building an
executable that uses them is building four things, with four separate CRT
states if they are each statically linked to the CRT. Copying the source
code of said libraries into the same project as the executable and building
THAT is one thing and then there is only one CRT state, but that is not
what most people do.

Static linking is only safe if the libraries do not expose CRT objects via
the API. dll or lib, allocating in one CRT state and freeing in another
results in heap issues.

Cheers,

Dana

[byrn]

It seems to me that the discussions are not being specific enough with which build/CRT
configuration is discussed.

In my opinion there are two CRT issues and two build issues:
1: Linking with static CRT(/MT) and linking with dynamic CRT(/MD).
2: Building static libraries and building dynamic libraries.

Therefore there a four configurations of which we supply the cmake code and binaries
for the two build configurations with just the dynamic CRT.

Allen

···

On Monday, June 10, 2013 08:28:17 AM Dana Robinson wrote:

Hi Roel,

Note that it says "...to link statically to the CRT _in a DLL_ unless...". Yes, linking the
CRT statically into a DLL that is meant for redistribution is a bad idea. HOWEVER, that
doesn't mean that static linking per se is bad. Linking a whole program statically
(including the CRT) is a convenient way to make self-contained executables that can
be distributed by just copying one binary around, that will not interfere with or depend
on system-installed runtimes and that are portable across a large range of OS's,
without having to wonder whether that OS has the right version of the CRT baked-in.

Yes, this is convenient, but it is easy to create incorrect programs when you link
several static libraries together when those libraries expose CRT objects.

So, the original question, to have a static build option of HDF, is quite reasonable IMO
(I have made my own Visual Studio projects in the past to do just that). The argument
that static builds might be misused when they are linked into a DLL which will then go
on and be used by programs that use a different version of the CRT doesn't seem like
a very convincing argument to me, since (a) when doing builds with dependency
chains this complicated, one should be building everything with the same version of
the compiler anyway, and (b) there are many scenarios (scenarios that happen more
than the dangerous one, IMO) where static building is perfectly safe.

So, the following assertion

“Personally, I'm not keen on making it easy for people to statically link to the CRT since
it's a documented bad idea”

is, I'd say, not quite true; it's only not recommended when doing so *in a DLL*, *and*
when that DLL is then used by programs that use different versions of the CRT.

This is not true. Even when each static library is built with the same compiler at the
same time and in the same Visual Studio solution. The reason for this is that each
static library is built as a separate entity, with its own CRT state. There is no "shared
static state" going on unless you build one thing. Building three libraries and then
building an executable that uses them is building four things, with four separate CRT
states if they are each statically linked to the CRT. Copying the source code of said
libraries into the same project as the executable and building THAT is one thing and
then there is only one CRT state, but that is not what most people do.

Static linking is only safe if the libraries do not expose CRT objects via the API. dll or
lib, allocating in one CRT state and freeing in another results in heap issues.

Cheers,

Dana

[Roel_Vanhout]

Hi,

This is not true. Even when each static library is built with the same
compiler at the same time and in the same Visual Studio solution. The reason
for this is that each static library is built as a separate entity, with its
own CRT state. There is no "shared static state" going on unless you build
one thing. Building three libraries and then building an executable that
uses them is building four things, with four separate CRT states if they are
each statically linked to the CRT. Copying the source code of said libraries
into the same project as the executable and building THAT is one thing and
then there is only one CRT state, but that is not what most people do.

OK I understand your point now I think - to check that I do, can I ask, are you
saying that when considering slide 4 in the deck you send earlier, you mean
that even when all libraries are compiled with VS 2008, there still is no
shared state between the runtimes?

In that case, I don't think that's correct. Consider
http://msdn.microsoft.com/en-US/library/ms235460(v=vs.80).aspx and the
other CRT-related pages linked on that page (this includes the page you linked
to earlier). This page explicitly describes the situation 'when crossing DLL
boundaries'. This does not include crossing 'static library' boundaries. To
verify, I just made a simple .exe in VS2008 that uses 2 static libraries. In
those libraries, I have one function each that accesses an environment variable
as per the example in the page linked above. When I set the variable in a
function in static library 1, then static library 2 can read that environment
variable. That means, according to the descriptions on the MSDN website, that
the CRT's are shared (I can send my project to anyone who wants to test, but
it's really 5 minutes worth of work to make it yourself).

My mental picture is as follows: there is one instance of the CRT per DLL that
implements it. When the CRT dll is loaded, it allocates its heap, initializes
tables with environment variables, etc. Now when the CRT is linked statically,
that code doesn't get included yet in the .lib that is produced. It's the linker
that will insert the CRT code and that code will be shared between all libraries
that are linked in statically at that point in time. That static CRT will be
initialized at the moment the application is started, and all functions in
libraries that have been linked into that application will use the same heap,
environment variables etc.

Static linking is only safe if the libraries do not expose CRT objects via
the API. dll or lib, allocating in one CRT state and freeing in another
results in heap issues.

Allocating memory in one instance of the CRT and then freeing it in another
will cause heap corruption, yes, but I don't think (and as far as I can tell
my test described above proves) that static libraries that have all been build
by the same compiler version and are then all linked into an application,
again by the same compiler, have separate CRT instances between them. One
application with no DLL's = one CRT instance.

(I'm not sure what happens when you link a static library build with a
different version of VS, but I don't think we're concerned with that question
here).

cheers,

roel

···

________________________________
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Dana Robinson
Sent: Monday, June 10, 2013 15:28
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] [netcdfgroup] Make the Cmake Windows build staticplease !

Hi Roel,

Note that it says "...to link statically to the CRT _in a DLL_ unless...". Yes, linking the CRT statically into a DLL that is meant for redistribution is a bad idea. HOWEVER, that doesn't mean that static linking per se is bad. Linking a whole program statically (including the CRT) is a convenient way to make self-contained executables that can be distributed by just copying one binary around, that will not interfere with or depend on system-installed runtimes and that are portable across a large range of OS's, without having to wonder whether that OS has the right version of the CRT baked-in.

Yes, this is convenient, but it is easy to create incorrect programs when you link several static libraries together when those libraries expose CRT objects.

So, the original question, to have a static build option of HDF, is quite reasonable IMO (I have made my own Visual Studio projects in the past to do just that). The argument that static builds might be misused when they are linked into a DLL which will then go on and be used by programs that use a different version of the CRT doesn't seem like a very convincing argument to me, since (a) when doing builds with dependency chains this complicated, one should be building everything with the same version of the compiler anyway, and (b) there are many scenarios (scenarios that happen more than the dangerous one, IMO) where static building is perfectly safe.

So, the following assertion

"Personally, I'm not keen on making it easy for people to statically link to the CRT since it's a documented bad idea"

is, I'd say, not quite true; it's only not recommended when doing so *in a DLL*, *and* when that DLL is then used by programs that use different versions of the CRT.

This is not true. Even when each static library is built with the same compiler at the same time and in the same Visual Studio solution. The reason for this is that each static library is built as a separate entity, with its own CRT state. There is no "shared static state" going on unless you build one thing. Building three libraries and then building an executable that uses them is building four things, with four separate CRT states if they are each statically linked to the CRT. Copying the source code of said libraries into the same project as the executable and building THAT is one thing and then there is only one CRT state, but that is not what most people do.

Static linking is only safe if the libraries do not expose CRT objects via the API. dll or lib, allocating in one CRT state and freeing in another results in heap issues.

Cheers,

Dana

[byrn]

One clarification - The CRT option IS a compiler flag, it's just simpler to think of it as a
link issue.

Allen

···

On Monday, June 10, 2013 08:56:14 AM Allen Byrne wrote:

It seems to me that the discussions are not being specific enough with which build/CRT
configuration is discussed.

In my opinion there are two CRT issues and two build issues:
1: Linking with static CRT(/MT) and linking with dynamic CRT(/MD).
2: Building static libraries and building dynamic libraries.

Therefore there a four configurations of which we supply the cmake code and binaries
for the two build configurations with just the dynamic CRT.

Allen

[derobins]

Hi Roel,

**
Hi,

> This is not true. Even when each static library is built with the same
> compiler at the same time and in the same Visual Studio solution. The
reason
> for this is that each static library is built as a separate entity, with
its
> own CRT state. There is no "shared static state" going on unless you
build
> one thing. Building three libraries and then building an executable that
> uses them is building four things, with four separate CRT states if they
are
> each statically linked to the CRT. Copying the source code of said
libraries
> into the same project as the executable and building THAT is one thing
and
> then there is only one CRT state, but that is not what most people do.

OK I understand your point now I think - to check that I do, can I ask,
are you
saying that when considering slide 4 in the deck you send earlier, you mean
that even when all libraries are compiled with VS 2008, there still is no
shared state between the runtimes?

Yes. Shared libraries do this, not static libraries. There is no shared
state between static libraries. The slide with the dashed lines (slide 5,
4 if you ignore the title slides - I should have numbered them!) is
incorrect.

In that case, I don't think that's correct. Consider
http://msdn.microsoft.com/en-US/library/ms235460(v=vs.80).aspx and the
other CRT-related pages linked on that page (this includes the page you
linked
to earlier). This page explicitly describes the situation 'when crossing
DLL
boundaries'. This does not include crossing 'static library' boundaries. To
verify, I just made a simple .exe in VS2008 that uses 2 static libraries.
In
those libraries, I have one function each that accesses an environment
variable
as per the example in the page linked above. When I set the variable in a
function in static library 1, then static library 2 can read that
environment
variable. That means, according to the descriptions on the MSDN website,
that
the CRT's are shared (I can send my project to anyone who wants to test,
but
it's really 5 minutes worth of work to make it yourself).

The sample code on that page (I switched it to VS2012 -
http://msdn.microsoft.com/en-US/library/ms235460(v=vs.110).aspx) has you
build an library and an executable that uses it. It says:

"The DLL and .exe file are built with /MD, so they share a single copy of
the CRT.

If you rebuild with /MT so that they use separate copies of the CRT,
running the resulting test1Main.exe results in an access violation."

/MT statically links to the CRT, so the example in your link explicitly
shows that the runtime state is not shared, even when everything is being
built by the same compiler at the same time.

Also, from this link (
http://msdn.microsoft.com/en-us/library/abx4dbyh(v=vs.110).aspx):

"Using the statically linked CRT implies that any state information saved
by the C runtime library will be local to that instance of the CRT."

Granted, this page talks about dlls, but that's usually what people are
building on Windows.

My mental picture is as follows: there is one instance of the CRT per DLL
that
implements it. When the CRT dll is loaded, it allocates its heap,
initializes
tables with environment variables, etc. Now when the CRT is linked
statically,
that code doesn't get included yet in the .lib that is produced. It's the
linker
that will insert the CRT code and that code will be shared between all
libraries
that are linked in statically at that point in time. That static CRT will
be
initialized at the moment the application is started, and all functions in
libraries that have been linked into that application will use the same
heap,
environment variables etc.

Your picture of static linkage is incorrect. Libraries are fully linked
with the CRT at creation and each executable or library has a separate
state due to the code being duplicated. Static CRT linking is no different
than statically linking any other library. This is why building the
example illustrated above with /MT causes issues.

I'll agree that this is a confusing issue! Perhaps there is some
enlightening text in my Windows system programming books. I'll check and
get back to everyone.

Cheers,

Dana

···

On Mon, Jun 10, 2013 at 9:30 AM, Roel Vanhout <rvanhout@riks.nl> wrote:

[roger]

Hi,

Noting the MinGW64(4.8) style of building: just tried the cmake(2.8.11.1) build approach for hdf5(1.8.11). Some gliches and I do appreciate the effort of build tools such as cmake.

I do try every now and then but fall back to building hdf5 and hdf5_hl by selecting the sources in NetBeans projects and after blending/updating a few config headers build it with the make files generated from NetBeans

I suspect most developers(including me) since we're all busy don't give feed back often enough. We get something working and don't have the time to share experience.

First thing with cmake build hit was the executable looking for a good make. make's on windows has differences and I have a gmake of recent enough that works so I pointed it to it. Get further then turn on parallel and keep filling paths to openmpi libraries and cmake rejects for a bit and then finally the right combination is hit and it is satisfied.
Configure done.
Generate done.

gmake goes at building H5detect.exe and soon hit undefined reference to '__imp_gethostname'

Now as a developer I know my end code won't depend on this H5detect.exe so I at this point just go back to building with my handy NetBeans make scripts which build hdf5 with openmpi readily and I'm in business with hdf5 1.8.11 upgrade. Able to build window's dll's that aren't dependent on being under cygwin; they are windows dll's

Of course I take care of a couple of H5_HAVE_WIN32_API preprocessor choices since mingw64 is closer to the !H5_HAVE_WIN32_API case. The distinction of WIN32 is for me more a distinction of using Microsoft tool chain. Instead my preference is portable compiler such as g++ 4.8. The portion I had to break/comment out this time until I need it is the H5PL plugin code

···

On 06/10/2013 01:28 PM, Allen Byrne wrote:

One clarification - The CRT option IS a compiler flag, it's just simpler to think of it as a link issue.

Allen

On Monday, June 10, 2013 08:56:14 AM Allen Byrne wrote:

It seems to me that the discussions are not being specific enough with which build/CRT configuration is discussed.

In my opinion there are two CRT issues and two build issues:

1: Linking with static CRT(/MT) and linking with dynamic CRT(/MD).

2: Building static libraries and building dynamic libraries.

Therefore there a four configurations of which we supply the cmake code and binaries for the two build configurations with just the dynamic CRT.

Allen

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

[derobins]

Hi all,

I stand corrected.

I just created a test project and inspected the assembler for the output
and it turns out that the CRT does share state when statically linked
together.

For example, this is what I get when I create a static library that calls
free via a library function:

foo_free:
  0000000000000040: 48 89 4C 24 08 mov qword ptr [rsp+8],rcx
  0000000000000045: 57 push rdi
  0000000000000046: 48 83 EC 20 sub rsp,20h
  000000000000004A: 48 8B FC mov rdi,rsp
  000000000000004D: B9 08 00 00 00 mov ecx,8
  0000000000000052: B8 CC CC CC CC mov eax,0CCCCCCCCh
  0000000000000057: F3 AB rep stos dword ptr [rdi]
  0000000000000059: 48 8B 4C 24 30 mov rcx,qword ptr [rsp+30h]
  000000000000005E: 48 8B 4C 24 30 mov rcx,qword ptr [rsp+30h]
  *0000000000000063: E8 00 00 00 00 call free*
  0000000000000068: 48 83 C4 20 add rsp,20h
  000000000000006C: 5F pop rdi
  000000000000006D: C3 ret

It clearly just has a placeholder (highlighted line) that will be resolved
when the final executable (dll or exe) is linked. As long as your
application is a monolith, with no internal dlls at any level, the CRT
issues should not exist. The linker will just copy in the static CRT code
and all C library calls will resolve to that.

Again, I stand corrected and I apologize for any confusion that I caused.

I'll talk to Elena and Allen about what we can do to get this set up and
document it. I'd still want it flagged as dangerous since I suspect it
will burn a lot of people.

Cheers,

Dana

···

On Mon, Jun 10, 2013 at 4:26 PM, Roger Martin <roger@quantumbioinc.com>wrote:

Hi,

Noting the MinGW64(4.8) style of building: just tried the cmake(2.8.11.1)
build approach for hdf5(1.8.11). Some gliches and I do appreciate the
effort of build tools such as cmake.

I do try every now and then but fall back to building hdf5 and hdf5_hl by
selecting the sources in NetBeans projects and after blending/updating a
few config headers build it with the make files generated from NetBeans

I suspect most developers(including me) since we're all busy don't give
feed back often enough. We get something working and don't have the time
to share experience.

First thing with cmake build hit was the executable looking for a good
make. make's on windows has differences and I have a gmake of recent enough
that works so I pointed it to it. Get further then turn on parallel and
keep filling paths to openmpi libraries and cmake rejects for a bit and
then finally the right combination is hit and it is satisfied.
Configure done.
Generate done.

gmake goes at building H5detect.exe and soon hit undefined reference to
'__imp_gethostname'

Now as a developer I know my end code won't depend on this H5detect.exe so
I at this point just go back to building with my handy NetBeans make
scripts which build hdf5 with openmpi readily and I'm in business with hdf5
1.8.11 upgrade. Able to build window's dll's that aren't dependent on being
under cygwin; they are windows dll's

Of course I take care of a couple of H5_HAVE_WIN32_API preprocessor
choices since mingw64 is closer to the !H5_HAVE_WIN32_API case. The
distinction of WIN32 is for me more a distinction of using Microsoft tool
chain. Instead my preference is portable compiler such as g++ 4.8. The
portion I had to break/comment out this time until I need it is the H5PL
plugin code

On 06/10/2013 01:28 PM, Allen Byrne wrote:

One clarification - The CRT option IS a compiler flag, it's just simpler
to think of it as a link issue.

Allen

On Monday, June 10, 2013 08:56:14 AM Allen Byrne wrote:

It seems to me that the discussions are not being specific enough with
which build/CRT configuration is discussed.

In my opinion there are two CRT issues and two build issues:

1: Linking with static CRT(/MT) and linking with dynamic CRT(/MD).

2: Building static libraries and building dynamic libraries.

Therefore there a four configurations of which we supply the cmake code
and binaries for the two build configurations with just the dynamic CRT.

Allen

_______________________________________________
Hdf-forum is for HDF software users discussion.Hdf-forum@lists.hdfgroup.orghttp://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

[byrn]

I have implemented a CMake way of handling this issue in the 1_8_cmake branch of
the hdf5 svn repository.

In the root CMakeFiles.txt file I added an "include UserMacros.cmake" command that
will load in an option and macro to change the flags to allow the use of /MT on
windows. I also modified all the CMake code to enable a hook into the compile and link
processes.

The root UserMacros.cmake file is a placeholder for user-defined CMake code. The
cmake code for the MT option is in the config/cmake folder under a new UserMacros
folder in the file Windows_MT.cmake. Simply copy the contents of the
Windows_MT.cmake file to the contents of the root UserMacros.cmake file, and enable
the option.

We would appreciate folks testing this out before we commit to the 1_8 branch.

Allen

···

On Monday, June 10, 2013 05:58:18 PM Dana Robinson wrote:

Hi all,

I stand corrected.

I just created a test project and inspected the assembler for the output and it turns
out that the CRT does share state when statically linked together.

For example, this is what I get when I create a static library that calls free via a library
function:

foo_free:
  0000000000000040: 48 89 4C 24 08 mov qword ptr [rsp+8],rcx
  0000000000000045: 57 push rdi
  0000000000000046: 48 83 EC 20 sub rsp,20h
  000000000000004A: 48 8B FC mov rdi,rsp
  000000000000004D: B9 08 00 00 00 mov ecx,8
  0000000000000052: B8 CC CC CC CC mov eax,0CCCCCCCCh
  0000000000000057: F3 AB rep stos dword ptr [rdi]
  0000000000000059: 48 8B 4C 24 30 mov rcx,qword ptr [rsp+30h]
  000000000000005E: 48 8B 4C 24 30 mov rcx,qword ptr [rsp+30h]
  *0000000000000063: E8 00 00 00 00 call free*
  0000000000000068: 48 83 C4 20 add rsp,20h
  000000000000006C: 5F pop rdi
  000000000000006D: C3 ret

It clearly just has a placeholder (highlighted line) that will be resolved when the final
executable (dll or exe) is linked. As long as your application is a monolith, with no
internal dlls at any level, the CRT issues should not exist. The linker will just copy in the
static CRT code and all C library calls will resolve to that.

Again, I stand corrected and I apologize for any confusion that I caused.

I'll talk to Elena and Allen about what we can do to get this set up and document it. I'd
still want it flagged as dangerous since I suspect it will burn a lot of people.

Cheers,

Dana

[psandhu]

I am hitting this problem as I am upgrading from 1.8.3 to 1.8.20. I was able to follow your instructions in building with the UserMacros option enabled. However I still get errors when building my program that built fine with 1.8.3

Do I have to define any variables or such ?