extend compound dataset

[mike.jackson]

Great Write up. I'll just add my own experiences and some suggestions to make our (the developers USING hdf5) easier.
  I am mainly an OS X Developer and do Windows development due to customer constraints so that is the perspective that this comes from.
  RULE #1: Always build ALL your libraries with the SAME EXACT version of Visual Studio, down to the Service Pack AND Security Patch Level.
  RULE #2: NEVER break rule #1.

Following those rules will allow you to have a much easier time with debugging/running/compiling on windows. What do I mean by exact versions? Each library that you are going to compile into your application MUST be linked against the same version of the C/C++ runtime libraries with respect to the following:
  Single/Multi Threaded
  Static/Dynamic Linked to the Runtimes
  Debug/Release Versions of each runtime.

Again, following the above will help the developer have a much easier time with Visual Studio Development.

If you get link warnings about adding "NODEFAULTLIB" then you are NOT following the above rules and nice random crashes and bugs WILL pop up in your program.

What can HDFGroup do to Help? PLEASE PLEASE PLEASE list the exact version of Visual Studio used to compile the binaries. This should be something like Visual Studio 2008 SP1 (9.0.xxx.xxx SPI) or if the original Visual Studio 2008 was used, then 9.0.xxx.xxx RTM). These values can be found by going to "Help->About Visual Studio". Also explicitly list HOW they were compiled with respect to Single/MultiThreaded, Static/Dynamic Runtime linkage, Dynamic/Static libraries, Debug/Release compiles.

  Having this information on the download page will help alleviate help emails from confused users.

   Again, these are my experiences, Yours may vary. Following those rules helps keep things simple for Visual Studio Development.

···

___________________________________________________________
Mike Jackson www.bluequartz.net
Principal Software Engineer mike.jackson@bluequartz.net
BlueQuartz Software Dayton, Ohio

On Mar 17, 2010, at 12:29 PM, Dana Robinson wrote:

I don't know if this solves people's linking problems but I thought I'd weigh in with what I know about Windows C runtime (CRT) linking.

From what I understand, it boils down to this:

Every library and executable in your application should use the same instance of the C runtime library.

The reason for this is that separate dlls have a separate internal state and thus different ideas about things like valid file handles and allocated memory. For example, if you malloc() some memory in a library linked to msvcr80.dll and then try to realloc() or free() it in a program linked to msvcr90.dll, you will have problems since msvcr90 doesn't know about the first allocation's internal memory handles (writing to memory allocated using a different C runtime should work fine, though, since the underlying internal memory handles are not being manipulated). Debug libraries (e.g. msvcr80d.dll) are also different in this respect since they are different dlls so using a library compiled in release mode with an executable compiled in debug mode can cause problems. Keep in mind that many things will appear to work just fine when you mix C runtime libraries - it's just when you pass CRT resources around that you'll run into issues. It's also important to remember that statically linking to a C runtime gives you a private version of that runtime that is NOT shared with any other components. For this reason, it is recommended that you only statically link against a library when you are building an executable and NOT when you are building a library.

You can check your dependencies with dumpbin if you have Visual Studio installed. Fire up a Visual Studio command prompt and use dumpbin /imports <path to library or program> to see which dynamic libraries a particular program or dll requires. The existing 32-bit HDF binaries show a dependency on msvcr80.dll (Visual C++ 8.0 / Visual Studio 2005), the VS2005 version of zlib is also linked to msvcr80.dll and szip does not require any C library functions (it's either just linked to kernel32 and uses Windows functions or it incorrectly statically links to the CRT - I'll have to check that out). The tools are an odd case - they are statically linked to HDF5 and have a private version of the C runtime that is separate from, say, msvcr80.dll but are also dynamically linked to zlib which is, in turn, linked to msvcr80.dll. This means that zlib code used by the tools and the rest of the tool code will have different CRT states.

So this is (pretty much) all well and good as long as you are using Visual Studio 2005. Problems can arise, however, when you try to use Visual Studio 2008 to build an application that links against VS2005 versions of HDF5. VS2008 will link your application against msvcr90.dll and this can cause the issues described above. Unfortunately, we do not distribute VS2008 binaries on the HDF5 downloads page but I'm sure this will be rectified soon.

We're looking into the library issues here and we'll have more information in the future, but I think a short term roadmap for us could be:

1) We need to offer Visual Studio 2008 HDF5 binaries that are linked against msvcr90.dll.

2) Our binaries (HDF5, szip, zlib) need to include debug libraries so that users can properly link when creating debug builds.

3) We need to educate our Windows users about these issues, probably via a README document in the distribution.

4) Our static tools should be 100% statically linked.

Also, in general, you should use the official Windows CRT installers to install the required C runtime, instead of just copying dlls around. You can find the proper installers on Microsoft's website (links which will surely go stale soon provided below). I have no idea why the CRTs aren't just considered a normal part of a Windows installation and maintained accordingly. You'd think that would make everyone's life easier.

Visual Studio 2005 SP1
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=200b2fd9-ae1a-4a14-984d-389c36f85647

Visual Studio 2008 SP1
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=fbee1648-7106-44a7-9649-6d9f6d58056e

I hope that helps.

Dana Robinson
Bioinformatics Software Engineer
The HDF Group
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

[Dr_X]

Hi All,
First of all, I appreciate Michael's prompt follow up and all the git info.
I have downloaded HDF5 git repository from
git clone git://gitorious.org/hdf5/hdf5-v18.git
szip from
git clone git://gitorious.org/hdf5/szip.git
and zlib from
http://zlib.net/zlib-1.2.4.tar.gz

If you don't know how to use git. Please install cygwin on your windows machine and start a terminal before you copy and paste above statements. Certainly create hosting directories for HDF5, SZIP, and ZLIB is mandatory. Follow Michael and Dana's note, I want to take no chance and recompile szip and zlib from the source as well.

Build szip and zlib first:
Use cmake to build szip and zlib-1.2.4\projects\vissualc6\zlib.dsw to build zlib. Visual Studio 2008 will automatically convert zlib.dsw to *.sln. And do remember to create new x64 configurations if your machine is 64bit. Both packages (DLL Release and LIB Release) were built fine (without any L4096 warning on /NODEFAULTLIB... :)).

Then use cmake to deal with hdf5. After locating all the right places for zlib and szip libraries and include directories, you are gold. I built hdf5 with HDF5_ENABLE_SZIP_SUPPORT and HDF5_ENABLE_Z_LIB_SUPPORT enabled for my own testing. I guess these two flags allow the created hdf5 library to be usefully without the companion of additional szip and zlib libraries.

Note, zip_perf project will fail on Windows due to gettimeofday() and S_IRWXU dependencies.

At least all the hdf5 .exe are running fine. Best wishes to all Windows 64bit users. And thanks Michael and Dana.

Best,
xunlei

···

On 3/17/2010 12:50 PM, Michael Jackson wrote:

Great Write up. I'll just add my own experiences and some suggestions to make our (the developers USING hdf5) easier.
I am mainly an OS X Developer and do Windows development due to customer constraints so that is the perspective that this comes from.
RULE #1: Always build ALL your libraries with the SAME EXACT version of Visual Studio, down to the Service Pack AND Security Patch Level.
RULE #2: NEVER break rule #1.

Following those rules will allow you to have a much easier time with debugging/running/compiling on windows. What do I mean by exact versions? Each library that you are going to compile into your application MUST be linked against the same version of the C/C++ runtime libraries with respect to the following:
Single/Multi Threaded
Static/Dynamic Linked to the Runtimes
Debug/Release Versions of each runtime.

Again, following the above will help the developer have a much easier time with Visual Studio Development.

If you get link warnings about adding "NODEFAULTLIB" then you are NOT following the above rules and nice random crashes and bugs WILL pop up in your program.

What can HDFGroup do to Help? PLEASE PLEASE PLEASE list the exact version of Visual Studio used to compile the binaries. This should be something like Visual Studio 2008 SP1 (9.0.xxx.xxx SPI) or if the original Visual Studio 2008 was used, then 9.0.xxx.xxx RTM). These values can be found by going to "Help->About Visual Studio". Also explicitly list HOW they were compiled with respect to Single/MultiThreaded, Static/Dynamic Runtime linkage, Dynamic/Static libraries, Debug/Release compiles.

Having this information on the download page will help alleviate help emails from confused users.

  Again, these are my experiences, Yours may vary. Following those rules helps keep things simple for Visual Studio Development.
___________________________________________________________
Mike Jackson www.bluequartz.net
Principal Software Engineer mike.jackson@bluequartz.net
BlueQuartz Software Dayton, Ohio

On Mar 17, 2010, at 12:29 PM, Dana Robinson wrote:

I don't know if this solves people's linking problems but I thought I'd weigh in with what I know about Windows C runtime (CRT) linking.

From what I understand, it boils down to this:

Every library and executable in your application should use the same instance of the C runtime library.

The reason for this is that separate dlls have a separate internal state and thus different ideas about things like valid file handles and allocated memory. For example, if you malloc() some memory in a library linked to msvcr80.dll and then try to realloc() or free() it in a program linked to msvcr90.dll, you will have problems since msvcr90 doesn't know about the first allocation's internal memory handles (writing to memory allocated using a different C runtime should work fine, though, since the underlying internal memory handles are not being manipulated). Debug libraries (e.g. msvcr80d.dll) are also different in this respect since they are different dlls so using a library compiled in release mode with an executable compiled in debug mode can cause problems. Keep in mind that many things will appear to work just fine when you mix C runtime libraries - it's just when you pass CRT resources around that you'll run into issues. It's also important to remember that statically linking to a C runtime gives you a private version of that runtime that is NOT shared with any other components. For this reason, it is recommended that you only statically link against a library when you are building an executable and NOT when you are building a library.

You can check your dependencies with dumpbin if you have Visual Studio installed. Fire up a Visual Studio command prompt and use dumpbin /imports <path to library or program> to see which dynamic libraries a particular program or dll requires. The existing 32-bit HDF binaries show a dependency on msvcr80.dll (Visual C++ 8.0 / Visual Studio 2005), the VS2005 version of zlib is also linked to msvcr80.dll and szip does not require any C library functions (it's either just linked to kernel32 and uses Windows functions or it incorrectly statically links to the CRT - I'll have to check that out). The tools are an odd case - they are statically linked to HDF5 and have a private version of the C runtime that is separate from, say, msvcr80.dll but are also dynamically linked to zlib which is, in turn, linked to msvcr80.dll. This means that zlib code used by the tools and the rest of the tool code will have different CRT states.

So this is (pretty much) all well and good as long as you are using Visual Studio 2005. Problems can arise, however, when you try to use Visual Studio 2008 to build an application that links against VS2005 versions of HDF5. VS2008 will link your application against msvcr90.dll and this can cause the issues described above. Unfortunately, we do not distribute VS2008 binaries on the HDF5 downloads page but I'm sure this will be rectified soon.

We're looking into the library issues here and we'll have more information in the future, but I think a short term roadmap for us could be:

1) We need to offer Visual Studio 2008 HDF5 binaries that are linked against msvcr90.dll.

2) Our binaries (HDF5, szip, zlib) need to include debug libraries so that users can properly link when creating debug builds.

3) We need to educate our Windows users about these issues, probably via a README document in the distribution.

4) Our static tools should be 100% statically linked.

Also, in general, you should use the official Windows CRT installers to install the required C runtime, instead of just copying dlls around. You can find the proper installers on Microsoft's website (links which will surely go stale soon provided below). I have no idea why the CRTs aren't just considered a normal part of a Windows installation and maintained accordingly. You'd think that would make everyone's life easier.

Visual Studio 2005 SP1
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=200b2fd9-ae1a-4a14-984d-389c36f85647

Visual Studio 2008 SP1
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=fbee1648-7106-44a7-9649-6d9f6d58056e

I hope that helps.

Dana Robinson
Bioinformatics Software Engineer
The HDF Group
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

[mike.jackson]

One final note. All the git stuff _should_ be now or very soon in the official SVN repo of "The HDFGroup". A few of us setup the gitorious site to collaborate easily on the "Cmakeification" of HDF5. Those changes will be pushed back to the official SVN repo "real soon now". What I am getting at is that the gitorious site will go away when the SVN gets updated. I do NOT want to fracture the HDF5 source/community by having multiple places to download HDF5 source codes.

  Dr. X. I am glad everything worked for you. Alternatively you can download an MSys+Git installer from google code <http://code.google.com/p/msysgit/downloads/list > which is probably an easier install for some due to NOT needing cygwin.

I actually have a "ZLib" that builds with CMake on my own machine. Not sure if anyone is interested in that. Adding that would make HDF5 completely configurable with CMake. Just a thought. Then again, that leads to maintenance nightmares ...

···

___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 3:06 PM, Dr. X wrote:

Hi All,
First of all, I appreciate Michael's prompt follow up and all the git info.
I have downloaded HDF5 git repository from
git clone git://gitorious.org/hdf5/hdf5-v18.git
szip from
git clone git://gitorious.org/hdf5/szip.git
and zlib from
http://zlib.net/zlib-1.2.4.tar.gz

If you don't know how to use git. Please install cygwin on your windows machine and start a terminal before you copy and paste above statements. Certainly create hosting directories for HDF5, SZIP, and ZLIB is mandatory. Follow Michael and Dana's note, I want to take no chance and recompile szip and zlib from the source as well.

Build szip and zlib first:
Use cmake to build szip and zlib-1.2.4\projects\vissualc6\zlib.dsw to build zlib. Visual Studio 2008 will automatically convert zlib.dsw to *.sln. And do remember to create new x64 configurations if your machine is 64bit. Both packages (DLL Release and LIB Release) were built fine (without any L4096 warning on /NODEFAULTLIB... :)).

Then use cmake to deal with hdf5. After locating all the right places for zlib and szip libraries and include directories, you are gold. I built hdf5 with HDF5_ENABLE_SZIP_SUPPORT and HDF5_ENABLE_Z_LIB_SUPPORT enabled for my own testing. I guess these two flags allow the created hdf5 library to be usefully without the companion of additional szip and zlib libraries.

Note, zip_perf project will fail on Windows due to gettimeofday() and S_IRWXU dependencies.

At least all the hdf5 .exe are running fine. Best wishes to all Windows 64bit users. And thanks Michael and Dana.

Best,
xunlei

On 3/17/2010 12:50 PM, Michael Jackson wrote:

Great Write up. I'll just add my own experiences and some suggestions to make our (the developers USING hdf5) easier.
I am mainly an OS X Developer and do Windows development due to customer constraints so that is the perspective that this comes from.
RULE #1: Always build ALL your libraries with the SAME EXACT version of Visual Studio, down to the Service Pack AND Security Patch Level.
RULE #2: NEVER break rule #1.

Following those rules will allow you to have a much easier time with debugging/running/compiling on windows. What do I mean by exact versions? Each library that you are going to compile into your application MUST be linked against the same version of the C/C++ runtime libraries with respect to the following:
Single/Multi Threaded
Static/Dynamic Linked to the Runtimes
Debug/Release Versions of each runtime.

Again, following the above will help the developer have a much easier time with Visual Studio Development.

If you get link warnings about adding "NODEFAULTLIB" then you are NOT following the above rules and nice random crashes and bugs WILL pop up in your program.

What can HDFGroup do to Help? PLEASE PLEASE PLEASE list the exact version of Visual Studio used to compile the binaries. This should be something like Visual Studio 2008 SP1 (9.0.xxx.xxx SPI) or if the original Visual Studio 2008 was used, then 9.0.xxx.xxx RTM). These values can be found by going to "Help->About Visual Studio". Also explicitly list HOW they were compiled with respect to Single/MultiThreaded, Static/Dynamic Runtime linkage, Dynamic/Static libraries, Debug/Release compiles.

Having this information on the download page will help alleviate help emails from confused users.

  Again, these are my experiences, Yours may vary. Following those rules helps keep things simple for Visual Studio Development.
___________________________________________________________
Mike Jackson www.bluequartz.net
Principal Software Engineer mike.jackson@bluequartz.net
BlueQuartz Software Dayton, Ohio

On Mar 17, 2010, at 12:29 PM, Dana Robinson wrote:

I don't know if this solves people's linking problems but I thought I'd weigh in with what I know about Windows C runtime (CRT) linking.

From what I understand, it boils down to this:

Every library and executable in your application should use the same instance of the C runtime library.

The reason for this is that separate dlls have a separate internal state and thus different ideas about things like valid file handles and allocated memory. For example, if you malloc() some memory in a library linked to msvcr80.dll and then try to realloc() or free() it in a program linked to msvcr90.dll, you will have problems since msvcr90 doesn't know about the first allocation's internal memory handles (writing to memory allocated using a different C runtime should work fine, though, since the underlying internal memory handles are not being manipulated). Debug libraries (e.g. msvcr80d.dll) are also different in this respect since they are different dlls so using a library compiled in release mode with an executable compiled in debug mode can cause problems. Keep in mind that many things will appear to work just fine when you mix C runtime libraries - it's just when you pass CRT resources around that you'll run into issues. It's also important to remember that statically linking to a C runtime gives you a private version of that runtime that is NOT shared with any other components. For this reason, it is recommended that you only statically link against a library when you are building an executable and NOT when you are building a library.

You can check your dependencies with dumpbin if you have Visual Studio installed. Fire up a Visual Studio command prompt and use dumpbin /imports <path to library or program> to see which dynamic libraries a particular program or dll requires. The existing 32-bit HDF binaries show a dependency on msvcr80.dll (Visual C++ 8.0 / Visual Studio 2005), the VS2005 version of zlib is also linked to msvcr80.dll and szip does not require any C library functions (it's either just linked to kernel32 and uses Windows functions or it incorrectly statically links to the CRT - I'll have to check that out). The tools are an odd case - they are statically linked to HDF5 and have a private version of the C runtime that is separate from, say, msvcr80.dll but are also dynamically linked to zlib which is, in turn, linked to msvcr80.dll. This means that zlib code used by the tools and the rest of the tool code will have different CRT states.

So this is (pretty much) all well and good as long as you are using Visual Studio 2005. Problems can arise, however, when you try to use Visual Studio 2008 to build an application that links against VS2005 versions of HDF5. VS2008 will link your application against msvcr90.dll and this can cause the issues described above. Unfortunately, we do not distribute VS2008 binaries on the HDF5 downloads page but I'm sure this will be rectified soon.

We're looking into the library issues here and we'll have more information in the future, but I think a short term roadmap for us could be:

1) We need to offer Visual Studio 2008 HDF5 binaries that are linked against msvcr90.dll.

2) Our binaries (HDF5, szip, zlib) need to include debug libraries so that users can properly link when creating debug builds.

3) We need to educate our Windows users about these issues, probably via a README document in the distribution.

4) Our static tools should be 100% statically linked.

Also, in general, you should use the official Windows CRT installers to install the required C runtime, instead of just copying dlls around. You can find the proper installers on Microsoft's website (links which will surely go stale soon provided below). I have no idea why the CRTs aren't just considered a normal part of a Windows installation and maintained accordingly. You'd think that would make everyone's life easier.

Visual Studio 2005 SP1
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=200b2fd9-ae1a-4a14-984d-389c36f85647

Visual Studio 2008 SP1
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=fbee1648-7106-44a7-9649-6d9f6d58056e

I hope that helps.

Dana Robinson
Bioinformatics Software Engineer
The HDF Group
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

[derobins]

On yet another final note, I installed a fresh version of 64-bit Windows 7
Professional in VMWare and attempted to duplicate Dr. X's original problem
using the 32-bit VS2005 versions of HDF5, szip and zlib binaries off The HDF
Group's website. Here's what I found:

1) msvcr90.dll and two versions of msvcr80.dll are installed by default in
Windows 7 Pro. I'm not sure which version Dr. X was using, but it appears
that Win7 now deploys with a few C runtimes installed. This is a plain Win7
install and not an OEM version that installs other stuff so they are not
there because the trial version of Donald Duck's Math Blaster or AOL
installed them. It could be that some versions of Win7 (Home?) don't have
them installed.

2) I was able to run executables out of the bin directory as long as the
zlib and szip dlls were in the directory or on the path. The intent of the
executables in the bin directory is to be 100% statically linked though they
are still dynamically linked to the compression libraries (they also need
msvcr##.dll since zlib is dynamically linked to that but see point 1,
above). We would like to change this but szip and zlib have different
licensing than HDF5 so we need to work those issues out, first.

3) Re: my earlier post - The szip binaries are statically linked to the C
runtime, which is not a good idea. I'll see about getting this changed.

We clearly have to do some work to get our versioning ducks in a row and
hopefully we'll get that set up soon. I have some other ideas about the
dynamic libraries that I've been kicking around so maybe now is the time to
float those to the community as well. In particular, I'd like for our dlls
to be named "zlib90.dll", "hdf590d.dll" and so forth, which I think might
make incorrect linking more apparent.

Dana

···

On Wed, Mar 17, 2010 at 2:13 PM, Michael Jackson < mike.jackson@bluequartz.net> wrote:

One final note. All the git stuff _should_ be now or very soon in the
official SVN repo of "The HDFGroup". A few of us setup the gitorious site to
collaborate easily on the "Cmakeification" of HDF5. Those changes will be
pushed back to the official SVN repo "real soon now". What I am getting at
is that the gitorious site will go away when the SVN gets updated. I do NOT
want to fracture the HDF5 source/community by having multiple places to
download HDF5 source codes.

Dr. X. I am glad everything worked for you. Alternatively you can download
an MSys+Git installer from google code <
http://code.google.com/p/msysgit/downloads/list> which is probably an
easier install for some due to NOT needing cygwin.

I actually have a "ZLib" that builds with CMake on my own machine. Not sure
if anyone is interested in that. Adding that would make HDF5 completely
configurable with CMake. Just a thought. Then again, that leads to
maintenance nightmares ...
___________________________________________________________

Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 3:06 PM, Dr. X wrote:

Hi All,

First of all, I appreciate Michael's prompt follow up and all the git
info.
I have downloaded HDF5 git repository from
git clone git://gitorious.org/hdf5/hdf5-v18.git
szip from
git clone git://gitorious.org/hdf5/szip.git
and zlib from
http://zlib.net/zlib-1.2.4.tar.gz

If you don't know how to use git. Please install cygwin on your windows
machine and start a terminal before you copy and paste above statements.
Certainly create hosting directories for HDF5, SZIP, and ZLIB is mandatory.
Follow Michael and Dana's note, I want to take no chance and recompile szip
and zlib from the source as well.

Build szip and zlib first:
Use cmake to build szip and zlib-1.2.4\projects\vissualc6\zlib.dsw to
build zlib. Visual Studio 2008 will automatically convert zlib.dsw to *.sln.
And do remember to create new x64 configurations if your machine is 64bit.
Both packages (DLL Release and LIB Release) were built fine (without any
L4096 warning on /NODEFAULTLIB... :)).

Then use cmake to deal with hdf5. After locating all the right places for
zlib and szip libraries and include directories, you are gold. I built hdf5
with HDF5_ENABLE_SZIP_SUPPORT and HDF5_ENABLE_Z_LIB_SUPPORT enabled for my
own testing. I guess these two flags allow the created hdf5 library to be
usefully without the companion of additional szip and zlib libraries.

Note, zip_perf project will fail on Windows due to gettimeofday() and
S_IRWXU dependencies.

At least all the hdf5 .exe are running fine. Best wishes to all Windows
64bit users. And thanks Michael and Dana.

Best,
xunlei

On 3/17/2010 12:50 PM, Michael Jackson wrote:

Great Write up. I'll just add my own experiences and some suggestions to
make our (the developers USING hdf5) easier.
I am mainly an OS X Developer and do Windows development due to customer
constraints so that is the perspective that this comes from.
RULE #1: Always build ALL your libraries with the SAME EXACT version of
Visual Studio, down to the Service Pack AND Security Patch Level.
RULE #2: NEVER break rule #1.

Following those rules will allow you to have a much easier time with
debugging/running/compiling on windows. What do I mean by exact versions?
Each library that you are going to compile into your application MUST be
linked against the same version of the C/C++ runtime libraries with respect
to the following:
Single/Multi Threaded
Static/Dynamic Linked to the Runtimes
Debug/Release Versions of each runtime.

Again, following the above will help the developer have a much easier
time with Visual Studio Development.

If you get link warnings about adding "NODEFAULTLIB" then you are NOT
following the above rules and nice random crashes and bugs WILL pop up in
your program.

What can HDFGroup do to Help? PLEASE PLEASE PLEASE list the exact version
of Visual Studio used to compile the binaries. This should be something like
Visual Studio 2008 SP1 (9.0.xxx.xxx SPI) or if the original Visual Studio
2008 was used, then 9.0.xxx.xxx RTM). These values can be found by going to
"Help->About Visual Studio". Also explicitly list HOW they were compiled
with respect to Single/MultiThreaded, Static/Dynamic Runtime linkage,
Dynamic/Static libraries, Debug/Release compiles.

Having this information on the download page will help alleviate help
emails from confused users.

Again, these are my experiences, Yours may vary. Following those rules
helps keep things simple for Visual Studio Development.
___________________________________________________________
Mike Jackson www.bluequartz.net
Principal Software Engineer mike.jackson@bluequartz.net
BlueQuartz Software Dayton, Ohio

On Mar 17, 2010, at 12:29 PM, Dana Robinson wrote:

I don't know if this solves people's linking problems but I thought I'd

weigh in with what I know about Windows C runtime (CRT) linking.

From what I understand, it boils down to this:

Every library and executable in your application should use the same
instance of the C runtime library.

The reason for this is that separate dlls have a separate internal state
and thus different ideas about things like valid file handles and allocated
memory. For example, if you malloc() some memory in a library linked to
msvcr80.dll and then try to realloc() or free() it in a program linked to
msvcr90.dll, you will have problems since msvcr90 doesn't know about the
first allocation's internal memory handles (writing to memory allocated
using a different C runtime should work fine, though, since the underlying
internal memory handles are not being manipulated). Debug libraries (e.g.
msvcr80d.dll) are also different in this respect since they are different
dlls so using a library compiled in release mode with an executable compiled
in debug mode can cause problems. Keep in mind that many things will appear
to work just fine when you mix C runtime libraries - it's just when you pass
CRT resources around that you'll run into issues. It's also important to
remember that statically linking to a C runtime gives you a private version
of that runtime that is NOT shared with any other components. For this
reason, it is recommended that you only statically link against a library
when you are building an executable and NOT when you are building a library.

You can check your dependencies with dumpbin if you have Visual Studio
installed. Fire up a Visual Studio command prompt and use dumpbin /imports
<path to library or program> to see which dynamic libraries a particular
program or dll requires. The existing 32-bit HDF binaries show a dependency
on msvcr80.dll (Visual C++ 8.0 / Visual Studio 2005), the VS2005 version of
zlib is also linked to msvcr80.dll and szip does not require any C library
functions (it's either just linked to kernel32 and uses Windows functions or
it incorrectly statically links to the CRT - I'll have to check that out).
The tools are an odd case - they are statically linked to HDF5 and have a
private version of the C runtime that is separate from, say, msvcr80.dll but
are also dynamically linked to zlib which is, in turn, linked to
msvcr80.dll. This means that zlib code used by the tools and the rest of
the tool code will have different CRT states.

So this is (pretty much) all well and good as long as you are using
Visual Studio 2005. Problems can arise, however, when you try to use Visual
Studio 2008 to build an application that links against VS2005 versions of
HDF5. VS2008 will link your application against msvcr90.dll and this can
cause the issues described above. Unfortunately, we do not distribute
VS2008 binaries on the HDF5 downloads page but I'm sure this will be
rectified soon.

We're looking into the library issues here and we'll have more
information in the future, but I think a short term roadmap for us could be:

1) We need to offer Visual Studio 2008 HDF5 binaries that are linked
against msvcr90.dll.

2) Our binaries (HDF5, szip, zlib) need to include debug libraries so
that users can properly link when creating debug builds.

3) We need to educate our Windows users about these issues, probably via
a README document in the distribution.

4) Our static tools should be 100% statically linked.

Also, in general, you should use the official Windows CRT installers to
install the required C runtime, instead of just copying dlls around. You
can find the proper installers on Microsoft's website (links which will
surely go stale soon provided below). I have no idea why the CRTs aren't
just considered a normal part of a Windows installation and maintained
accordingly. You'd think that would make everyone's life easier.

Visual Studio 2005 SP1

http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=200b2fd9-ae1a-4a14-984d-389c36f85647

Visual Studio 2008 SP1

http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=fbee1648-7106-44a7-9649-6d9f6d58056e

I hope that helps.

Dana Robinson
Bioinformatics Software Engineer
The HDF Group
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________

Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

[mike.jackson]

please don't do that..

  Boost does this and absolutely wreaks havoc on things like CMake and other build tools that are relying on a library naming scheme NOT changing. In particular if you are going to switch up to CMake in the near future you are going to have to sync with CMake for releases so that FindHDF5.cmake can stay constantly updated with all the new flavors of Visual Studio, GCC, ICC, Absoft, Borland that come out. Read back through some of the CMake archives and look for things like "CMake can't find boost, but I have it installed". Messages like that hit about once a month because Boost seems to want to change their naming scheme at their leisure. Now look at Qt from Nokia. CMake has FindQt4.cmake which doesn't get that many updates and seems to always work because the library naming is consistent from release to release and across platforms.

   So in light of this I wold make it painfully obvious on your download site how your binaries were built. Eclipse has a "click through" page for OS X downloads that has a short blurb regarding the needed things for Eclipse on 64 Bit OS X. I suggest the HDFGroup does something like that also. Heck, you could have CMake configure a "Compiler_Used.txt" file for each build with a description of how the binaries were built.

  Thanks for listening.

···

___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 3:51 PM, Dana Robinson wrote:
-----8<---------------

In particular, I'd like for our dlls to be named "zlib90.dll", "hdf590d.dll" and so forth, which I think might make incorrect linking more apparent.

-----8<---------------

[derobins]

Good point. I hadn't thought of that. I'll leave them unchanged.

Dana

···

On Wed, Mar 17, 2010 at 3:08 PM, Michael Jackson < mike.jackson@bluequartz.net> wrote:

please don't do that..

Boost does this and absolutely wreaks havoc on things like CMake and other
build tools that are relying on a library naming scheme NOT changing. In
particular if you are going to switch up to CMake in the near future you are
going to have to sync with CMake for releases so that FindHDF5.cmake can
stay constantly updated with all the new flavors of Visual Studio, GCC, ICC,
Absoft, Borland that come out. Read back through some of the CMake archives
and look for things like "CMake can't find boost, but I have it installed".
Messages like that hit about once a month because Boost seems to want to
change their naming scheme at their leisure. Now look at Qt from Nokia.
CMake has FindQt4.cmake which doesn't get that many updates and seems to
always work because the library naming is consistent from release to release
and across platforms.

So in light of this I wold make it painfully obvious on your download site
how your binaries were built. Eclipse has a "click through" page for OS X
downloads that has a short blurb regarding the needed things for Eclipse on
64 Bit OS X. I suggest the HDFGroup does something like that also. Heck, you
could have CMake configure a "Compiler_Used.txt" file for each build with a
description of how the binaries were built.

Thanks for listening.

___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 3:51 PM, Dana Robinson wrote:
-----8<---------------

In particular, I'd like for our dlls to be named "zlib90.dll",

"hdf590d.dll" and so forth, which I think might make incorrect linking more
apparent.

-----8<---------------

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

[Mitchell_Scott_AES]

I'd like to agree that appending the 'd' to the debug libraries name is a bad thing <tm>. It played particular havoc with the HDF5DotNet library because it links the hdf DLLs in code:

[DllImport("hdf5dll.dll",
           CharSet=CharSet::Auto,
           CallingConvention=CallingConvention::StdCall)]

And HDF5DotNet don't work when you've attached the debug libraries and they're named "hdf5ddll.dll".

Scott

···

-----Original Message-----
From: hdf-forum-bounces@hdfgroup.org [mailto:hdf-forum-bounces@hdfgroup.org]
On Behalf Of Michael Jackson
Sent: Wednesday, March 17, 2010 4:09 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] HDF5-1.8.4 patch1 on Windows7 64bit

please don't do that..

  Boost does this and absolutely wreaks havoc on things like CMake and
other build tools that are relying on a library naming scheme NOT
changing. In particular if you are going to switch up to CMake in the
near future you are going to have to sync with CMake for releases so
that FindHDF5.cmake can stay constantly updated with all the new
flavors of Visual Studio, GCC, ICC, Absoft, Borland that come out.
Read back through some of the CMake archives and look for things like
"CMake can't find boost, but I have it installed". Messages like that
hit about once a month because Boost seems to want to change their
naming scheme at their leisure. Now look at Qt from Nokia. CMake has
FindQt4.cmake which doesn't get that many updates and seems to always
work because the library naming is consistent from release to release
and across platforms.

   So in light of this I wold make it painfully obvious on your
download site how your binaries were built. Eclipse has a "click
through" page for OS X downloads that has a short blurb regarding the
needed things for Eclipse on 64 Bit OS X. I suggest the HDFGroup does
something like that also. Heck, you could have CMake configure a
"Compiler_Used.txt" file for each build with a description of how the
binaries were built.

  Thanks for listening.
___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 3:51 PM, Dana Robinson wrote:
-----8<---------------
> In particular, I'd like for our dlls to be named "zlib90.dll",
> "hdf590d.dll" and so forth, which I think might make incorrect
> linking more apparent.
-----8<---------------

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

This e-mail and any files transmitted with it may be proprietary and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the sender.
Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of ITT Corporation. The recipient should check this e-mail and any attachments for the presence of viruses. ITT accepts no liability for any damage caused by any virus transmitted by this e-mail.

[mike.jackson]

Funny, I'll argue the OTHER way. If we add the "d" or "_debug" to the libraries, then you can have BOTH debug and Release installed at the same time. CMake based projects have the logic to distinguish between them and setup the Visual Studio Project (and others like makefiles) so that the linking occurs correctly. My proposal would be to either add an Option to the CMake based system to allow for the tagging of debug libraries in the name (Probably already does this, I think.) or to generate a small code snippet that could be included in .Net projects that has the correct name of the library (if that is possible). I'm open to any other suggestions.

···

___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 4:21 PM, Mitchell, Scott - IS wrote:

I'd like to agree that appending the 'd' to the debug libraries name is a bad thing <tm>. It played particular havoc with the HDF5DotNet library because it links the hdf DLLs in code:

[DllImport("hdf5dll.dll",
          CharSet=CharSet::Auto,
          CallingConvention=CallingConvention::StdCall)]

And HDF5DotNet don't work when you've attached the debug libraries and they're named "hdf5ddll.dll".

Scott

-----Original Message-----
From: hdf-forum-bounces@hdfgroup.org [mailto:hdf-forum-bounces@hdfgroup.org]
On Behalf Of Michael Jackson
Sent: Wednesday, March 17, 2010 4:09 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] HDF5-1.8.4 patch1 on Windows7 64bit

please don't do that..

Boost does this and absolutely wreaks havoc on things like CMake and
other build tools that are relying on a library naming scheme NOT
changing. In particular if you are going to switch up to CMake in the
near future you are going to have to sync with CMake for releases so
that FindHDF5.cmake can stay constantly updated with all the new
flavors of Visual Studio, GCC, ICC, Absoft, Borland that come out.
Read back through some of the CMake archives and look for things like
"CMake can't find boost, but I have it installed". Messages like that
hit about once a month because Boost seems to want to change their
naming scheme at their leisure. Now look at Qt from Nokia. CMake has
FindQt4.cmake which doesn't get that many updates and seems to always
work because the library naming is consistent from release to release
and across platforms.

  So in light of this I wold make it painfully obvious on your
download site how your binaries were built. Eclipse has a "click
through" page for OS X downloads that has a short blurb regarding the
needed things for Eclipse on 64 Bit OS X. I suggest the HDFGroup does
something like that also. Heck, you could have CMake configure a
"Compiler_Used.txt" file for each build with a description of how the
binaries were built.

Thanks for listening.
___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 3:51 PM, Dana Robinson wrote:
-----8<---------------

In particular, I'd like for our dlls to be named "zlib90.dll",
"hdf590d.dll" and so forth, which I think might make incorrect
linking more apparent.

-----8<---------------

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

This e-mail and any files transmitted with it may be proprietary and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the sender.
Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of ITT Corporation. The recipient should check this e-mail and any attachments for the presence of viruses. ITT accepts no liability for any damage caused by any virus transmitted by this e-mail.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

[werner]

Does CMake support putting release/debug into separate directories? That is how
I organize libraries in my case:

lib/arch-msys-Optimize/libhdf5.dll
lib/arch-msys-Debug/libhdf5.dll
lib/arch-linux64-Optimize/hdf5.so
lib/arch-linux64-Debug/hdf5.so
lib/arch-linux32-Optimize/hdf5.so
lib/arch-linux32-Debug/hdf5.so

and similar for other platforms. Works well in my experiences on all systems,
I haven't worked with .net & co.

  Werner

···

On Wed, 17 Mar 2010 15:28:22 -0500, Michael Jackson <mike.jackson@bluequartz.net> wrote:

Funny, I'll argue the OTHER way. If we add the "d" or "_debug" to the
libraries, then you can have BOTH debug and Release installed at the
same time. CMake based projects have the logic to distinguish between
them and setup the Visual Studio Project (and others like makefiles)
so that the linking occurs correctly. My proposal would be to either
add an Option to the CMake based system to allow for the tagging of
debug libraries in the name (Probably already does this, I think.) or
to generate a small code snippet that could be included in .Net
projects that has the correct name of the library (if that is
possible). I'm open to any other suggestions.
___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 4:21 PM, Mitchell, Scott - IS wrote:

I'd like to agree that appending the 'd' to the debug libraries name
is a bad thing <tm>. It played particular havoc with the HDF5DotNet
library because it links the hdf DLLs in code:

[DllImport("hdf5dll.dll",
          CharSet=CharSet::Auto,
          CallingConvention=CallingConvention::StdCall)]

And HDF5DotNet don't work when you've attached the debug libraries
and they're named "hdf5ddll.dll".

Scott

-----Original Message-----
From: hdf-forum-bounces@hdfgroup.org [mailto:hdf-forum-bounces@hdfgroup.org
]
On Behalf Of Michael Jackson
Sent: Wednesday, March 17, 2010 4:09 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] HDF5-1.8.4 patch1 on Windows7 64bit

please don't do that..

Boost does this and absolutely wreaks havoc on things like CMake and
other build tools that are relying on a library naming scheme NOT
changing. In particular if you are going to switch up to CMake in the
near future you are going to have to sync with CMake for releases so
that FindHDF5.cmake can stay constantly updated with all the new
flavors of Visual Studio, GCC, ICC, Absoft, Borland that come out.
Read back through some of the CMake archives and look for things
like
"CMake can't find boost, but I have it installed". Messages like that
hit about once a month because Boost seems to want to change their
naming scheme at their leisure. Now look at Qt from Nokia. CMake has
FindQt4.cmake which doesn't get that many updates and seems to always
work because the library naming is consistent from release to release
and across platforms.

  So in light of this I wold make it painfully obvious on your
download site how your binaries were built. Eclipse has a "click
through" page for OS X downloads that has a short blurb regarding the
needed things for Eclipse on 64 Bit OS X. I suggest the HDFGroup does
something like that also. Heck, you could have CMake configure a
"Compiler_Used.txt" file for each build with a description of how the
binaries were built.

Thanks for listening.
___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 3:51 PM, Dana Robinson wrote:
-----8<---------------

In particular, I'd like for our dlls to be named "zlib90.dll",
"hdf590d.dll" and so forth, which I think might make incorrect
linking more apparent.

-----8<---------------

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

This e-mail and any files transmitted with it may be proprietary and
are intended solely for the use of the individual or entity to whom
they are addressed. If you have received this e-mail in error please
notify the sender.
Please note that any views or opinions presented in this e-mail are
solely those of the author and do not necessarily represent those of
ITT Corporation. The recipient should check this e-mail and any
attachments for the presence of viruses. ITT accepts no liability
for any damage caused by any virus transmitted by this e-mail.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

[derobins]

And actually, my logic behind the numbering (e.g. hdf590.dll) is to
distinguish msvcr## linkage and not an HDF version number. This should be
fairly stable over time and I believe that VS2010 even keeps the same msvcr
version as VS2008 (9.0).

As for .NET interop, I'd imagine that you'll have the same problems that C
users have if your libraries are linked to different runtime libraries and
so having the d or 90 suffix would help ensure that everything is correct
for your application.

I'm glad this has come up. I'd really like to help get the linkage issues
ironed out in a way that solves problems without adding extra headaches.

Dana

···

On Wed, Mar 17, 2010 at 3:28 PM, Michael Jackson < mike.jackson@bluequartz.net> wrote:

Funny, I'll argue the OTHER way. If we add the "d" or "_debug" to the
libraries, then you can have BOTH debug and Release installed at the same
time. CMake based projects have the logic to distinguish between them and
setup the Visual Studio Project (and others like makefiles) so that the
linking occurs correctly. My proposal would be to either add an Option to
the CMake based system to allow for the tagging of debug libraries in the
name (Probably already does this, I think.) or to generate a small code
snippet that could be included in .Net projects that has the correct name of
the library (if that is possible). I'm open to any other suggestions.
___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 4:21 PM, Mitchell, Scott - IS wrote:

I'd like to agree that appending the 'd' to the debug libraries name is a

bad thing <tm>. It played particular havoc with the HDF5DotNet library
because it links the hdf DLLs in code:

[DllImport("hdf5dll.dll",
         CharSet=CharSet::Auto,
         CallingConvention=CallingConvention::StdCall)]

And HDF5DotNet don't work when you've attached the debug libraries and
they're named "hdf5ddll.dll".

Scott

-----Original Message-----

From: hdf-forum-bounces@hdfgroup.org [mailto:
hdf-forum-bounces@hdfgroup.org]
On Behalf Of Michael Jackson
Sent: Wednesday, March 17, 2010 4:09 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] HDF5-1.8.4 patch1 on Windows7 64bit

please don't do that..

Boost does this and absolutely wreaks havoc on things like CMake and
other build tools that are relying on a library naming scheme NOT
changing. In particular if you are going to switch up to CMake in the
near future you are going to have to sync with CMake for releases so
that FindHDF5.cmake can stay constantly updated with all the new
flavors of Visual Studio, GCC, ICC, Absoft, Borland that come out.
Read back through some of the CMake archives and look for things like
"CMake can't find boost, but I have it installed". Messages like that
hit about once a month because Boost seems to want to change their
naming scheme at their leisure. Now look at Qt from Nokia. CMake has
FindQt4.cmake which doesn't get that many updates and seems to always
work because the library naming is consistent from release to release
and across platforms.

So in light of this I wold make it painfully obvious on your
download site how your binaries were built. Eclipse has a "click
through" page for OS X downloads that has a short blurb regarding the
needed things for Eclipse on 64 Bit OS X. I suggest the HDFGroup does
something like that also. Heck, you could have CMake configure a
"Compiler_Used.txt" file for each build with a description of how the
binaries were built.

Thanks for listening.
___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 3:51 PM, Dana Robinson wrote:
-----8<---------------

In particular, I'd like for our dlls to be named "zlib90.dll",
"hdf590d.dll" and so forth, which I think might make incorrect
linking more apparent.

-----8<---------------

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

This e-mail and any files transmitted with it may be proprietary and are
intended solely for the use of the individual or entity to whom they are
addressed. If you have received this e-mail in error please notify the
sender.
Please note that any views or opinions presented in this e-mail are solely
those of the author and do not necessarily represent those of ITT
Corporation. The recipient should check this e-mail and any attachments for
the presence of viruses. ITT accepts no liability for any damage caused by
any virus transmitted by this e-mail.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

[mike.jackson]

Cmake basically works by having a single configuration per build directory so you set the install directory for debug and release which is perfectly fine. The nice thing about Cmake is that the FindHDF5.Cmake will use an environment variable to help find the libraries more easily. With separate directories for debug and release we would have to teach the Cmake code to look for maybe two different env vars instead of just one.

Mike Jackson

···

Sent from my iPod

On Mar 17, 2010, at 17:18, "Werner Benger" <werner@cct.lsu.edu> wrote:

Does CMake support putting release/debug into separate directories? That is how
I organize libraries in my case:

lib/arch-msys-Optimize/libhdf5.dll
lib/arch-msys-Debug/libhdf5.dll
lib/arch-linux64-Optimize/hdf5.so
lib/arch-linux64-Debug/hdf5.so
lib/arch-linux32-Optimize/hdf5.so
lib/arch-linux32-Debug/hdf5.so

and similar for other platforms. Works well in my experiences on all systems,
I haven't worked with .net & co.

   Werner

On Wed, 17 Mar 2010 15:28:22 -0500, Michael Jackson <mike.jackson@bluequartz.net > > wrote:

Funny, I'll argue the OTHER way. If we add the "d" or "_debug" to the
libraries, then you can have BOTH debug and Release installed at the
same time. CMake based projects have the logic to distinguish between
them and setup the Visual Studio Project (and others like makefiles)
so that the linking occurs correctly. My proposal would be to either
add an Option to the CMake based system to allow for the tagging of
debug libraries in the name (Probably already does this, I think.) or
to generate a small code snippet that could be included in .Net
projects that has the correct name of the library (if that is
possible). I'm open to any other suggestions.
___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 4:21 PM, Mitchell, Scott - IS wrote:

I'd like to agree that appending the 'd' to the debug libraries name
is a bad thing <tm>. It played particular havoc with the HDF5DotNet
library because it links the hdf DLLs in code:

[DllImport("hdf5dll.dll",
         CharSet=CharSet::Auto,
         CallingConvention=CallingConvention::StdCall)]

And HDF5DotNet don't work when you've attached the debug libraries
and they're named "hdf5ddll.dll".

Scott

-----Original Message-----
From: hdf-forum-bounces@hdfgroup.org [mailto:hdf-forum-bounces@hdfgroup.org
]
On Behalf Of Michael Jackson
Sent: Wednesday, March 17, 2010 4:09 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] HDF5-1.8.4 patch1 on Windows7 64bit

please don't do that..

Boost does this and absolutely wreaks havoc on things like CMake and
other build tools that are relying on a library naming scheme NOT
changing. In particular if you are going to switch up to CMake in the
near future you are going to have to sync with CMake for releases so
that FindHDF5.cmake can stay constantly updated with all the new
flavors of Visual Studio, GCC, ICC, Absoft, Borland that come out.
Read back through some of the CMake archives and look for things
like
"CMake can't find boost, but I have it installed". Messages like that
hit about once a month because Boost seems to want to change their
naming scheme at their leisure. Now look at Qt from Nokia. CMake has
FindQt4.cmake which doesn't get that many updates and seems to always
work because the library naming is consistent from release to release
and across platforms.

So in light of this I wold make it painfully obvious on your
download site how your binaries were built. Eclipse has a "click
through" page for OS X downloads that has a short blurb regarding the
needed things for Eclipse on 64 Bit OS X. I suggest the HDFGroup does
something like that also. Heck, you could have CMake configure a
"Compiler_Used.txt" file for each build with a description of how the
binaries were built.

Thanks for listening.
___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 3:51 PM, Dana Robinson wrote:
-----8<---------------

In particular, I'd like for our dlls to be named "zlib90.dll",
"hdf590d.dll" and so forth, which I think might make incorrect
linking more apparent.

-----8<---------------

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

This e-mail and any files transmitted with it may be proprietary and
are intended solely for the use of the individual or entity to whom
they are addressed. If you have received this e-mail in error please
notify the sender.
Please note that any views or opinions presented in this e-mail are
solely those of the author and do not necessarily represent those of
ITT Corporation. The recipient should check this e-mail and any
attachments for the presence of viruses. ITT accepts no liability
for any damage caused by any virus transmitted by this e-mail.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

--
___________________________________________________________________________

Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

[mike.jackson]

As an example of "extreme" naming take a look at <http://www.boost.org/doc/libs/1_42_0/more/getting_started/windows.html > in section 6.3 Library Naming. This gets dicey quick as not every option is supported on every OS.

   Also, MS may be keeping the same version of the C/C++ runtimes but I have a hard time believing that the C/C++ runtimes between VS2008 and VS2010 are going to be interchangeable ( glass is half-empty attitude) but I would be happy to be proven wrong.

···

___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 5:30 PM, Dana Robinson wrote:

And actually, my logic behind the numbering (e.g. hdf590.dll) is to distinguish msvcr## linkage and not an HDF version number. This should be fairly stable over time and I believe that VS2010 even keeps the same msvcr version as VS2008 (9.0).

As for .NET interop, I'd imagine that you'll have the same problems that C users have if your libraries are linked to different runtime libraries and so having the d or 90 suffix would help ensure that everything is correct for your application.

I'm glad this has come up. I'd really like to help get the linkage issues ironed out in a way that solves problems without adding extra headaches.

Dana

On Wed, Mar 17, 2010 at 3:28 PM, Michael Jackson <mike.jackson@bluequartz.net > > wrote:
Funny, I'll argue the OTHER way. If we add the "d" or "_debug" to the libraries, then you can have BOTH debug and Release installed at the same time. CMake based projects have the logic to distinguish between them and setup the Visual Studio Project (and others like makefiles) so that the linking occurs correctly. My proposal would be to either add an Option to the CMake based system to allow for the tagging of debug libraries in the name (Probably already does this, I think.) or to generate a small code snippet that could be included in .Net projects that has the correct name of the library (if that is possible). I'm open to any other suggestions.
___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 4:21 PM, Mitchell, Scott - IS wrote:

I'd like to agree that appending the 'd' to the debug libraries name is a bad thing <tm>. It played particular havoc with the HDF5DotNet library because it links the hdf DLLs in code:

[DllImport("hdf5dll.dll",
         CharSet=CharSet::Auto,
         CallingConvention=CallingConvention::StdCall)]

And HDF5DotNet don't work when you've attached the debug libraries and they're named "hdf5ddll.dll".

Scott

-----Original Message-----
From: hdf-forum-bounces@hdfgroup.org [mailto:hdf-forum-bounces@hdfgroup.org]
On Behalf Of Michael Jackson
Sent: Wednesday, March 17, 2010 4:09 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] HDF5-1.8.4 patch1 on Windows7 64bit

please don't do that..

Boost does this and absolutely wreaks havoc on things like CMake and
other build tools that are relying on a library naming scheme NOT
changing. In particular if you are going to switch up to CMake in the
near future you are going to have to sync with CMake for releases so
that FindHDF5.cmake can stay constantly updated with all the new
flavors of Visual Studio, GCC, ICC, Absoft, Borland that come out.
Read back through some of the CMake archives and look for things like
"CMake can't find boost, but I have it installed". Messages like that
hit about once a month because Boost seems to want to change their
naming scheme at their leisure. Now look at Qt from Nokia. CMake has
FindQt4.cmake which doesn't get that many updates and seems to always
work because the library naming is consistent from release to release
and across platforms.

So in light of this I wold make it painfully obvious on your
download site how your binaries were built. Eclipse has a "click
through" page for OS X downloads that has a short blurb regarding the
needed things for Eclipse on 64 Bit OS X. I suggest the HDFGroup does
something like that also. Heck, you could have CMake configure a
"Compiler_Used.txt" file for each build with a description of how the
binaries were built.

Thanks for listening.
___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 17, 2010, at 3:51 PM, Dana Robinson wrote:
-----8<---------------
In particular, I'd like for our dlls to be named "zlib90.dll",
"hdf590d.dll" and so forth, which I think might make incorrect
linking more apparent.
-----8<---------------

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

This e-mail and any files transmitted with it may be proprietary and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the sender.
Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of ITT Corporation. The recipient should check this e-mail and any attachments for the presence of viruses. ITT accepts no liability for any damage caused by any virus transmitted by this e-mail.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

[byrn]

I think have a handle on this runtime library problem. I was able to get consistent results to come to the following conclusion:

Users must have a runtime library version at least equal to the version used to build product. Also, it is compiler dependent; mscvr80.dll for VS2005 and mscvr90.dll for VS2008.

I keep the machines that I build on up to date with MS updates, and there was a recent security patch that advanced the version of the runtime library for VS2005. I found the same version in a redistributable download from MS called VS2008 C++ security patch. Yes it has the mscvr80.dll version needed.

I have been able to rebuild all versions of szip(removed static dependency), zlib and hdf5 1.8.4-patch1 with the BIND define that was recommended and I will be making all these versions available as soon as I get the documentation updated.

Comments?

Allen

···

As an example of "extreme" naming take a look at <http://www.boost.org/doc/libs/1_42_0/more/getting_started/windows.html
> in section 6.3 Library Naming. This gets dicey quick as not every
option is supported on every OS.

   Also, MS may be keeping the same version of the C/C++ runtimes but
I have a hard time believing that the C/C++ runtimes between VS2008
and VS2010 are going to be interchangeable ( glass is half-empty
attitude) but I would be happy to be proven wrong.

___________________________________________________________
Mike Jackson www.bluequartz.net

[mike.jackson]

As I am still learning about this myself, does this mean that I can build something with a RTM version of Visual Studio 2005 and still link against the HDF5 libraries that were built with VS 2005 SP1 + Security Update? That would seem to indicate that my application would need/use the newer c/c++ runtimes that the latest HDF5 would need? Everytime I try something like this (usually through accident) I get random crashes and other irregularities with HDF5 operation.

···

___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 22, 2010, at 10:56 AM, Allen D Byrne wrote:

I think have a handle on this runtime library problem. I was able to get consistent results to come to the following conclusion:

Users must have a runtime library version at least equal to the version used to build product. Also, it is compiler dependent; mscvr80.dll for VS2005 and mscvr90.dll for VS2008.

I keep the machines that I build on up to date with MS updates, and there was a recent security patch that advanced the version of the runtime library for VS2005. I found the same version in a redistributable download from MS called VS2008 C++ security patch. Yes it has the mscvr80.dll version needed.

I have been able to rebuild all versions of szip(removed static dependency), zlib and hdf5 1.8.4-patch1 with the BIND define that was recommended and I will be making all these versions available as soon as I get the documentation updated.

Comments?

Allen

As an example of "extreme" naming take a look at <http://www.boost.org/doc/libs/1_42_0/more/getting_started/windows.html

in section 6.3 Library Naming. This gets dicey quick as not every

option is supported on every OS.

  Also, MS may be keeping the same version of the C/C++ runtimes but
I have a hard time believing that the C/C++ runtimes between VS2008
and VS2010 are going to be interchangeable ( glass is half-empty
attitude) but I would be happy to be proven wrong.

___________________________________________________________
Mike Jackson www.bluequartz.net

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

[derobins]

Does the SP1 patch replace the msvcr80.dll file? If so, then everything
should be fine. If not, and there are two versions of msvcr80 on the system
(perhaps using SxS deployment), then you may have problems since different
dlls have different state and won't share CRT resources.

Dana

···

On Mon, Mar 22, 2010 at 9:58 AM, Michael Jackson < mike.jackson@bluequartz.net> wrote:

As I am still learning about this myself, does this mean that I can build
something with a RTM version of Visual Studio 2005 and still link against
the HDF5 libraries that were built with VS 2005 SP1 + Security Update? That
would seem to indicate that my application would need/use the newer c/c++
runtimes that the latest HDF5 would need? Everytime I try something like
this (usually through accident) I get random crashes and other
irregularities with HDF5 operation.

___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 22, 2010, at 10:56 AM, Allen D Byrne wrote:

I think have a handle on this runtime library problem. I was able to get

consistent results to come to the following conclusion:

Users must have a runtime library version at least equal to the version
used to build product. Also, it is compiler dependent; mscvr80.dll for
VS2005 and mscvr90.dll for VS2008.

I keep the machines that I build on up to date with MS updates, and there
was a recent security patch that advanced the version of the runtime library
for VS2005. I found the same version in a redistributable download from MS
called VS2008 C++ security patch. Yes it has the mscvr80.dll version needed.

I have been able to rebuild all versions of szip(removed static
dependency), zlib and hdf5 1.8.4-patch1 with the BIND define that was
recommended and I will be making all these versions available as soon as I
get the documentation updated.

Comments?

Allen

As an example of "extreme" naming take a look at <

http://www.boost.org/doc/libs/1_42_0/more/getting_started/windows.html

in section 6.3 Library Naming. This gets dicey quick as not every

option is supported on every OS.

Also, MS may be keeping the same version of the C/C++ runtimes but
I have a hard time believing that the C/C++ runtimes between VS2008
and VS2010 are going to be interchangeable ( glass is half-empty
attitude) but I would be happy to be proven wrong.

___________________________________________________________
Mike Jackson www.bluequartz.net

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

[byrn]

I think that while you can build it (I haven't tested this) it will not run unless the newer version is available. (Also I assumed that you would be using the BIND define in your application?). Of course this is just a guess and would welcome any contrary proofs?

Allen

···

As I am still learning about this myself, does this mean that I can
build something with a RTM version of Visual Studio 2005 and still
link against the HDF5 libraries that were built with VS 2005 SP1 +
Security Update? That would seem to indicate that my application would
need/use the newer c/c++ runtimes that the latest HDF5 would need?
Everytime I try something like this (usually through accident) I get
random crashes and other irregularities with HDF5 operation.
___________________________________________________________
Mike Jackson www.bluequartz.net

On Mar 22, 2010, at 10:56 AM, Allen D Byrne wrote:

> I think have a handle on this runtime library problem. I was able to
> get consistent results to come to the following conclusion:
>
> Users must have a runtime library version at least equal to the
> version used to build product. Also, it is compiler dependent;
> mscvr80.dll for VS2005 and mscvr90.dll for VS2008.
>
> I keep the machines that I build on up to date with MS updates, and
> there was a recent security patch that advanced the version of the
> runtime library for VS2005. I found the same version in a
> redistributable download from MS called VS2008 C++ security patch.
> Yes it has the mscvr80.dll version needed.
>
> I have been able to rebuild all versions of szip(removed static
> dependency), zlib and hdf5 1.8.4-patch1 with the BIND define that
> was recommended and I will be making all these versions available as
> soon as I get the documentation updated.
>
> Comments?
>
> Allen
>
>> As an example of "extreme" naming take a look at <http://www.boost.org/doc/libs/1_42_0/more/getting_started/windows.html
>>> in section 6.3 Library Naming. This gets dicey quick as not every
>> option is supported on every OS.
>>
>> Also, MS may be keeping the same version of the C/C++ runtimes but
>> I have a hard time believing that the C/C++ runtimes between VS2008
>> and VS2010 are going to be interchangeable ( glass is half-empty
>> attitude) but I would be happy to be proven wrong.
>>
>> ___________________________________________________________
>> Mike Jackson www.bluequartz.net
>>
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> Hdf-forum@hdfgroup.org
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org