hdf5 and ImageMagick conflicts on Windows 32-bit

Hdf-forum archives
1

Dear all,

  I need to support both hdf5 and tiff files in the same program, and I observed a conflict in hdf5 and ImageMagick include files on Windows 7, when compiling with VC10 for 32-bit architecture.
  In this case, H5public.h defines "typedef int ssize_t;" whereas magick-config.h defines "typedef long ssize_t;". Indeed both "int" and "long" are 4-byte variables, but anyway the compiler complaints that there is a type redefinition with the wrong data type.
  You can observe it even with a stupid "hello world" program, just including the two lines:

#include <hdf5.h>
#include <wand/MagickWand.h>

  A possible patch: I changed in H5public.h the ssize_t definition from

/* Define the ssize_t type if it not is defined */
#if H5_SIZEOF_SSIZE_T==0
/* Undefine this size, we will re-define it in one of the sections below */
#undef H5_SIZEOF_SSIZE_T
#if H5_SIZEOF_SIZE_T==H5_SIZEOF_INT
typedef int ssize_t;
# define H5_SIZEOF_SSIZE_T H5_SIZEOF_INT
#elif H5_SIZEOF_SIZE_T==H5_SIZEOF_LONG
typedef long ssize_t;
# define H5_SIZEOF_SSIZE_T H5_SIZEOF_LONG
#elif H5_SIZEOF_SIZE_T==H5_SIZEOF_LONG_LONG
typedef long long ssize_t;
# define H5_SIZEOF_SSIZE_T H5_SIZEOF_LONG_LONG
#else /* Can't find matching type for ssize_t */
# error "nothing appropriate for ssize_t"
#endif

to

/* Define the ssize_t type if it not is defined */
#if H5_SIZEOF_SSIZE_T==0
/* Undefine this size, we will re-define it in one of the sections below */
#undef H5_SIZEOF_SSIZE_T
#if H5_SIZEOF_SIZE_T==H5_SIZEOF_LONG_LONG
typedef long long ssize_t;
# define H5_SIZEOF_SSIZE_T H5_SIZEOF_LONG_LONG
#elif H5_SIZEOF_SIZE_T==H5_SIZEOF_LONG
typedef long ssize_t;
# define H5_SIZEOF_SSIZE_T H5_SIZEOF_LONG
#elif H5_SIZEOF_SIZE_T==H5_SIZEOF_INT
typedef int ssize_t;
# define H5_SIZEOF_SSIZE_T H5_SIZEOF_INT
#else /* Can't find matching type for ssize_t */
# error "nothing appropriate for ssize_t"
#endif

  (ie just inverting the order from INT - LONG - LONG LONG to LONG LONG - LONG - INT.)
  In this way ssize_t is of type long and there is no conflict with ImageMagick. And I think there is no real change in HDF5 too, since sizeof(int)==sizeof(long).

  First of all: did anybody ever observe any problem of this kind?
  Second question: are there comments to the patch?

  Regards,
   Andrea

···

------------------------------------------------------------
Dr. Andrea Parenti DESY (FS-EC Group)

mail: andrea.parenti@desy.de Notkestrasse 85
web: http://www.desy.de/~parenti/ D-22607 Hamburg
phone: +49 (0)40 8998 3154 Germany
------------------------------------------------------------

2

That is ONLY valid on 32 bit windows. On 64 bit windows long is 8 bytes in length. The use of "long" should just be banned, IMHO.

···

___________________________________________________________
Mike Jackson Principal Software Engineer
BlueQuartz Software Dayton, Ohio
mike.jackson@bluequartz.net www.bluequartz.net

On Dec 1, 2011, at 9:53 AM, Andrea Parenti wrote:

(ie just inverting the order from INT - LONG - LONG LONG to LONG LONG - LONG - INT.)
In this way ssize_t is of type long and there is no conflict with ImageMagick. And I think there is no real change in HDF5 too, since sizeof(int)==sizeof(long).

3

Dear Mike,

  I am sorry to contradict what you say, but what I get is
   sizeof(int) == sizeof(long) == 4
  compiling on Windows 7 (64 bits) with Visual Studio 2008. The data type size is the same when I compile a 64-bit executable and a 32-bit executable.
  In order to have an 8-byte variable I must use long long, in fact I get
   sizeof(long long) == 8

  But maybe this is compiler dependent (I'm not an expert of Visual Studio, I usually work on Linux).

   Andrea

···

------------------------------------------------------------
Dr. Andrea Parenti DESY (FS-EC Group)

mail: andrea.parenti@desy.de Notkestrasse 85
web: http://www.desy.de/~parenti/ D-22607 Hamburg
phone: +49 (0)40 8998 3154 Germany
------------------------------------------------------------

On Thu, 1 Dec 2011, Michael Jackson wrote:

That is ONLY valid on 32 bit windows. On 64 bit windows long is 8 bytes in length. The use of "long" should just be banned, IMHO.
___________________________________________________________
Mike Jackson Principal Software Engineer
BlueQuartz Software Dayton, Ohio
mike.jackson@bluequartz.net www.bluequartz.net

On Dec 1, 2011, at 9:53 AM, Andrea Parenti wrote:

(ie just inverting the order from INT - LONG - LONG LONG to LONG LONG - LONG - INT.) In this way ssize_t is of type long and there is no conflict with ImageMagick. And I think there is no real change in HDF5 too, since sizeof(int)==sizeof(long).

4

Hi all,

(VERY Red faced).
That is on OS X that there is a difference:

32 Bit:
sizeof(int): 4
sizeof(long): 4
sizeof(long long): 8

64 Bit
sizeof(int): 4
sizeof(long): 8
sizeof(long long): 8

Again, sorry for the confusion. Trying to keep it all straight is confusing. Over the years I have just stayed away from "long" where absolutely possible.

Windows is LLP64, most other platforms are LP64.

http://en.wikipedia.org/wiki/64-bit (scroll down to 64-bit data models)
http://www.unix.org/version2/whatsnew/lp64_wp.html

(ie just inverting the order from INT - LONG - LONG LONG to LONG LONG - LONG - INT.) In this way ssize_t is of type long and there is no conflict with ImageMagick. And I think there is no real change in HDF5 too, since sizeof(int)==sizeof(long).

The problem is that ssize_t appears in our public API, yet is not
standard C (it's POSIX). Typically, in a project that started on
Linux that is being ported to Windows, the Linux project will
implement a hard-coded work-around, as we have in HDF5. The problem
comes when these work-arounds collide, as they have here. Configure
and build systems like CMake wouldn't even help here since there are
essentially two choices, neither of which is standard and either of
which will make someone's upstream software unhappy.

Unfortunately, although changing the order of the #defines solves the
problem for ImageMagick, it will cause the same problem for anyone who
expected the former typedef. Since so much software exists that
builds on HDF5 and uses the public headers it is very likely that we
would break something, possibly in an important customer, so we've
decided to leave the behavior as it is. As a work-around, you'll
probably have to hack up the header file for one of the packages.

The real solution is to remove non-ANSI (or non-HDF group specific)
types from the public API to avoid this type of problem in the future.
Since this would amount to an API-breaking change, it will have to
wait for a major version change (HDF5 1.10.0) to be considered. I
will enter an issue in our bug-tracking system so that it'll be on our
radar in the future.

Thank you for bringing this to our attention!

Cheers,

Dana

···

On Fri, Dec 2, 2011 at 8:25 AM, Michael Jackson <mike.jackson@bluequartz.net> wrote:

5

(VERY Red faced).
  That is on OS X that there is a difference:

32 Bit:
sizeof(int): 4
sizeof(long): 4
sizeof(long long): 8

64 Bit
sizeof(int): 4
sizeof(long): 8
sizeof(long long): 8

Again, sorry for the confusion. Trying to keep it all straight is confusing. Over the years I have just stayed away from "long" where absolutely possible.

···

___________________________________________________________
Mike Jackson Principal Software Engineer
BlueQuartz Software Dayton, Ohio
mike.jackson@bluequartz.net www.bluequartz.net

On Dec 2, 2011, at 8:39 AM, Andrea Parenti wrote:

Dear Mike,

I am sorry to contradict what you say, but what I get is
  sizeof(int) == sizeof(long) == 4
compiling on Windows 7 (64 bits) with Visual Studio 2008. The data type size is the same when I compile a 64-bit executable and a 32-bit executable.
In order to have an 8-byte variable I must use long long, in fact I get
  sizeof(long long) == 8

But maybe this is compiler dependent (I'm not an expert of Visual Studio, I usually work on Linux).

  Andrea

------------------------------------------------------------
Dr. Andrea Parenti DESY (FS-EC Group)

mail: andrea.parenti@desy.de Notkestrasse 85
web: http://www.desy.de/~parenti/ D-22607 Hamburg
phone: +49 (0)40 8998 3154 Germany
------------------------------------------------------------

On Thu, 1 Dec 2011, Michael Jackson wrote:

That is ONLY valid on 32 bit windows. On 64 bit windows long is 8 bytes in length. The use of "long" should just be banned, IMHO.
___________________________________________________________
Mike Jackson Principal Software Engineer
BlueQuartz Software Dayton, Ohio
mike.jackson@bluequartz.net www.bluequartz.net

On Dec 1, 2011, at 9:53 AM, Andrea Parenti wrote:

(ie just inverting the order from INT - LONG - LONG LONG to LONG LONG - LONG - INT.) In this way ssize_t is of type long and there is no conflict with ImageMagick. And I think there is no real change in HDF5 too, since sizeof(int)==sizeof(long).

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org