Compile minimal example with MinGW

Hello ladies and gentlemen!

I'm trying to compile a minimal HDF5 example via MinGW on w32:

#include "hdf5.h"

int main()
{
   hid_t file_id;
   hid_t dataset_id;
   hid_t attribute_id;
   hid_t space_id;
   hssize_t nscans, npeaks;
   double minmz, maxmz;

   printf("Opening MS.abf ... ");
   file_id = H5Fopen("MS.abf", H5F_ACC_RDONLY, H5P_DEFAULT);
   if(file_id > 0)
     printf("done.\n");
   else{
     printf("error!\n");
     return 1;
   };

   dataset_id = H5Dopen(file_id, "rettime", H5P_DEFAULT);
   space_id = H5Dget_space(dataset_id);
   nscans = H5Sget_simple_extent_npoints(space_id);
   printf("Total scans: %lld.\n", nscans);
   H5Sclose(space_id);
   H5Dclose(dataset_id);

   dataset_id = H5Dopen(file_id, "specdata/mz", H5P_DEFAULT);
   space_id = H5Dget_space(dataset_id);
   npeaks = H5Sget_simple_extent_npoints(space_id);
   printf("Total MS peaks: %lld.\n", npeaks);
   H5Sclose(space_id);

   attribute_id = H5Aopen(dataset_id, "min", H5P_DEFAULT);
   H5Aread(attribute_id, H5T_NATIVE_DOUBLE, &minmz);
   H5Sclose(attribute_id);
   attribute_id = H5Aopen(dataset_id, "max", H5P_DEFAULT);
   H5Aread(attribute_id, H5T_NATIVE_DOUBLE, &maxmz);
   H5Sclose(attribute_id);
   printf("M/z range: %f..%f.\n", minmz, maxmz);

   H5Dclose(dataset_id);

   H5Fclose(file_id);

   return;
}

I'm running

>gcc test.c -I"C:\Progra~2\HDF_Group\HDF5\1.8.12\include" -L"C:\Progra~2\HDF_Group\HDF5\1.8.12\lib" -lhdf5 -otest.exe

but I get

C:\Users\paramon\AppData\Local\Temp\ccqrZOee.o:test.c:(.text+0x177): undefined reference to `H5T_NATIVE_DOUBLE_g'
C:\Users\paramon\AppData\Local\Temp\ccqrZOee.o:test.c:(.text+0x1c5): undefined reference to `H5T_NATIVE_DOUBLE_g'
c:/mingw/bin/../lib/gcc/mingw32/4.8.1/../../../../mingw32/bin/ld.exe: C:\Users\paramon\AppData\Local\Temp\ccqrZOee.o: bad reloc address 0x20 in section `.eh_frame'
c:/mingw/bin/../lib/gcc/mingw32/4.8.1/../../../../mingw32/bin/ld.exe: final link failed: Invalid operation
collect2.exe: error: ld returned 1 exit status

It seems to be some problem with H5T_NATIVE_DOUBLE or, more generally, with DLLVARs. Without them, the program compiles and runs fine.

Does anyone has experience with building HDF5-dependant code on w32? I wish to write a small demo example for my HDF5-based format; I do not have much C experience, so maybe I'm missing something really simple.

Best wishes,
Andrey Paramonov

···

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Hi Andrei,

was hdf5_EXPORTS defined when compiling HDF5?

You could check the hdf5.dll if its appropriately defined for export:

$ nm hdf5.dll | grep H5T_NATIVE_DOUBLE
000000006b60ac80 B H5T_NATIVE_DOUBLE_ALIGN_g
000000006b60abf8 B H5T_NATIVE_DOUBLE_COMP_ALIGN_g
000000006b57d6c4 D H5T_NATIVE_DOUBLE_g
000000006b60ad60 B H5T_NATIVE_DOUBLE_NEG_INF_g
000000006b60ad58 B H5T_NATIVE_DOUBLE_POS_INF_g

Werner

···

On 15.05.2014 15:14, Андрей Парамонов wrote:

Hello ladies and gentlemen!

I'm trying to compile a minimal HDF5 example via MinGW on w32:

#include "hdf5.h"

int main()
{
hid_t file_id;
hid_t dataset_id;
hid_t attribute_id;
hid_t space_id;
hssize_t nscans, npeaks;
double minmz, maxmz;

printf("Opening MS.abf ... ");
file_id = H5Fopen("MS.abf", H5F_ACC_RDONLY, H5P_DEFAULT);
if(file_id > 0)
printf("done.\n");
else{
printf("error!\n");
return 1;
};

dataset_id = H5Dopen(file_id, "rettime", H5P_DEFAULT);
space_id = H5Dget_space(dataset_id);
nscans = H5Sget_simple_extent_npoints(space_id);
printf("Total scans: %lld.\n", nscans);
H5Sclose(space_id);
H5Dclose(dataset_id);

dataset_id = H5Dopen(file_id, "specdata/mz", H5P_DEFAULT);
space_id = H5Dget_space(dataset_id);
npeaks = H5Sget_simple_extent_npoints(space_id);
printf("Total MS peaks: %lld.\n", npeaks);
H5Sclose(space_id);

attribute_id = H5Aopen(dataset_id, "min", H5P_DEFAULT);
H5Aread(attribute_id, H5T_NATIVE_DOUBLE, &minmz);
H5Sclose(attribute_id);
attribute_id = H5Aopen(dataset_id, "max", H5P_DEFAULT);
H5Aread(attribute_id, H5T_NATIVE_DOUBLE, &maxmz);
H5Sclose(attribute_id);
printf("M/z range: %f..%f.\n", minmz, maxmz);

H5Dclose(dataset_id);

H5Fclose(file_id);

return;
}

I'm running

>gcc test.c -I"C:\Progra~2\HDF_Group\HDF5\1.8.12\include" -L"C:\Progra~2\HDF_Group\HDF5\1.8.12\lib" -lhdf5 -otest.exe

but I get

C:\Users\paramon\AppData\Local\Temp\ccqrZOee.o:test.c:(.text+0x177): undefined reference to `H5T_NATIVE_DOUBLE_g'
C:\Users\paramon\AppData\Local\Temp\ccqrZOee.o:test.c:(.text+0x1c5): undefined reference to `H5T_NATIVE_DOUBLE_g'
c:/mingw/bin/../lib/gcc/mingw32/4.8.1/../../../../mingw32/bin/ld.exe: C:\Users\paramon\AppData\Local\Temp\ccqrZOee.o: bad reloc address 0x20 in section `.eh_frame'
c:/mingw/bin/../lib/gcc/mingw32/4.8.1/../../../../mingw32/bin/ld.exe: final link failed: Invalid operation
collect2.exe: error: ld returned 1 exit status

It seems to be some problem with H5T_NATIVE_DOUBLE or, more generally, with DLLVARs. Without them, the program compiles and runs fine.

Does anyone has experience with building HDF5-dependant code on w32? I wish to write a small demo example for my HDF5-based format; I do not have much C experience, so maybe I'm missing something really simple.

Best wishes,
Andrey Paramonov

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

I turns out that instead of

>gcc test.c -I"C:\Progra~2\HDF_Group\HDF5\1.8.12\include" -L"C:\Progra~2\HDF_Group\HDF5\1.8.12\lib" -lhdf5 -otest.exe

I should have specified

>gcc test.c -I"C:\Progra~2\HDF_Group\HDF5\1.8.12\include" -L"C:\Progra~2\HDF_Group\HDF5\1.8.12\bin" -lhdf5 -otest.exe

Now my program builds and works like a charm!

Thank you for your support,
Andrey Paramonov

···

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

15.05.2014 19:37, Werner Benger пишет:

Hi Andrei,

was hdf5_EXPORTS defined when compiling HDF5?

You could check the hdf5.dll if its appropriately defined for export:

$ nm hdf5.dll | grep H5T_NATIVE_DOUBLE
000000006b60ac80 B H5T_NATIVE_DOUBLE_ALIGN_g
000000006b60abf8 B H5T_NATIVE_DOUBLE_COMP_ALIGN_g
000000006b57d6c4 D H5T_NATIVE_DOUBLE_g
000000006b60ad60 B H5T_NATIVE_DOUBLE_NEG_INF_g
000000006b60ad58 B H5T_NATIVE_DOUBLE_POS_INF_g

For official 1.8.12 w32 shared binary, I get

>nm hdf5.dll | grep H5T_NATIVE_DOUBLE
C:\MinGW\bin\nm.exe: hdf5.dll: no symbols

However, Dependency Walker does list H5T_NATIVE_DOUBLE_g among exported symbols of hdf5.dll.

I wish that my minimal example can be compiled and run only with minimal work (single gcc invocation, no need for cmake etc). Is it possible?

Best wishes,
Andrey Paramonov

···

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

15.05.2014 19:37, Werner Benger пишет:

Hi Andrei,

was hdf5_EXPORTS defined when compiling HDF5?

You could check the hdf5.dll if its appropriately defined for export:

$ nm hdf5.dll | grep H5T_NATIVE_DOUBLE
000000006b60ac80 B H5T_NATIVE_DOUBLE_ALIGN_g
000000006b60abf8 B H5T_NATIVE_DOUBLE_COMP_ALIGN_g
000000006b57d6c4 D H5T_NATIVE_DOUBLE_g
000000006b60ad60 B H5T_NATIVE_DOUBLE_NEG_INF_g
000000006b60ad58 B H5T_NATIVE_DOUBLE_POS_INF_g

For official 1.8.12 w32 shared binary, I get

>nm hdf5.dll | grep H5T_NATIVE_DOUBLE
C:\MinGW\bin\nm.exe: hdf5.dll: no symbols

Hm, it seems this HDF5.dll was compiled differently then than mine. There might be a hdf5.dll.a in your version that contains those symbols instead, and in that case you would need to link against that one instead of the .dll .

However, Dependency Walker does list H5T_NATIVE_DOUBLE_g among exported symbols of hdf5.dll.

I wish that my minimal example can be compiled and run only with minimal work (single gcc invocation, no need for cmake etc). Is it possible?

I am not using cmake myself, neither the official HDF5 configure/makefile (they seem to have some issue under mingw, it might be trivial, but couldn't identify it yet). I can share you my makefiles if you're interested.

         Werner

···

On 16.05.2014 07:40, Андрей Парамонов wrote:

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

16.05.2014 12:15, Werner Benger пишет:

Hm, it seems this HDF5.dll was compiled differently then than mine.
There might be a hdf5.dll.a in your version that contains those symbols
instead, and in that case you would need to link against that one
instead of the .dll .

But the hdf5.dll from official binaries *does* contain those symbols! I can see the from Dependency Walker, and I can import and use them from Delphi code (but for the latter, I had to create a huge wrapper which I want to avoid for my minimal C example).

I am not using cmake myself, neither the official HDF5
configure/makefile (they seem to have some issue under mingw, it might
be trivial, but couldn't identify it yet). I can share you my makefiles
if you're interested.

In fact, I'd like to avoid makefiles also :wink: It seems strange that for such a minimal example one couldn't just invoke gcc directly.

Could you please share your HDF5 binaries?

Best wishes,
Andrey Paramonov

···

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Hi Andrey,

  seems my hdf5.dll is 144MB in size, so I would need to put in on some webserver to download, my email doesn't like to send such large files. Note that it's not the newest 1.8.13 but an older version, and it's compiled with mingw64. Do you really want it? I could also compile a 32bit binary as well, but I think I would need to re-install mingw 32 bit, which otherwise I don't use any more.

And yes, you could compile HDF5 also without a makefile. Mostly it's just compiling all .c files to .o, plus two special cases for H5detect and H5libsettings, which need to built as binary first and then generate some source from this. I have these rules in my own makefile for convenience, but it could also be done manually.

Poka,
     Werner

···

On 16.05.2014 10:24, Андрей Парамонов wrote:

16.05.2014 12:15, Werner Benger пишет:

Hm, it seems this HDF5.dll was compiled differently then than mine.
There might be a hdf5.dll.a in your version that contains those symbols
instead, and in that case you would need to link against that one
instead of the .dll .

But the hdf5.dll from official binaries *does* contain those symbols! I can see the from Dependency Walker, and I can import and use them from Delphi code (but for the latter, I had to create a huge wrapper which I want to avoid for my minimal C example).

I am not using cmake myself, neither the official HDF5
configure/makefile (they seem to have some issue under mingw, it might
be trivial, but couldn't identify it yet). I can share you my makefiles
if you're interested.

In fact, I'd like to avoid makefiles also :wink: It seems strange that for such a minimal example one couldn't just invoke gcc directly.

Could you please share your HDF5 binaries?

Best wishes,
Andrey Paramonov

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

16.05.2014 14:11, Werner Benger пишет:

Hi Andrey,

  seems my hdf5.dll is 144MB in size, so I would need to put in on some
webserver to download, my email doesn't like to send such large files.
Note that it's not the newest 1.8.13 but an older version, and it's
compiled with mingw64. Do you really want it?

Maybe it is possible that I go the other way around? I can download static version of HDF5 binaries from official website, and try to compile my example using it. Do you know the magic spell I should input in command line to do so?

I do not want to recompile HDF5 library because it seems non-trivial, and because I hardly believe it can change anything compared to official binaries...

Best wishes,
Andrey Paramonov

···

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

16.05.2014 14:11, Werner Benger пишет:

Hi Andrey,

  seems my hdf5.dll is 144MB in size, so I would need to put in on some
webserver to download, my email doesn't like to send such large files.
Note that it's not the newest 1.8.13 but an older version, and it's
compiled with mingw64. Do you really want it?

Maybe it is possible that I go the other way around? I can download static version of HDF5 binaries from official website, and try to compile my example using it. Do you know the magic spell I should input in command line to do so?

The official HDF5 binaries are compiled with Visual Studio, not MinGW, as far as I know. While it is C and can be linked with each other, it may need some additional work to extract those symbols from the dll to a .a file that is needed for mingw to link. It's possible, there are some instructions out there on how to make MinGW dll/.a from an existing DLL. There are tools like http://sourceforge.net/apps/trac/mingw-w64/wiki/gendef which help extracting such symbols and making them gcc-compatible, but in my experience that's all effortsome and it's just easier to compile HDF5 with mingw itself.

I do not want to recompile HDF5 library because it seems non-trivial, and because I hardly believe it can change anything compared to official binaries...

I find it rather easy and do it frequently, just not using cmake or the official configure script, which has issues under msys/mingw. I can share those makefiles, but I'd need to prepare them to work standalone for just HDF5, usually I use them as part of a larger application.

There are difference to the official binaries because mingw can put symbols into the DLL's itself, while for MSVC-compiled binaries such symbols are usually provided as external .def files if they are not part of the DLL. I'm not experienced with MS Visual Studio so don't know too well those options, but there are differences beside all basic compatibilies.

      Werner

···

On 16.05.2014 12:21, Андрей Парамонов wrote:

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

16.05.2014 14:45, Werner Benger пишет:

···

On 16.05.2014 12:21, Андрей Парамонов wrote:

I do not want to recompile HDF5 library because it seems non-trivial,
and because I hardly believe it can change anything compared to
official binaries...

I find it rather easy and do it frequently, just not using cmake or the
official configure script, which has issues under msys/mingw. I can
share those makefiles, but I'd need to prepare them to work standalone
for just HDF5, usually I use them as part of a larger application.

It would be great if you could share the makefiles, and some instructions for how to cast those on official tarball :wink:

Best wishes,
Andrey Paramonov

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Will do. Working on making them standalone for just HDF5 and will let you know as soon as it's ready.

     Werner

···

On 16.05.2014 12:46, Андрей Парамонов wrote:

16.05.2014 14:45, Werner Benger пишет:

On 16.05.2014 12:21, Андрей Парамонов wrote:

I do not want to recompile HDF5 library because it seems non-trivial,
and because I hardly believe it can change anything compared to
official binaries...

I find it rather easy and do it frequently, just not using cmake or the
official configure script, which has issues under msys/mingw. I can
share those makefiles, but I'd need to prepare them to work standalone
for just HDF5, usually I use them as part of a larger application.

It would be great if you could share the makefiles, and some instructions for how to cast those on official tarball :wink:

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

Here, use this URL for SVN checkout:

    https://subversion.assembla.com/svn/FiberHDF5/hdf5

I just verified it to work under both Linux and MinGW64.

Once checked out, just do a make in the hdf5/ subdir, in Windows/Msys use mingw32-make .

Msys make should work as well, but I didn't test it, mingw32-make is much faster.

        Werner

···

On 16.05.2014 13:04, Werner Benger wrote:

On 16.05.2014 12:46, Андрей Парамонов wrote:

16.05.2014 14:45, Werner Benger пишет:

On 16.05.2014 12:21, Андрей Парамонов wrote:

I do not want to recompile HDF5 library because it seems non-trivial,
and because I hardly believe it can change anything compared to
official binaries...

I find it rather easy and do it frequently, just not using cmake or the
official configure script, which has issues under msys/mingw. I can
share those makefiles, but I'd need to prepare them to work standalone
for just HDF5, usually I use them as part of a larger application.

It would be great if you could share the makefiles, and some instructions for how to cast those on official tarball :wink:

Will do. Working on making them standalone for just HDF5 and will let you know as soon as it's ready.

--
  ___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

16.05.2014 15:38, Werner Benger пишет:

16.05.2014 14:45, Werner Benger пишет:

I do not want to recompile HDF5 library because it seems non-trivial,
and because I hardly believe it can change anything compared to
official binaries...

I find it rather easy and do it frequently, just not using cmake or the
official configure script, which has issues under msys/mingw. I can
share those makefiles, but I'd need to prepare them to work standalone
for just HDF5, usually I use them as part of a larger application.

It would be great if you could share the makefiles, and some
instructions for how to cast those on official tarball :wink:

Will do. Working on making them standalone for just HDF5 and will let
you know as soon as it's ready.

Here, use this URL for SVN checkout:

    https://subversion.assembla.com/svn/FiberHDF5/hdf5

I just verified it to work under both Linux and MinGW64.

Once checked out, just do a make in the hdf5/ subdir, in Windows/Msys
use mingw32-make .

Thank you!

I have checked out GNUmakefile.rules into Makefile my example program dir (is it correct?). How do I run mingw32-make?

Best wishes,
Andrey Paramonov

···

On 16.05.2014 13:04, Werner Benger wrote:

On 16.05.2014 12:46, Андрей Парамонов wrote:

On 16.05.2014 12:21, Андрей Парамонов wrote:

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

By doing

    svn checkout https://subversion.assembla.com/svn/FiberHDF5/hdf5

You would get those two files plus three subdirectories

    hdf5/
    make/
    zlib123/

These are included via SVN externals. Then you just cd into this main directory and type "mingw32-make". It should traverse into the zlib123/ folder first and then hdf5/ , finally printing a message that it's all done.

mingw32-make is usually included with a mingw installation. Are you using msys and bash as command prompt?

            Werner

···

On 16.05.2014 14:15, Андрей Парамонов wrote:

16.05.2014 15:38, Werner Benger пишет:

On 16.05.2014 13:04, Werner Benger wrote:

On 16.05.2014 12:46, Андрей Парамонов wrote:

16.05.2014 14:45, Werner Benger пишет:

On 16.05.2014 12:21, Андрей Парамонов wrote:

I do not want to recompile HDF5 library because it seems non-trivial,
and because I hardly believe it can change anything compared to
official binaries...

I find it rather easy and do it frequently, just not using cmake or the
official configure script, which has issues under msys/mingw. I can
share those makefiles, but I'd need to prepare them to work standalone
for just HDF5, usually I use them as part of a larger application.

It would be great if you could share the makefiles, and some
instructions for how to cast those on official tarball :wink:

Will do. Working on making them standalone for just HDF5 and will let
you know as soon as it's ready.

Here, use this URL for SVN checkout:

    https://subversion.assembla.com/svn/FiberHDF5/hdf5

I just verified it to work under both Linux and MinGW64.

Once checked out, just do a make in the hdf5/ subdir, in Windows/Msys
use mingw32-make .

Thank you!

I have checked out GNUmakefile.rules into Makefile my example program dir (is it correct?). How do I run mingw32-make?

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

Just to follow up, you would do just the following on the command line using bash:

   svn checkout https://subversion.assembla.com/svn/FiberHDF5/hdf5
   cd hdf5
   mingw32-make

Provided "svn", "mingw32-make" and gcc (both a 64-bit gcc and 32-bit gcc will work) are in your PATH. The file hdf5.dll will be placed together with zlib in a subfolder of the newly created lib/ directory.

        Werner

···

On 16.05.2014 19:11, Werner Benger wrote:

On 16.05.2014 14:15, Андрей Парамонов wrote:

16.05.2014 15:38, Werner Benger пишет:

On 16.05.2014 13:04, Werner Benger wrote:

On 16.05.2014 12:46, Андрей Парамонов wrote:

16.05.2014 14:45, Werner Benger пишет:

On 16.05.2014 12:21, Андрей Парамонов wrote:

I do not want to recompile HDF5 library because it seems non-trivial,
and because I hardly believe it can change anything compared to
official binaries...

I find it rather easy and do it frequently, just not using cmake or the
official configure script, which has issues under msys/mingw. I can
share those makefiles, but I'd need to prepare them to work standalone
for just HDF5, usually I use them as part of a larger application.

It would be great if you could share the makefiles, and some
instructions for how to cast those on official tarball :wink:

Will do. Working on making them standalone for just HDF5 and will let
you know as soon as it's ready.

Here, use this URL for SVN checkout:

    https://subversion.assembla.com/svn/FiberHDF5/hdf5

I just verified it to work under both Linux and MinGW64.

Once checked out, just do a make in the hdf5/ subdir, in Windows/Msys
use mingw32-make .

Thank you!

I have checked out GNUmakefile.rules into Makefile my example program dir (is it correct?). How do I run mingw32-make?

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362