Windows: Adjustments for building with MSYS2/MinGW-w64 toolchain

Hello,

I work in French company where I develop an data access layer used to manage data from our tokamak WEST.
This access layer called IMAS (the same as ITER) will use as backend HDF5 (currently it uses MDSplus).

i am in charge to porting the IMAS access layer under Windows, and I use MinGW-w64 provide by MSYS2 to compile the components and all libraries.

I have downloaded the version 1.10.3 of HDF5 and I have made some adjustments to be able to compile correctly.
Now, I want to share my modifications, but I don’t find any repo on https://github.com/HDFGroup

How can I proceed to share my modifications wit this community?

I can describe here which adjustments I have made and why but I think that GitHub will be the good place to do that, right?

Hey,

Sorry to say, but i don’t know this either. i was just there for some help to get everything to work.

And i did not do any modifications on the library, even if there could be quite some things made easier i guess.

Regards,

Markus

Hi Herve!

02.10.2018 15:20, Herve Ancher пишет:

Now, I want to share my modifications, but I don’t find any repo on
https://github.com/HDFGroup

The repo is available at
https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5

But the public issue tracker is yet to come :-/

Best wishes,
Andrey Paramonov

Hi,

OK, so maybe I can contribute on develop branch with restricted access, and make pull requests?
For that, I will need an account on your Bitbucket if possible.

Regards, Hervé.

Hi Herve!

02.10.2018 15:57, Herve Ancher пишет:

OK, so maybe I can contribute on develop branch with restricted access,
and make pull requests?

I believe HDF Group recommends that you submit patches via
help@hdfgroup.org , but many users would appreciate if you CC your
proposed patches to this list.

I think you may fork


and drop a pointer to your patched branch, in case you prefer github
interface.

Best wishes,
Andrey Paramonov

Hi,

Thanks for explanations.

I have forked the Github repo and made my proper modifications.
A new pull request is opened here: https://github.com/live-clones/hdf5/pull/3

Edit: The pull request is now closed because I made it on Git clone repository. How Can I make a PR on Bitbucket?

Regards.

Please submit a patch file so we can test your solution.

Allen

Please find my patch based on my last commit “Adjustments for Windows MSYS2/MinGW” (I can’t attach file, so here is the code):

From df685877d9844f98813f3043458c88d82be1a36e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Herv=C3=A9=20Ancher?= <herve.ancher@cea.fr>
Date: Wed, 3 Oct 2018 10:42:14 +0200
Subject: [PATCH] Adjustments for Windows MSYS2/MinGW

---
 .gitignore            |  8 +++++
 configure             |  2 +-
 src/H5private.h       | 12 +++++--
 src/H5system.c        | 73 ++++++++++++++++++++++---------------------
 tools/src/h5ls/h5ls.c |  5 +++
 5 files changed, 62 insertions(+), 38 deletions(-)

diff --git a/.gitignore b/.gitignore
index 3caf16a1c6..c04e7f36a0 100644
--- a/.gitignore
+++ b/.gitignore
@@ -27,6 +27,9 @@ bin/missing
 bin/test-driver
 /build*
 /configure
+/config.log
+/config.lt
+/config.status
 m4/libtool.m4
 m4/ltoptions.m4
 m4/ltsugar.m4
@@ -41,3 +44,8 @@ src/H5overflow.h
 src/H5version.h
 
 /.classpath
+
+# Compilation objects
+**/*.o
+**/*.lo
+**/*.exe
diff --git a/configure b/configure
index e767bc9ccb..e6b198379e 100755
--- a/configure
+++ b/configure
@@ -7353,7 +7353,7 @@ fi
 	  ;;
 	-lkernel32)
 	  case $host_os in
-	  *cygwin*) ;;
+	  *cygwin* | *msys*) ;;
 	  *) ac_cv_fc_libs="$ac_cv_fc_libs $ac_arg"
 	    ;;
 	  esac
diff --git a/src/H5private.h b/src/H5private.h
index 8974e46120..a8e8ec5710 100644
--- a/src/H5private.h
+++ b/src/H5private.h
@@ -1110,7 +1110,11 @@ typedef off_t               h5_stat_size_t;
 #define HDmemset(X,C,Z)    memset(X,C,Z)
 #endif /* HDmemset */
 #ifndef HDmkdir
-    #define HDmkdir(S,M)    mkdir(S,M)
+    #if defined __MINGW32__
+        #define HDmkdir(S,M)    mkdir(S)
+    #else
+        #define HDmkdir(S,M)    mkdir(S,M)
+    #endif
 #endif /* HDmkdir */
 #ifndef HDmkfifo
 #define HDmkfifo(S,M)    mkfifo(S,M)
@@ -1232,7 +1236,11 @@ typedef off_t               h5_stat_size_t;
 #define HDsetbuf(F,S)    setbuf(F,S)
 #endif /* HDsetbuf */
 #ifndef HDsetenv
-    #define HDsetenv(N,V,O)    setenv(N,V,O)
+    #if defined __MINGW32__
+        #define HDsetenv(N,V,O)    Wsetenv(N,V,O)
+    #else
+        #define HDsetenv(N,V,O)    setenv(N,V,O)
+    #endif
 #endif /* HDsetenv */
 #ifndef HDsetgid
 #define HDsetgid(G)    setgid(G)
diff --git a/src/H5system.c b/src/H5system.c
index 186d8fa575..a64e5508fd 100644
--- a/src/H5system.c
+++ b/src/H5system.c
@@ -726,6 +726,44 @@ done:
 FUNC_LEAVE_NOAPI(ret_value)
 } /* end H5_make_time() */
 
+#if defined(H5_HAVE_WIN32_API) || defined(__MINGW32__)
+
+/*-------------------------------------------------------------------------
+ * Function:    Wsetenv
+ *
+ * Purpose:     Wrapper function for setenv on Windows systems.
+ *              Interestingly, getenv *is* available in the Windows
+ *              POSIX layer, just not setenv.
+ *
+ * Return:      Success:    0
+ *              Failure:    non-zero error code
+ *
+ * Programmer:  Dana Robinson
+ *              February 2016
+ *
+ *-------------------------------------------------------------------------
+ */
+int
+Wsetenv(const char *name, const char *value, int overwrite)
+{
+    size_t bufsize;
+    errno_t err;
+
+    /* If we're not overwriting, check if the environment variable exists.
+     * If it does (i.e.: the required buffer size to store the variable's
+     * value is non-zero), then return an error code.
+     */
+    if(!overwrite) {
+        err = getenv_s(&bufsize, NULL, 0, name);
+        if (err || bufsize)
+            return (int)err;
+    } /* end if */
+
+    return (int)_putenv_s(name, value);
+} /* end Wsetenv() */
+
+#endif // H5_HAVE_WIN32_API || __MINGW32__
+
 #ifdef H5_HAVE_WIN32_API
 
 /* Offset between 1/1/1601 and 1/1/1970 in 100 nanosecond units */
@@ -783,41 +821,6 @@ Wgettimeofday(struct timeval *tv, struct timezone *tz)
   return 0;
 } /* end Wgettimeofday() */
 
-
-/*-------------------------------------------------------------------------
- * Function:    Wsetenv
- *
- * Purpose:     Wrapper function for setenv on Windows systems.
- *              Interestingly, getenv *is* available in the Windows
- *              POSIX layer, just not setenv.
- *
- * Return:      Success:    0
- *              Failure:    non-zero error code
- *
- * Programmer:  Dana Robinson
- *              February 2016
- *
- *-------------------------------------------------------------------------
- */
-int
-Wsetenv(const char *name, const char *value, int overwrite)
-{
-    size_t bufsize;
-    errno_t err;
-
-    /* If we're not overwriting, check if the environment variable exists.
-     * If it does (i.e.: the required buffer size to store the variable's
-     * value is non-zero), then return an error code.
-     */
-    if(!overwrite) {
-        err = getenv_s(&bufsize, NULL, 0, name);
-        if (err || bufsize)
-            return (int)err;
-    } /* end if */
-
-    return (int)_putenv_s(name, value);
-} /* end Wsetenv() */
-
 #ifdef H5_HAVE_WINSOCK2_H
 #pragma comment(lib, "advapi32.lib")
 #endif
diff --git a/tools/src/h5ls/h5ls.c b/tools/src/h5ls/h5ls.c
index 4bc15260cf..53a45f95fd 100644
--- a/tools/src/h5ls/h5ls.c
+++ b/tools/src/h5ls/h5ls.c
@@ -28,6 +28,11 @@
 #include "h5tools_dump.h"
 #include "h5trav.h"
 
+#if defined(H5_HAVE_GETCONSOLESCREENBUFFERINFO) && defined(__MINGW32__)
+#include <windows.h>
+#include <wincon.h>
+#endif
+
 /* Name of tool */
 #define PROGRAMNAME "h5ls"
 
-- 
2.19.0

And another one:

From 85462d660674bee7356392df84fafbe37b454659 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Herv=C3=A9=20Ancher?= <herve.ancher@cea.fr>
Date: Fri, 5 Oct 2018 08:41:08 +0200
Subject: [PATCH] Expose HDflock -> Wflock for Windows with MinGW toolchain

---
 .gitignore      |  5 ++-
 src/H5private.h |  6 +++-
 src/H5system.c  | 94 ++++++++++++++++++++++++-------------------------
 3 files changed, 56 insertions(+), 49 deletions(-)

diff --git a/.gitignore b/.gitignore
index c04e7f36a0..787a634a02 100644
--- a/.gitignore
+++ b/.gitignore
@@ -45,7 +45,10 @@ src/H5version.h
 
 /.classpath
 
-# Compilation objects
+# Compilation objects and directories
 **/*.o
 **/*.lo
+**/*.la
 **/*.exe
+**/.deps/
+**/.libs/
diff --git a/src/H5private.h b/src/H5private.h
index a8e8ec5710..f61ae94781 100644
--- a/src/H5private.h
+++ b/src/H5private.h
@@ -827,7 +827,11 @@ H5_DLL H5_ATTR_CONST int Nflock(int fd, int operation);
     #elif defined(H5_HAVE_FCNTL)
         #define HDflock(F,L)    Pflock(F,L)
     #else
-        #define HDflock(F,L)    Nflock(F,L)
+        #if defined __MINGW32__
+            #define HDflock(F,L)    Wflock(F,L)
+        #else
+            #define HDflock(F,L)    Nflock(F,L)
+        #endif /* H5_HAVE_FLOCK */
     #endif /* H5_HAVE_FLOCK */
 #endif /* HDflock */
 #ifndef HDfloor
diff --git a/src/H5system.c b/src/H5system.c
index a64e5508fd..099a7067a5 100644
--- a/src/H5system.c
+++ b/src/H5system.c
@@ -762,6 +762,53 @@ Wsetenv(const char *name, const char *value, int overwrite)
     return (int)_putenv_s(name, value);
 } /* end Wsetenv() */
 
+/*-------------------------------------------------------------------------
+ * Function:    Wflock
+ *
+ * Purpose:     Wrapper function for flock on Windows systems
+ *
+ * Return:      Success:    0
+ *              Failure:    -1
+ *
+ *-------------------------------------------------------------------------
+ */
+int
+Wflock(int H5_ATTR_UNUSED fd, int H5_ATTR_UNUSED operation) {
+
+/* This is a no-op while we implement a Win32 VFD */
+#if 0
+int
+Wflock(int fd, int operation) {
+
+    HANDLE          hFile;
+    DWORD           dwFlags = LOCKFILE_FAIL_IMMEDIATELY;
+    DWORD           dwReserved = 0;
+                    /* MAXDWORD for entire file */
+    DWORD           nNumberOfBytesToLockLow = MAXDWORD;
+    DWORD           nNumberOfBytesToLockHigh = MAXDWORD;
+                    /* Must initialize OVERLAPPED struct */
+    OVERLAPPED      overlapped = {0};
+
+    /* Get Windows HANDLE */
+    hFile = _get_osfhandle(fd);
+
+    /* Convert to Windows flags */
+    if(operation & LOCK_EX)
+        dwFlags |= LOCKFILE_EXCLUSIVE_LOCK;
+
+    /* Lock or unlock */
+    if(operation & LOCK_UN)
+        if(0 == UnlockFileEx(hFile, dwReserved, nNumberOfBytesToLockLow,
+                            nNumberOfBytesToLockHigh, &overlapped))
+            return -1;
+    else
+        if(0 == LockFileEx(hFile, dwFlags, dwReserved, nNumberOfBytesToLockLow,
+                            nNumberOfBytesToLockHigh, &overlapped))
+            return -1;
+#endif /* 0 */
+    return 0;
+} /* end Wflock() */
+
 #endif // H5_HAVE_WIN32_API || __MINGW32__
 
 #ifdef H5_HAVE_WIN32_API
@@ -866,53 +913,6 @@ int c99_vsnprintf(char* str, size_t size, const char* format, va_list ap)
 }
 
 
-/*-------------------------------------------------------------------------
- * Function:    Wflock
- *
- * Purpose:     Wrapper function for flock on Windows systems
- *
- * Return:      Success:    0
- *              Failure:    -1
- *
- *-------------------------------------------------------------------------
- */
-int
-Wflock(int H5_ATTR_UNUSED fd, int H5_ATTR_UNUSED operation) {
-
-/* This is a no-op while we implement a Win32 VFD */
-#if 0
-int
-Wflock(int fd, int operation) {
-
-    HANDLE          hFile;
-    DWORD           dwFlags = LOCKFILE_FAIL_IMMEDIATELY;
-    DWORD           dwReserved = 0;
-                    /* MAXDWORD for entire file */
-    DWORD           nNumberOfBytesToLockLow = MAXDWORD;
-    DWORD           nNumberOfBytesToLockHigh = MAXDWORD;
-                    /* Must initialize OVERLAPPED struct */
-    OVERLAPPED      overlapped = {0};
-
-    /* Get Windows HANDLE */
-    hFile = _get_osfhandle(fd);
-
-    /* Convert to Windows flags */
-    if(operation & LOCK_EX)
-        dwFlags |= LOCKFILE_EXCLUSIVE_LOCK;
-
-    /* Lock or unlock */
-    if(operation & LOCK_UN)
-        if(0 == UnlockFileEx(hFile, dwReserved, nNumberOfBytesToLockLow,
-                            nNumberOfBytesToLockHigh, &overlapped))
-            return -1;
-    else
-        if(0 == LockFileEx(hFile, dwFlags, dwReserved, nNumberOfBytesToLockLow,
-                            nNumberOfBytesToLockHigh, &overlapped))
-            return -1;
-#endif /* 0 */
-    return 0;
-} /* end Wflock() */
-
 
  /*--------------------------------------------------------------------------
   * Function:    Wnanosleep
-- 
2.19.0

You can see the last commits on my repo:

Hi,

I try to find my PR or my patch directly inside the BitBucket of HDF5 project, but my modifications (available above) seem to never been merged.

Is it possible to review and integrate my modifications? I would like to use the official repo instead mine, to be compliant with all platforms.

Regards.

Dear all,

I come back about my patches (see above), do them have been merged into the official branch?

Please review them, with these patches I am able to compile with MinGW-w64 under Windows.

Thanks.

Can you try sending an email help@hdfgroup.org or uploading the path to BitBucket

Hi Hervé,
I would like to compile or use a GCC compiled version of HDF5 (ideally 1.8.21 but above could work) on Windows. I am using Winbuilds (http://win-builds.org/doku.php) as Windows GCC toolchain.

Could you share your modified HDF5 repository with me or send me a compiled version?

We did get the mingw support improved with your help - but recent development has affected some parts and we are currently revisiting the mingw support.
I think there is a planned release soon, and we plan to have the mingw builds improved.

Allen