Hi. I'm no compiler expert, but I'm trying to build hdf5-1.8.10-patch1 for local installation in a user account on a Cray super computer, using the Cray Compiler (Version 8.1.1). I configure like so:
CC="cc" ./configure --prefix=<...>
(We have a module wrapper cc program which points to the cray compiler.)
However, most of the configure tests fail, and ultimately the build fails, because of the following error:
CC-2289 craycc: ERROR in command line
Invalid characters after option '-s' in command line item '-std=c99'.
All the compiler commands executed pass in -std=c99, but Cray C uses a different syntax for that. ("-h c99") This seems to be controlled by H5_CFLAGS variable in configure.ac. What is the correct flag to pass to configure, or other trick, to modify that to the correct syntax?
I too have had the same problem with both HDF5 1.8.5 and 1.8.8 with the Cray
compiler.
The "-std=c99" flag gets added because the HDF5 configure script incorrectly
identifies the Cray compiler has the Intel compiler and consequently it
automatically sets a load of compiler flags that are intended for the Intel
compilers.
To workarround this edit config/linux-gnulibcl and comment out the two lines
that source files named "intel-flags" and "intel-fflags".
It would be handy if future versions of HDF5 could detect the Cray compiler
correctly.
I worked in a Cray about 3 to 4 years ago. Back then the cc and ftn were actually cross compilers
provided by PGI. Maybe they are changed and supported by Intel.
One of our staff has more recent experience with Cray XE6,
Cray Linux Environment (CLE) PrgEnv-pgi/4.0.46 hopper.nersc.gov pgcc 12.5-0 64-bit target on x86-64 Linux -tp shanghai
pgf90 12.5-0 64-bit target on x86-64 Linux -tp shanghai
pgCC 12.5-0 64-bit target on x86-64 Linux -tp shanghai
After your have downloaded the tar file, untar it into hdf5-1.8.11-pre1/,
cd into it and tried to follow the instruction in release_docs/INSTALL_parallel,
under "2.4. Red Storm (Cray XT3) (for v1.8 and later)" or
"2.5. Hopper (Cray XE6) (for v1.8 and later)".
The section 2.4 is more detailed but it is old information for Red Storm.
The section 2.5 is current. Please see which one is more relevant to your
Cray. (And let me know too.)
If it still fails for you in any way, please send me all these information:
1. Machine information (Cray so and so)
2. All commands used and their output.
3. compiler versions (e.g. output of "cc --version")
Thanks.
-Albert Cheng
The HDF Group
···
On 4/24/13 9:57 AM, Neil wrote:
Hi,
I too have had the same problem with both HDF5 1.8.5 and 1.8.8 with the Cray
compiler.
The "-std=c99" flag gets added because the HDF5 configure script incorrectly
identifies the Cray compiler has the Intel compiler and consequently it
automatically sets a load of compiler flags that are intended for the Intel
compilers.
To workarround this edit config/linux-gnulibcl and comment out the two lines
that source files named "intel-flags" and "intel-fflags".
It would be handy if future versions of HDF5 could detect the Cray compiler
correctly.
I managed to replicate the issue.
We test only with the default PGI compilers, which work quite fine. However when I switch to the Cray compilers, I see the same issue you guys are seeing.
We'll add it to our bug tracking system. Thanks for reporting!
Mohamad
···
On 4/26/2013 9:47 AM, Albert Cheng wrote:
Hi Neil,
I worked in a Cray about 3 to 4 years ago. Back then the cc and ftn were actually cross compilers
provided by PGI. Maybe they are changed and supported by Intel.
One of our staff has more recent experience with Cray XE6,
Cray Linux Environment (CLE) PrgEnv-pgi/4.0.46 hopper.nersc.gov pgcc 12.5-0 64-bit target on x86-64 Linux -tp shanghai
pgf90 12.5-0 64-bit target on x86-64 Linux -tp shanghai
pgCC 12.5-0 64-bit target on x86-64 Linux -tp shanghai
After your have downloaded the tar file, untar it into hdf5-1.8.11-pre1/,
cd into it and tried to follow the instruction in release_docs/INSTALL_parallel,
under "2.4. Red Storm (Cray XT3) (for v1.8 and later)" or
"2.5. Hopper (Cray XE6) (for v1.8 and later)".
The section 2.4 is more detailed but it is old information for Red Storm.
The section 2.5 is current. Please see which one is more relevant to your
Cray. (And let me know too.)
If it still fails for you in any way, please send me all these information:
1. Machine information (Cray so and so)
2. All commands used and their output.
3. compiler versions (e.g. output of "cc --version")
Thanks.
-Albert Cheng
The HDF Group
On 4/24/13 9:57 AM, Neil wrote:
Hi,
I too have had the same problem with both HDF5 1.8.5 and 1.8.8 with the Cray
compiler.
The "-std=c99" flag gets added because the HDF5 configure script incorrectly
identifies the Cray compiler has the Intel compiler and consequently it
automatically sets a load of compiler flags that are intended for the Intel
compilers.
To workarround this edit config/linux-gnulibcl and comment out the two lines
that source files named "intel-flags" and "intel-fflags".
It would be handy if future versions of HDF5 could detect the Cray compiler
correctly.
Oh and please follow Albert's instructions about giving us more information about your system and build failure.
More information = Better fix
Thanks,
Mohamad
···
On 4/26/2013 9:56 AM, Mohamad Chaarawi wrote:
I managed to replicate the issue.
We test only with the default PGI compilers, which work quite fine. However when I switch to the Cray compilers, I see the same issue you guys are seeing.
We'll add it to our bug tracking system. Thanks for reporting!
Mohamad
On 4/26/2013 9:47 AM, Albert Cheng wrote:
Hi Neil,
I worked in a Cray about 3 to 4 years ago. Back then the cc and ftn were actually cross compilers
provided by PGI. Maybe they are changed and supported by Intel.
One of our staff has more recent experience with Cray XE6,
Cray Linux Environment (CLE) PrgEnv-pgi/4.0.46 hopper.nersc.gov pgcc 12.5-0 64-bit target on x86-64 Linux -tp shanghai
pgf90 12.5-0 64-bit target on x86-64 Linux -tp shanghai
pgCC 12.5-0 64-bit target on x86-64 Linux -tp shanghai
After your have downloaded the tar file, untar it into hdf5-1.8.11-pre1/,
cd into it and tried to follow the instruction in release_docs/INSTALL_parallel,
under "2.4. Red Storm (Cray XT3) (for v1.8 and later)" or
"2.5. Hopper (Cray XE6) (for v1.8 and later)".
The section 2.4 is more detailed but it is old information for Red Storm.
The section 2.5 is current. Please see which one is more relevant to your
Cray. (And let me know too.)
If it still fails for you in any way, please send me all these information:
1. Machine information (Cray so and so)
2. All commands used and their output.
3. compiler versions (e.g. output of "cc --version")
Thanks.
-Albert Cheng
The HDF Group
On 4/24/13 9:57 AM, Neil wrote:
Hi,
I too have had the same problem with both HDF5 1.8.5 and 1.8.8 with the Cray
compiler.
The "-std=c99" flag gets added because the HDF5 configure script incorrectly
identifies the Cray compiler has the Intel compiler and consequently it
automatically sets a load of compiler flags that are intended for the Intel
compilers.
To workarround this edit config/linux-gnulibcl and comment out the two lines
that source files named "intel-flags" and "intel-fflags".
It would be handy if future versions of HDF5 could detect the Cray compiler
correctly.
Thanks for the replies. I'll try out the 1.8.11 snapshot later this week and
let you know how I get on as well as looking at the version information
produced by the Cray compilers.
As it happens the cc, CC and ftn commands on the Cray (both under Catamount
and CLE) are all wrapper scripts to the underlying compilers that were
created by Cray.
The scripts just examine the user's environment to build up the compiler
flags, include options and libraries to link against. So if you have the
PrgEnv-pgi module loaded the wrapper scripts will call the PGI compilers for
example. In the case of the Cray compilers the wrapper scripts will call
craycc etc. under the covers.
Hi again,
As promised, here is the version output information for the Cray compiler
(CCE). Note: this version information is obtained using the command: "cc
-V".
cc -v and cc --version are not supported.
cc -V
/opt/cray/xt-asyncpe/5.17/bin/cc: INFO: Compiling with
CRAYPE_COMPILE_TARGET=native.
Cray C : Version 8.1.7 Tue Apr 30, 2013 09:48:17
I have also tried out HDF5 1.8.11-pre2, but it also incorrectly identifies
the Cray compiler as the Intel compiler.
Hope this helps,
Neil.
What kind of Cray and OS version do you use?
Does it have a frontend for compiling and a backend for
compute? If so, what kind of machine and OS version
is the frontend?
There are problems with the configure process for the Cray platform
that we will not be able to fix it in time for this pending release.
We plan to fix it in the next release.
Meanwhile, can you follow the instruction in release_docs/INSTALL_parallel,
under section "2.4. Hopper (Cray XE6) (for v1.8 and later)"
to see if you can build HDF5 in your Cray?
By the way, are you building the serial HDF5 library or the Parallel HDF5
which requires MPI support?
-Albert Chen
The HDF Group
···
On 4/30/13 4:57 AM, Neil wrote:
Hi again,
As promised, here is the version output information for the Cray compiler (CCE). Note: this version information is obtained using the command: "cc -V".
>cc -v| and |cc --version| are not supported.
>cc -V
/opt/cray/xt-asyncpe/5.17/bin/cc: INFO: Compiling with CRAYPE_COMPILE_TARGET=native.
Cray C : Version 8.1.7 Tue Apr 30, 2013 09:48:17|
I have also tried out HDF5 1.8.11-pre2, but it also incorrectly identifies the Cray compiler as the Intel compiler.
We are using a Cray XK6 running Cray's Compute Linux Environment (CLE) with
the Cray CCE compiler. The front end nodes run SLES Linux and the compute
nodes run CLE. So when building HDF5 (and any other application/library)
it's a cross compilation.
I can get the library to build successfully, both serial and parallel
versions by hacking the "config/linux-gnulibcl" to remove the inclusion of
the intel-flags and intel-fflags files (as stated previously). It would be
handy if the configure script could correctly identify the Cray CCE compiler
as some users may not be aware that this is the problem.
We use pretty much the same method as described in the
"release_docs/INSTALL_parallel". As the doc says, the key is to set the
RUNPARALLEL and RUNSERIAL environment variables.
With the Cray CCE compiler it doesn't generate Fortran module files by
default, so you have to make sure that "-em" is set in FFLAGS. For CFLAGS
and CXXFLAGS we also have to set "-h tolerant -h system_alloc".
I'll try to post our configure options for both serial and parallel builds
tomorrow when I next have access to the system in case this is of help.
Since the configure process is executed in the front end which is a Linux system,
configure naturally thinks it is working in a Linux system, therefore it goes to
the Linux configure settings. I think the right solution is to tell configure to
build for a cross-compiling Cray system. This will take some effort and won't
be available in the pending release v1.8.11. We are planning to provide it
in the next release.
-Albert Cheng
The HDF Group
···
On 5/2/13 3:49 PM, Neil wrote:
Hi Albert,
We are using a Cray XK6 running Cray's Compute Linux Environment (CLE) with
the Cray CCE compiler. The front end nodes run SLES Linux and the compute
nodes run CLE. So when building HDF5 (and any other application/library)
it's a cross compilation.
I can get the library to build successfully, both serial and parallel
versions by hacking the "config/linux-gnulibcl" to remove the inclusion of
the intel-flags and intel-fflags files (as stated previously). It would be
handy if the configure script could correctly identify the Cray CCE compiler
as some users may not be aware that this is the problem.
We use pretty much the same method as described in the
"release_docs/INSTALL_parallel". As the doc says, the key is to set the
RUNPARALLEL and RUNSERIAL environment variables.
With the Cray CCE compiler it doesn't generate Fortran module files by
default, so you have to make sure that "-em" is set in FFLAGS. For CFLAGS
and CXXFLAGS we also have to set "-h tolerant -h system_alloc".
I'll try to post our configure options for both serial and parallel builds
tomorrow when I next have access to the system in case this is of help.
Since the configure process is executed in the front end which is a Linux system,
configure naturally thinks it is working in a Linux system, therefore it goes to
the Linux configure settings. I think the right solution is to tell configure to
build for a cross-compiling Cray system. This will take some effort and won't
be available in the pending release v1.8.11. We are planning to provide it
in the next release.
The Cray wrappers that you provide with configure (cc and ftn) should do all the cross compiling you need.
The bug that Neil is talking about does not have to do with cross compiling. There is just an issue when Cray compilers are used in the Cray build wrappers. This issue, we hope, would be fixed in the next release.
We recommend now that you don't use the cray compilers. Usually Cray supercomputers provide other compilers like PGI and GNU that should work fine.
