HDF5 1.6.10 and 1.8.4 release candidates are available for testing

Dear members of the HDF5 community,

Release candidates for HDF5 1.6.10 and HDF5 1.8.4 are available for testing with your favorite application or on the system/with compilers you use. You may download the source tar balls from

ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v16/hdf5-1.6.10-pre1.tar
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v18/hdf5-1.8.4-pre1.tar

Please let us know about the problems you find. We do not promise to fix them, but will be glad to document and address them in the future.

Thank you for your help in making HDF5 the best software in the world :-)!

Elena

Is there somewhere to get a "what is changed" or "what is new?" or "What is Fixed" ?

···

---
Mike Jackson www.bluequartz.net

On Oct 28, 2009, at 11:13 PM, Elena Pourmal wrote:

Dear members of the HDF5 community,

Release candidates for HDF5 1.6.10 and HDF5 1.8.4 are available for testing with your favorite application or on the system/with compilers you use. You may download the source tar balls from

ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v16/hdf5-1.6.10-pre1.tar
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v18/hdf5-1.8.4-pre1.tar

Please let us know about the problems you find. We do not promise to fix them, but will be glad to document and address them in the future.

Thank you for your help in making HDF5 the best software in the world :-)!

Elena

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

I have built 1.6.10 and 1.8.4 on an Intel Mac running Mac OS X 10.6.1
with Xcode 3.2.1's clang compiler. They built, but there are tonnes of
warnings. Many easy to fix; many are serious. Build transcript
attached. No tests failed.

Also, I notice that sprintf() is used a lot, ex:

char meta_name[1024]
sprintf(meta_name, "%%s%s", meta_ext);

This is very poor practice; the following would be much safer (for
hopefully obvious reasons):

char meta_name[1024]
snprintf(meta_name, sizeof (meta_name), "%%s%s", meta_ext);

hth,

Archive.zip (35.7 KB)

···

On 10/28/09 10:13 PM, Elena Pourmal said:

Release candidates for HDF5 1.6.10 and HDF5 1.8.4 are available for
testing with your favorite application or on the system/with compilers
you use. You may download the source tar balls from

ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v16/hdf5-1.6.10-
pre1.tar
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v18/hdf5-1.8.4-
pre1.tar

Please let us know about the problems you find. We do not promise to
fix them, but will be glad to document and address them in the future.

--
____________________________________________________________
Sean McBride, B. Eng sean@rogue-research.com
Rogue Research www.rogue-research.com
Mac Software Developer Montréal, Québec, Canada

Hi Elena,

A Thursday 29 October 2009 04:13:19 Elena Pourmal escrigué:

Dear members of the HDF5 community,

Release candidates for HDF5 1.6.10 and HDF5 1.8.4 are available for
testing with your favorite application or on the system/with compilers
you use. You may download the source tar balls from

ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v16/hdf5-1.6.10-pre
1.tar
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v18/hdf5-1.8.4-pre1
.tar

Please let us know about the problems you find. We do not promise to
fix them, but will be glad to document and address them in the future.

I've just tested the packages and all tests passes here. Good job! :slight_smile:

However, I've noticed that configure will set the optimization flag to 0 (-
O0), even though compilation mode is set to "production". That happens for
both 1.6.10-pre1 and 1.8.4-pre1. I suppose that this is not intended and you
may want to fix this. I'm attaching the output of configure script on my
machine for your inspection.

Thanks!

configure-hdf5-1.8.4-pre1.out (4.71 KB)

···

--
Francesc Alted

Please check RELEASE.txt file in the release_docs directory. It is also available in the ftp directory where tar ball is.

Elena

···

On Oct 29, 2009, at 9:35 AM, Michael Jackson wrote:

Is there somewhere to get a "what is changed" or "what is new?" or "What is Fixed" ?

---
Mike Jackson www.bluequartz.net

On Oct 28, 2009, at 11:13 PM, Elena Pourmal wrote:

Dear members of the HDF5 community,

Release candidates for HDF5 1.6.10 and HDF5 1.8.4 are available for testing with your favorite application or on the system/with compilers you use. You may download the source tar balls from

ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v16/hdf5-1.6.10-pre1.tar
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v18/hdf5-1.8.4-pre1.tar

Please let us know about the problems you find. We do not promise to fix them, but will be glad to document and address them in the future.

Thank you for your help in making HDF5 the best software in the world :-)!

Elena

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Sean,

thanks a lot! Noted.

Elena

···

On Oct 29, 2009, at 4:29 PM, Sean McBride wrote:

On 10/28/09 10:13 PM, Elena Pourmal said:

Release candidates for HDF5 1.6.10 and HDF5 1.8.4 are available for
testing with your favorite application or on the system/with compilers
you use. You may download the source tar balls from

ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v16/hdf5-1.6.10-
pre1.tar
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v18/hdf5-1.8.4-
pre1.tar

Please let us know about the problems you find. We do not promise to
fix them, but will be glad to document and address them in the future.

I have built 1.6.10 and 1.8.4 on an Intel Mac running Mac OS X 10.6.1
with Xcode 3.2.1's clang compiler. They built, but there are tonnes of
warnings. Many easy to fix; many are serious. Build transcript
attached. No tests failed.

Also, I notice that sprintf() is used a lot, ex:

char meta_name[1024]
sprintf(meta_name, "%%s%s", meta_ext);

This is very poor practice; the following would be much safer (for
hopefully obvious reasons):

char meta_name[1024]
snprintf(meta_name, sizeof (meta_name), "%%s%s", meta_ext);

hth,

--
____________________________________________________________
Sean McBride, B. Eng sean@rogue-research.com
Rogue Research www.rogue-research.com
Mac Software Developer Montréal, Québec, Canada
<Archive>

Elena,

I should add that I did not enable some torturous set of warnings. :slight_smile:
These are all warnings that are generated even when there are no warning
flags at all!

I do hope you will fix them. As for sprintf, I realise that's a long
term job...

Cheers,

Sean

···

On 10/29/09 4:36 PM, Elena Pourmal said:

Sean,

thanks a lot! Noted.

Elena
On Oct 29, 2009, at 4:29 PM, Sean McBride wrote:

On 10/28/09 10:13 PM, Elena Pourmal said:

Release candidates for HDF5 1.6.10 and HDF5 1.8.4 are available for
testing with your favorite application or on the system/with
compilers
you use. You may download the source tar balls from

ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v16/hdf5-1.6.10-
pre1.tar
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v18/hdf5-1.8.4-
pre1.tar

Please let us know about the problems you find. We do not promise to
fix them, but will be glad to document and address them in the
future.

I have built 1.6.10 and 1.8.4 on an Intel Mac running Mac OS X 10.6.1
with Xcode 3.2.1's clang compiler. They built, but there are tonnes
of
warnings. Many easy to fix; many are serious. Build transcript
attached. No tests failed.

Also, I notice that sprintf() is used a lot, ex:

char meta_name[1024]
sprintf(meta_name, "%%s%s", meta_ext);

This is very poor practice; the following would be much safer (for
hopefully obvious reasons):

char meta_name[1024]
snprintf(meta_name, sizeof (meta_name), "%%s%s", meta_ext);

Hi Vesa,

···

On Oct 30, 2009, at 4:29 AM, Francesc Alted wrote:

Hi Elena,

A Thursday 29 October 2009 04:13:19 Elena Pourmal escrigué:

Dear members of the HDF5 community,

Release candidates for HDF5 1.6.10 and HDF5 1.8.4 are available for
testing with your favorite application or on the system/with compilers
you use. You may download the source tar balls from

ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v16/hdf5-1.6.10-pre
1.tar
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v18/hdf5-1.8.4-pre1
.tar

Please let us know about the problems you find. We do not promise to
fix them, but will be glad to document and address them in the future.

I've just tested the packages and all tests passes here. Good job! :slight_smile:

However, I've noticed that configure will set the optimization flag to 0 (-
O0), even though compilation mode is set to "production". That happens for
both 1.6.10-pre1 and 1.8.4-pre1. I suppose that this is not intended and you
may want to fix this. I'm attaching the output of configure script on my
machine for your inspection.

  This is because of some code that we have that is causing gcc to generate bad code with higher levels of optimization. I'm working on a fix, but didn't have time to get it in for the this release. Hopefully, it should be done shortly after the release and be included in the next release.

  Quincey

Mike,

Can your customers use 1.8 with 1.6 compatibility option?

Elena

···

On Oct 30, 2009, at 9:09 AM, Michael Jackson wrote:

On Oct 30, 2009, at 6:31 AM, Quincey Koziol wrote:

Hi Vesa,

On Oct 30, 2009, at 4:29 AM, Francesc Alted wrote:

Hi Elena,

A Thursday 29 October 2009 04:13:19 Elena Pourmal escrigué:

Dear members of the HDF5 community,

Release candidates for HDF5 1.6.10 and HDF5 1.8.4 are available for
testing with your favorite application or on the system/with compilers
you use. You may download the source tar balls from

ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v16/hdf5-1.6.10-pre
1.tar
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v18/hdf5-1.8.4-pre1
.tar

Please let us know about the problems you find. We do not promise to
fix them, but will be glad to document and address them in the future.

I've just tested the packages and all tests passes here. Good job! :slight_smile:

However, I've noticed that configure will set the optimization flag to 0 (-
O0), even though compilation mode is set to "production". That happens for
both 1.6.10-pre1 and 1.8.4-pre1. I suppose that this is not intended and you
may want to fix this. I'm attaching the output of configure script on my
machine for your inspection.

  This is because of some code that we have that is causing gcc to generate bad code with higher levels of optimization. I'm working on a fix, but didn't have time to get it in for the this release. Hopefully, it should be done shortly after the release and be included in the next release.

  Quincey

Does this mean that 1.6.10 will always be stuck with this issue? Can the 1.6.10 release just wait a bit until you get the patches in place? It was stated that 1.6.10 will be the last in the 1.6 series. It would be nice to know that those issues finally got taken care of before the code base was abandoned. I still have customers that need 1.6 for their applications.

Mike Jackson

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Does this mean that 1.6.10 will always be stuck with this issue? Can the 1.6.10 release just wait a bit until you get the patches in place? It was stated that 1.6.10 will be the last in the 1.6 series. It would be nice to know that those issues finally got taken care of before the code base was abandoned. I still have customers that need 1.6 for their applications.

Mike Jackson

···

On Oct 30, 2009, at 6:31 AM, Quincey Koziol wrote:

Hi Vesa,

On Oct 30, 2009, at 4:29 AM, Francesc Alted wrote:

Hi Elena,

A Thursday 29 October 2009 04:13:19 Elena Pourmal escrigué:

Dear members of the HDF5 community,

Release candidates for HDF5 1.6.10 and HDF5 1.8.4 are available for
testing with your favorite application or on the system/with compilers
you use. You may download the source tar balls from

ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v16/hdf5-1.6.10-pre
1.tar
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v18/hdf5-1.8.4-pre1
.tar

Please let us know about the problems you find. We do not promise to
fix them, but will be glad to document and address them in the future.

I've just tested the packages and all tests passes here. Good job! :slight_smile:

However, I've noticed that configure will set the optimization flag to 0 (-
O0), even though compilation mode is set to "production". That happens for
both 1.6.10-pre1 and 1.8.4-pre1. I suppose that this is not intended and you
may want to fix this. I'm attaching the output of configure script on my
machine for your inspection.

  This is because of some code that we have that is causing gcc to generate bad code with higher levels of optimization. I'm working on a fix, but didn't have time to get it in for the this release. Hopefully, it should be done shortly after the release and be included in the next release.

  Quincey

I have not tried to compile HDF5 1.8 with the 1.6 compatibility mode yet. I'll have to read up on that. It would also require me to update my build system with HDF5 1.8 which probably is not a small endeavor. It just seems that if a fix is "close" then why put out the 1.6.10 release now instead of waiting for the fix to be placed into the 1.6.x branch.

Mike

Elena Pourmal wrote:

···

Mike,

Can your customers use 1.8 with 1.6 compatibility option?

Elena
On Oct 30, 2009, at 9:09 AM, Michael Jackson wrote:

On Oct 30, 2009, at 6:31 AM, Quincey Koziol wrote:

Hi Vesa,

On Oct 30, 2009, at 4:29 AM, Francesc Alted wrote:

Hi Elena,

A Thursday 29 October 2009 04:13:19 Elena Pourmal escrigu�:

Dear members of the HDF5 community,

Release candidates for HDF5 1.6.10 and HDF5 1.8.4 are available for
testing with your favorite application or on the system/with compilers
you use. You may download the source tar balls from

ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v16/hdf5-1.6.10-pre

1.tar
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v18/hdf5-1.8.4-pre1

.tar

Please let us know about the problems you find. We do not promise to
fix them, but will be glad to document and address them in the future.

I've just tested the packages and all tests passes here. Good job! :slight_smile:

However, I've noticed that configure will set the optimization flag to 0 (-
O0), even though compilation mode is set to "production". That happens for
both 1.6.10-pre1 and 1.8.4-pre1. I suppose that this is not intended and you
may want to fix this. I'm attaching the output of configure script on my
machine for your inspection.

    This is because of some code that we have that is causing gcc to generate bad code with higher levels of optimization. I'm working on a fix, but didn't have time to get it in for the this release. Hopefully, it should be done shortly after the release and be included in the next release.

    Quincey

Does this mean that 1.6.10 will always be stuck with this issue? Can the 1.6.10 release just wait a bit until you get the patches in place? It was stated that 1.6.10 will be the last in the 1.6 series. It would be nice to know that those issues finally got taken care of before the code base was abandoned. I still have customers that need 1.6 for their applications.

Mike Jackson

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Sean,

Just to let you know that I entered a bug report into our system. We will try to address the warnings you reported in the future releases.

All,

The HDF developers are aware of the warnings produced by multiple compilers on multiple platforms in the multiple versions of the HDF(4,5) software. We are working on the code clean-up.
We will be more than happy to consider patches sent by the members of the community to help us with this task.

Thank you!

Elena

···

On Oct 29, 2009, at 4:29 PM, Sean McBride wrote:

On 10/28/09 10:13 PM, Elena Pourmal said:

Release candidates for HDF5 1.6.10 and HDF5 1.8.4 are available for
testing with your favorite application or on the system/with compilers
you use. You may download the source tar balls from

ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v16/hdf5-1.6.10-
pre1.tar
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v18/hdf5-1.8.4-
pre1.tar

Please let us know about the problems you find. We do not promise to
fix them, but will be glad to document and address them in the future.

I have built 1.6.10 and 1.8.4 on an Intel Mac running Mac OS X 10.6.1
with Xcode 3.2.1's clang compiler. They built, but there are tonnes of
warnings. Many easy to fix; many are serious. Build transcript
attached. No tests failed.

Also, I notice that sprintf() is used a lot, ex:

char meta_name[1024]
sprintf(meta_name, "%%s%s", meta_ext);

This is very poor practice; the following would be much safer (for
hopefully obvious reasons):

char meta_name[1024]
snprintf(meta_name, sizeof (meta_name), "%%s%s", meta_ext);

hth,

--
____________________________________________________________
Sean McBride, B. Eng sean@rogue-research.com
Rogue Research www.rogue-research.com
Mac Software Developer Montréal, Québec, Canada
<Archive>

Sean,

That is why! I was surprise to see so few warnings :slight_smile: Please don't hesitate to send us the complete set.

We are working on code clean up, but it may take some time. I should also say that we are going to drop support for 1.6 after 1.6.10 release. Therefore, most of the work will be done on 1.8 and later.

Thanks again!

Elena

···

On Oct 29, 2009, at 5:14 PM, Sean McBride wrote:

Elena,

I should add that I did not enable some torturous set of warnings. :slight_smile:
These are all warnings that are generated even when there are no warning
flags at all!

I do hope you will fix them. As for sprintf, I realise that's a long
term job...

Cheers,

Sean

On 10/29/09 4:36 PM, Elena Pourmal said:

Sean,

thanks a lot! Noted.

Elena
On Oct 29, 2009, at 4:29 PM, Sean McBride wrote:

On 10/28/09 10:13 PM, Elena Pourmal said:

Release candidates for HDF5 1.6.10 and HDF5 1.8.4 are available for
testing with your favorite application or on the system/with
compilers
you use. You may download the source tar balls from

ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v16/hdf5-1.6.10-
pre1.tar
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v18/hdf5-1.8.4-
pre1.tar

Please let us know about the problems you find. We do not promise to
fix them, but will be glad to document and address them in the
future.

I have built 1.6.10 and 1.8.4 on an Intel Mac running Mac OS X 10.6.1
with Xcode 3.2.1's clang compiler. They built, but there are tonnes
of
warnings. Many easy to fix; many are serious. Build transcript
attached. No tests failed.

Also, I notice that sprintf() is used a lot, ex:

char meta_name[1024]
sprintf(meta_name, "%%s%s", meta_ext);

This is very poor practice; the following would be much safer (for
hopefully obvious reasons):

char meta_name[1024]
snprintf(meta_name, sizeof (meta_name), "%%s%s", meta_ext);

Does 1.6.9 build properly with high optimisation settings? If so, and
1.6.10 does not, I'd call that a showstopper regression.

···

On 10/30/09 11:10 PM, Michael Jackson said:

just seems that if a fix is "close" then why put out the 1.6.10 release
now instead of waiting for the fix to be placed into the 1.6.x branch.

--
____________________________________________________________
Sean McBride, B. Eng sean@rogue-research.com
Rogue Research www.rogue-research.com
Mac Software Developer Montréal, Québec, Canada

Sean McBride wrote:

···

On 10/30/09 11:10 PM, Michael Jackson said:

just seems that if a fix is "close" then why put out the 1.6.10 release
now instead of waiting for the fix to be placed into the 1.6.x branch.
     
Does 1.6.9 build properly with high optimisation settings? If so, and
1.6.10 does not, I'd call that a showstopper regression.

Actually, I think I do end up building with -O3 on gcc but have never noticed and problems that I am aware of. Anyone care to comment on what sections of the source are know to generate bad code by GCC? and what versions of GCC was the bad code generated with? Or was it all of the gcc versions?

Mike Jackson.

All,

gcc compilers version 3.* and higher exposed the problem in the HDF5 datatype conversion code that is currently bypassed by disabling optimization (see config/gnu-flags file in the source distribution for more details). As Quincey wrote, we have been working on the fix, but it will take time to finish it. Since none of our paying customers is affected by this bug, we will not hold the releases.

Currently there are no plans to bring this fix to 1.6 branch because it is a substantial task and we do not have resources to undertake it. We encourage everyone to switch to 1.8.* version of the library. 1.8 libraries should work seamlessly with the applications written for 1.6 and earlier. Please see "API Compatibility Macros in HDF5" document for more information http://www.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html. Please let us know about any problems you encounter when using 1.6 application with 1.8 (and later).

Have a great weekend!

Elena

···

On Oct 30, 2009, at 10:35 PM, Michael Jackson wrote:

Sean McBride wrote:

On 10/30/09 11:10 PM, Michael Jackson said:

just seems that if a fix is "close" then why put out the 1.6.10 release
now instead of waiting for the fix to be placed into the 1.6.x branch.

Does 1.6.9 build properly with high optimisation settings? If so, and
1.6.10 does not, I'd call that a showstopper regression.

Actually, I think I do end up building with -O3 on gcc but have never noticed and problems that I am aware of. Anyone care to comment on what sections of the source are know to generate bad code by GCC? and what versions of GCC was the bad code generated with? Or was it all of the gcc versions?

Mike Jackson.

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

A Friday 30 October 2009 22:43:48 Elena Pourmal escrigué:

All,

gcc compilers version 3.* and higher exposed the problem in the HDF5
datatype conversion code that is currently bypassed by disabling
optimization (see config/gnu-flags file in the source distribution for
more details). As Quincey wrote, we have been working on the fix, but
it will take time to finish it. Since none of our paying customers is
affected by this bug, we will not hold the releases.

Currently there are no plans to bring this fix to 1.6 branch because
it is a substantial task and we do not have resources to undertake it.
We encourage everyone to switch to 1.8.* version of the library.

[clip]

Well, no problem in using 1.8.x on a gcc-based platform. A more hairy issue
is that many Linux distributions will be using versions of HDF5 without your
future patch for gcc 4.x applied, and that will force the use of un-optimized
HDF5 library for long time, perhaps for several *years* in some distributions.

Just to have an insight of how much the slowdown would be, I compiled 1.8.4-
pre1 with -O0 (default) and -O2 optimization levels and then ran the PyTables
test suite (the light one) against both. For the -O0, the best time of three
runs is 33.8 s, while for -O2 is 28.3 s. This is a good 20% faster, which is
nothing to sneeze at.

Furthermore, the -O2 flag in gcc/HDF5 makes all PyTables tests to pass. The
HDF5 tests fails indeed:

Testing hard special long double -> float conversions Child
didn't exit normally.
Testing hard special long double -> double conversions Child
didn't exit normally.
Testing hard normalized long double -> signed char conversions Command
terminated by signal 11

and that seems related with long double support (which is not supported by
PyTables).

My question is: could you confirm that the HDF5/-O2 bug affects mainly to the
'long double' type, or it is more general? I'm asking this because I could
recommend my users to compile HDF5 with -O2 in order to run their applications
more quickly (but safely!) on gcc platforms.

Thanks,

···

--
Francesc Alted

Hello! Sean McBride wrote:

Also, I notice that sprintf() is used a lot, ex:

char meta_name[1024]
sprintf(meta_name, "%%s%s", meta_ext);

This is very poor practice; the following would be much safer (for hopefully obvious reasons):

char meta_name[1024]
snprintf(meta_name, sizeof (meta_name), "%%s%s", meta_ext);

.. and since snprintf leaves the buffer unterminated in the case of long output, a just-in-case null termination should be added after the snprintf statement, for example:

meta_name[(sizeof meta_name)-1]='\0';

To avoid adding such an extra statement, you could try replacing the snprintf with a safe version of snprintf:

int safe_snprintf(char *buffer, size_t size, const char *format, ...) {
    int return_value=0;
    va_list ap;
    va_start(ap, format);

    if(size>0) {
      return_value = vsnprintf(buffer, size-1, format, ap);
      buffer[size-1]='\0';
    }

    va_end(ap);
    return return_value;
}

If anyone finds any bug in that safe_snprintf function, please let us know.
Thank you.

Best Regards,
Vesa Paatero

--- Original message follows ---

···

Date: Thu, 29 Oct 2009 17:29:32 -0400
From: "Sean McBride" <sean@rogue-research.com>
To: <hdf-forum@hdfgroup.org>, "Elena Pourmal" <epourmal@hdfgroup.org>
Subject: Re: [Hdf-forum] HDF5 1.6.10 and 1.8.4 release candidates are
        available for testing
Message-ID: <20091029212932.599807380@kingu.local>
Content-Type: text/plain; charset="iso-8859-1"

On 10/28/09 10:13 PM, Elena Pourmal said:

Release candidates for HDF5 1.6.10 and HDF5 1.8.4 are available for
testing with your favorite application or on the system/with compilers
you use. You may download the source tar balls from

ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v16/hdf5-1.6.10-
pre1.tar
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots/v18/hdf5-1.8.4-
pre1.tar

Please let us know about the problems you find. We do not promise to
fix them, but will be glad to document and address them in the future.

I have built 1.6.10 and 1.8.4 on an Intel Mac running Mac OS X 10.6.1
with Xcode 3.2.1's clang compiler. They built, but there are tonnes of
warnings. Many easy to fix; many are serious. Build transcript
attached. No tests failed.

Also, I notice that sprintf() is used a lot, ex:

char meta_name[1024]
sprintf(meta_name, "%%s%s", meta_ext);

This is very poor practice; the following would be much safer (for
hopefully obvious reasons):

char meta_name[1024]
snprintf(meta_name, sizeof (meta_name), "%%s%s", meta_ext);

hth,

--
____________________________________________________________
Sean McBride, B. Eng sean@rogue-research.com
Rogue Research www.rogue-research.com
Mac Software Developer Montr?al, Qu?bec, Canada
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/zip
Size: 36550 bytes
Desc: not available
URL: <http://mail.hdfgroup.org/pipermail/hdf-forum_hdfgroup.org/attachments/20091029/b1f1e11a/attachment.zip>

------------------------------

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

End of Hdf-forum Digest, Vol 4, Issue 31
****************************************

On Mac OS X, the man page states:

"The snprintf() and vsnprintf() functions will write at most n-1 of the
characters printed into the output string (the n'th character then gets
the terminating `\0'); if the return value is greater than or equal to
the n argument, the string was too short and some of the printed
characters were discarded. The output is always null-terminated."

I would be surprised if that differed across OSes.

You're probably thinking of strncpy(). On BSD systems strlcpy() does
the right thing, and is easy to implement if it does not exist. strcpy
() of course is evil.

···

On 10/30/09 11:00 AM, Vesa Paatero said:

char meta_name[1024]
sprintf(meta_name, "%%s%s", meta_ext);

This is very poor practice; the following would be much safer (for

hopefully obvious reasons):

char meta_name[1024]
snprintf(meta_name, sizeof (meta_name), "%%s%s", meta_ext);

.. and since snprintf leaves the buffer unterminated in the case of long
output, a just-in-case null termination should be added after the
snprintf statement, for example:

--
____________________________________________________________
Sean McBride, B. Eng sean@rogue-research.com
Rogue Research www.rogue-research.com
Mac Software Developer Montréal, Québec, Canada

Hi! Sean McBride wrote:

On Mac OS X, the man page states:

"The snprintf() and vsnprintf() functions will write at most n-1 of the
characters printed into the output string (the n'th character then gets
the terminating `\0'); if the return value is greater than or equal to
the n argument, the string was too short and some of the printed
characters were discarded. The output is always null-terminated."

I would be surprised if that differed across OSes.

Unfortunately it does. In Borland compilers (BDS 2006 and earlier, it seems), the
documentation says:

···

-----------------------------------------------
Sends formatted output to a buffer of a maximum length specified by nsize.

If the number of bytes to output is:

< nsize, then all of the characters have been written, including the terminating '\0' character.

== nsize, then nsize characters are written, with no terminating '\0' character.

nsize, then only nsize characters are written, with no terminating '\0' character.

If nsize is 0, then the string will not be written to (and may be NULL).

Return Value

Number of bytes output, or, if nsize is 0, the number of bytes needed, not including the terminating '\0' character.
-----------------------------------------------

According to some web pages, snprintf was not standardized in C89/C90 but only in C99. So, it is a case of pre-C99 incompatibility.

For best compatibility, snprintf functions should be somehow secured (as proposed in my previous email). This depends, of course, on the level of determination to support pre-C99 compilation for HDF5. However, when buffers are long enough, both functions work--so the most important thing is to avoid deliberate use of the truncating capability of C99 as that may vary between platforms.

I am glad, however, that this came up now as I was planning to try compiling HDF5 with a Borland compiler in near future.

Best Regards,
Vesa Paatero

-----Original Message-----
From: Sean McBride [mailto:sean@rogue-research.com]
Sent: 30. lokakuuta 2009 17:06
To: hdf-forum@hdfgroup.org; Vesa Paatero
Subject: Re: [Hdf-forum] Hdf-forum Digest, Vol 4, Issue 31

On 10/30/09 11:00 AM, Vesa Paatero said:

char meta_name[1024]
sprintf(meta_name, "%%s%s", meta_ext);

This is very poor practice; the following would be much safer (for

hopefully obvious reasons):

char meta_name[1024]
snprintf(meta_name, sizeof (meta_name), "%%s%s", meta_ext);

.. and since snprintf leaves the buffer unterminated in the case of long
output, a just-in-case null termination should be added after the
snprintf statement, for example:

On Mac OS X, the man page states:

"The snprintf() and vsnprintf() functions will write at most n-1 of the
characters printed into the output string (the n'th character then gets
the terminating `\0'); if the return value is greater than or equal to
the n argument, the string was too short and some of the printed
characters were discarded. The output is always null-terminated."

I would be surprised if that differed across OSes.

You're probably thinking of strncpy(). On BSD systems strlcpy() does
the right thing, and is easy to implement if it does not exist. strcpy
() of course is evil.

--
____________________________________________________________
Sean McBride, B. Eng sean@rogue-research.com
Rogue Research www.rogue-research.com
Mac Software Developer Montréal, Québec, Canada