NaN detection?

Hdf-forum archives
1

It doesn't look like h5diff does the right thing with NaNs.

A NaN should never be equal to itself, yet h5diff happily says they are equal. I tried using --nan, which I know is supposed to turn *off* NaN detection, just in case it was backwards.

QASNOW:~>h5diff --version
h5diff: Version 1.8.10
QASNOW:~>h5dump test.h5 | grep nan | head -2
      (18,345,0,0): 1.35366e+06, nan, 1.28531e-11,
      (19,345,0,0): 9.00816e+06, nan, -4.65661e-10,
QASNOW:~>h5diff test.h5 test.h5
QASNOW:~>echo $?
0
QASNOW:~>h5diff --nan test.h5 test.h5
QASNOW:~>echo $?
0

This is on OS X, but the behavior is the same on my Linux machine.

For now we can do the h5dump | grep nan, as I did above to ensure our programs are not dumping NaNs.

Thanks,
Dave

2

A NaN should never be equal to itself, yet h5diff happily says they are
equal.

Though clearly NaN != NaN from an IEEE equality perspective, h5diff
tells you the differences between things. A NaN is not "different"
from a NaN in the sense that two files containing identical bit
patterns (some of which may be NaN) are not different.

- Rhys

3

I agree that h5diff is just comparing bit patterns, but obviously h5diff knows when a bit pattern equals NaN and when it doesn't. So it could refuse to says that two NaNs are equal.

At the very least there ought to be a command-line option to make it so h5diff reports NaNs as different.

Dave

···

On Feb 16, 2013, at 1:12 PM, Rhys Ulerich wrote:

A NaN should never be equal to itself, yet h5diff happily says they are
equal.

Though clearly NaN != NaN from an IEEE equality perspective, h5diff
tells you the differences between things. A NaN is not "different"
from a NaN in the sense that two files containing identical bit
patterns (some of which may be NaN) are not different.

- Rhys

4

I agree that h5diff is just comparing bit patterns, but obviously h5diff

knows when a bit pattern equals NaN and when it doesn't. So it could refuse
to says that two NaNs are equal.

But then a file containing a NaN would be different from itself, a clearly
useless state of affairs.

5

This was exactly the reason why The HDF Group decided that h5diff should behave this way. BTW, the current h5diff behavior with NaN is documented in the tool's RM page http://www.hdfgroup.org/HDF5/doc/RM/Tools.html#Tools-Diff.

Dave,

In your opinion, what would be the use case when NaNs should be treated as not equal when compared to each other by h5diff?

Thank you!

Elena

···

On Feb 16, 2013, at 4:59 PM, Rhys Ulerich wrote:

> I agree that h5diff is just comparing bit patterns, but obviously h5diff knows when a bit pattern equals NaN and when it doesn't. So it could refuse to says that two NaNs are equal.

But then a file containing a NaN would be different from itself, a clearly useless state of affairs.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Elena Pourmal The HDF Group http://hdfgroup.org
1800 So. Oak St., Suite 203, Champaign IL 61820
217.531.6112
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

6

While I agree that two NaNs are technically equal, in the hdf5 sense, a NaN always indicates a programmatic/numerical error. So an option to h5diff which alerted the user to the presence of NaNs is not "clearly useless."

I want to know immediately when a code is producing them. It doesn't seem unreasonable to assume that NaNs could be detected when h5diff-ing, especially given that h5diff is doing NaN detection in the first place.

We've had cases where NaNs crept in to accepted results, and then because the code continued to produce NaNs, we assumed the code was working because h5diff showed no differences. I can of course prevent this in the future by doing an h5dump of each generated hdf5 file, and grep-ing for NaNs, and I guess that's what I'll do.

Dave

···

> I agree that h5diff is just comparing bit patterns, but obviously h5diff knows when a bit pattern equals NaN and when it doesn't. So it could refuse to says that two NaNs are equal.

But then a file containing a NaN would be different from itself, a clearly useless state of affairs.

This was exactly the reason why The HDF Group decided that h5diff should behave this way. BTW, the current h5diff behavior with NaN is documented in the tool's RM page http://www.hdfgroup.org/HDF5/doc/RM/Tools.html#Tools-Diff.

Dave,

In your opinion, what would be the use case when NaNs should be treated as not equal when compared to each other by h5diff?

Thank you!

Elena

7

Hi Dave,

While I agree that two NaNs are technically equal, in the hdf5 sense, a NaN always indicates a programmatic/numerical error. So an option to h5diff which alerted the user to the presence of NaNs is not "clearly useless."

I want to know immediately when a code is producing them. It doesn't seem unreasonable to assume that NaNs could be detected when h5diff-ing, especially given that h5diff is doing NaN detection in the first place.

We've had cases where NaNs crept in to accepted results, and then because the code continued to produce NaNs, we assumed the code was working because h5diff showed no differences. I can of course prevent this in the future by doing an h5dump of each generated hdf5 file, and grep-ing for NaNs, and I guess that's what I'll do.

I don't think we thought about the HDF5 tools as of data validation tools.

Using h5dump to dump GBs or TBs of data to get NaN doesn't make much sense if h5diff already reads each element and reports the differences, i.e., I see your point. My personal opinion is that an application should be checking the validity of the data, but I can see the HDF5 tools as a "second line of defense" :slight_smile:

If we provide an option to h5diff to choose NaN behavior, will it help? What do other members of this FORUM think?

Thank you!

Elena

···

On Feb 16, 2013, at 6:53 PM, Dave Wade-Stein wrote:

Dave

> I agree that h5diff is just comparing bit patterns, but obviously h5diff knows when a bit pattern equals NaN and when it doesn't. So it could refuse to says that two NaNs are equal.

But then a file containing a NaN would be different from itself, a clearly useless state of affairs.

This was exactly the reason why The HDF Group decided that h5diff should behave this way. BTW, the current h5diff behavior with NaN is documented in the tool's RM page http://www.hdfgroup.org/HDF5/doc/RM/Tools.html#Tools-Diff.

Dave,

In your opinion, what would be the use case when NaNs should be treated as not equal when compared to each other by h5diff?

Thank you!

Elena

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

8

I don't think we thought about the HDF5 tools as of data validation tools.

Right, it's not obvious, but as I said, once your codes start producing NaNs, something is wrong, so in the long term, you don't want your h5 files containing NaNs, and you want to know when it occurs.

Using h5dump to dump GBs or TBs of data to get NaN doesn't make much sense if h5diff already reads each element and reports the differences, i.e., I see your point. My personal opinion is that an application should be checking the validity of the data, but I can see the HDF5 tools as a "second line of defense" :slight_smile:

Yes, many of our files are GBs of data, so h5dump-ing them to find NaNs doesn't make sense given that h5diff is reading the data.

As for an app checking the validity of its dumped data, I agree in principle, but in practice that would result in additional overhead for scientific codes. In my case, I'd much rather do it after the run, using h5 tools, rather than at each dump of the data.

If we provide an option to h5diff to choose NaN behavior, will it help? What do other members of this FORUM think?

For us, it would be great.

I'd love to be able to do

h5diff --check-for-nans file1.h5 file2.h5

and have it tell me that NaNs were detected in the files.

Thanks!
Dave

9

Dave,

I don't think we thought about the HDF5 tools as of data validation tools.

Right, it's not obvious, but as I said, once your codes start producing NaNs, something is wrong, so in the long term, you don't want your h5 files containing NaNs, and you want to know when it occurs.

Using h5dump to dump GBs or TBs of data to get NaN doesn't make much sense if h5diff already reads each element and reports the differences, i.e., I see your point. My personal opinion is that an application should be checking the validity of the data, but I can see the HDF5 tools as a "second line of defense" :slight_smile:

Yes, many of our files are GBs of data, so h5dump-ing them to find NaNs doesn't make sense given that h5diff is reading the data.

As for an app checking the validity of its dumped data, I agree in principle, but in practice that would result in additional overhead for scientific codes. In my case, I'd much rather do it after the run, using h5 tools, rather than at each dump of the data.

If we provide an option to h5diff to choose NaN behavior, will it help? What do other members of this FORUM think?

For us, it would be great.

I'd love to be able to do

h5diff --check-for-nans file1.h5 file2.h5

and have it tell me that NaNs were detected in the files.

I entered an enhancement request into our issues DB.

Elena

···

On Feb 17, 2013, at 5:10 PM, Dave Wade-Stein wrote:

Thanks!
Dave
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

10

Thanks! Your help is greatly appreciated.

Dave

···

On Feb 17, 2013, at 8:01 PM, Elena Pourmal wrote:

Dave,

On Feb 17, 2013, at 5:10 PM, Dave Wade-Stein wrote:

I don't think we thought about the HDF5 tools as of data validation tools.

Right, it's not obvious, but as I said, once your codes start producing NaNs, something is wrong, so in the long term, you don't want your h5 files containing NaNs, and you want to know when it occurs.

Using h5dump to dump GBs or TBs of data to get NaN doesn't make much sense if h5diff already reads each element and reports the differences, i.e., I see your point. My personal opinion is that an application should be checking the validity of the data, but I can see the HDF5 tools as a "second line of defense" :slight_smile:

Yes, many of our files are GBs of data, so h5dump-ing them to find NaNs doesn't make sense given that h5diff is reading the data.

As for an app checking the validity of its dumped data, I agree in principle, but in practice that would result in additional overhead for scientific codes. In my case, I'd much rather do it after the run, using h5 tools, rather than at each dump of the data.

If we provide an option to h5diff to choose NaN behavior, will it help? What do other members of this FORUM think?

For us, it would be great.

I'd love to be able to do

h5diff --check-for-nans file1.h5 file2.h5

and have it tell me that NaNs were detected in the files.

I entered an enhancement request into our issues DB.

Elena

Thanks!
Dave

11

I think adding a flag to specify the NaN behavior is a great idea. I also
think that, h5diff might not be the most efficient and useful tool to find
NaNs.

IMO, it might be better to create a specific tool to search an h5 file for
NaNs or maybe add the ability of the library to throw an error if a NaN is
written. (A diff will necessitate a bit-wise comparison of every element,
whereas if you are only looking for NaNs then there's no need to read in an
element twice and then perform a bitwise comparison. It may be faster to
read each element in once, and then only NaNs will need to have all their
bits checked--vectorization and optimization considerations neglected.)

It is ultimately the burden of the developer/application to ensure that one
is not chugging along computing NaNs and burning precious CPU hours, but
the whole point of HDF5 is to make data manipulation easier, and data sets
self descriptive. In this regard, the ability to optionally throw an
exception when a NaN is written, or perhaps track NaNs and their locations
would likely be useful to many users.

Izaak Beekman

···

===================================
(301)244-9367
UMD-CP Visiting Graduate Student
Aerospace Engineering
ibeekman@umiacs.umd.edu
ibeekman@umd.edu

12

It would seem this functionality could be implemented via a user-specified filter, similar to data conversion and compression of datasets, but in this case just passing-through all numbers except NaN's, which may then employ even a platform-specific isnan() function or some bit test; throwing a C++ exception from such a filter might not be a good idea since that would bypass cleaning up HDF5-internal objects during data writing, but at least it could lead to a "write error" or similar, and create an application-specific error stack with locations where NaN's have been detected.

          Werner

···

On Wed, 20 Feb 2013 04:41:53 +0100, Izaak Beekman <ibeekman@umiacs.umd.edu> wrote:

I think adding a flag to specify the NaN behavior is a great idea. I also think that, h5diff might not be the most efficient and useful tool to find NaNs.
IMO, it might be better to create a specific tool to search an h5 file for NaNs or maybe add the ability of the library to throw an error if a NaN is written. (A diff will necessitate a bit-wise comparison of every element, whereas if you are only looking for NaNs >then there's no need to read in an element twice and then perform a bitwise comparison. It may be faster to read each element in once, and then only NaNs will need to have all their bits checked--vectorization and optimization considerations neglected.)
It is ultimately the burden of the developer/application to ensure that one is not chugging along computing NaNs and burning precious CPU hours, but the whole point of HDF5 is to make data manipulation easier, and data sets self descriptive. In this >regard, the ability to optionally throw an exception when a NaN is written, or perhaps track NaNs and their locations would likely be useful to many users.

Izaak Beekman

(301)244-9367
UMD-CP Visiting Graduate Student
Aerospace Engineering
ibeekman@umiacs.umd.edu
ibeekman@umd.edu

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

13

I mean exception in the abstract sense, just some error, not specific to C++

Izaak Beekman

···

===================================
(301)244-9367
UMD-CP Visiting Graduate Student
Aerospace Engineering
ibeekman@umiacs.umd.edu
ibeekman@umd.edu

On Wed, Feb 20, 2013 at 2:46 AM, Werner Benger <werner@cct.lsu.edu> wrote:

**
It would seem this functionality could be implemented via a user-specified
filter, similar to data conversion and compression of datasets, but in this
case just passing-through all numbers except NaN's, which may then employ
even a platform-specific isnan() function or some bit test; throwing a C++
exception from such a filter might not be a good idea since that would
bypass cleaning up HDF5-internal objects during data writing, but at least
it could lead to a "write error" or similar, and create an
application-specific error stack with locations where NaN's have been
detected.

         Werner

On Wed, 20 Feb 2013 04:41:53 +0100, Izaak Beekman <ibeekman@umiacs.umd.edu> > wrote:

I think adding a flag to specify the NaN behavior is a great idea. I also
think that, h5diff might not be the most efficient and useful tool to find
NaNs.

IMO, it might be better to create a specific tool to search an h5 file for
NaNs or maybe add the ability of the library to throw an error if a NaN is
written. (A diff will necessitate a bit-wise comparison of every element,
whereas if you are only looking for NaNs then there's no need to read in an
element twice and then perform a bitwise comparison. It may be faster to
read each element in once, and then only NaNs will need to have all their
bits checked--vectorization and optimization considerations neglected.)

It is ultimately the burden of the developer/application to ensure that
one is not chugging along computing NaNs and burning precious CPU hours,
but the whole point of HDF5 is to make data manipulation easier, and data
sets self descriptive. In this regard, the ability to optionally throw an
exception when a NaN is written, or perhaps track NaNs and their locations
would likely be useful to many users.

Izaak Beekman

(301)244-9367
UMD-CP Visiting Graduate Student
Aerospace Engineering
ibeekman@umiacs.umd.edu
ibeekman@umd.edu

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362