Weird H5Aread() error in HDF5 files - possibly a bug

Samer_Afach · July 23, 2015, 4:54pm

Dear pros:

So a colleague of mine created an HDF5 for testing purposes using Labview, and I used my C++ program to read attributes from it, and that worked with the attached function with no problem.

Now I created my own HDF5 files with exactly the same structure, but with the difference that I'm using non-compound types. The attribute names are exactly the same. So I started my program and loaded my new files exactly like I do with the other files I got from my colleague, and got a crash. I pursued the issue with a debugger and found the situation attached in the image. It's a very weird situation that I can't understand. On the right, you'll find that all values are positive and read return is zero (so no error), and yet the assignment operator is throwing an error and the variable str, which is char*, is not accessible.

How could this happen? I could provide any additional information you require.

Thank you for any help you provide.

All the best,
Samer

gheber · July 23, 2015, 7:16pm

Is the string a (1) fixed-length or (2) variable-length string?
(What does h5dump say? Or programmatically via "htri_t H5Tis_variable_str( hid_t dtype_id )"...)
Your code should work fine for (2) and a scalar attribute, but won't work with (1).
You'd have to allocate the buffer for the string(s) in (1) yourself.

G.

···

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of webmaster@afach.de
Sent: Thursday, July 23, 2015 11:55 AM
To: HDF Users Discussion List
Subject: [Hdf-forum] Weird H5Aread() error in HDF5 files - possibly a bug

Dear pros:

So a colleague of mine created an HDF5 for testing purposes using Labview, and I used my C++ program to read attributes from it, and that worked with the attached function with no problem.

Now I created my own HDF5 files with exactly the same structure, but with the difference that I'm using non-compound types. The attribute names are exactly the same. So I started my program and loaded my new files exactly like I do with the other files I got from my colleague, and got a crash. I pursued the issue with a debugger and found the situation attached in the image. It's a very weird situation that I can't understand. On the right, you'll find that all values are positive and read return is zero (so no error), and yet the assignment operator is throwing an error and the variable str, which is char*, is not accessible.

How could this happen? I could provide any additional information you require.

Thank you for any help you provide.

All the best,
Samer

Samer_Afach · July 23, 2015, 9:35pm

Dear Gerd, Dear experts in hdf:

The problem is even beyond reading the string (the string I wrote is

a scalar). Here I present the problem.

To make the problem clearer, I created a compilable minimal example

by truncating parts of my program and resorting it, in order to show
the problem’s weird behavior. Please compile the main.cpp file with

�      g++ main.cpp

-std=c++11 -lhdf5_hl -lhdf5 -lhdf5_hl_cpp -lhdf5_cpp -lszip -lz
-ldl -o exe

The program creates a data file, writes it, reads it again to a

string buffer, then opens it with H5LTopen_file_image(). This will
cause the program to crash when trying to read attributes. Nothing
works properly with image opening for some reason. I even introduced
std::vector to ensure contiguity in memory.

On the other hand, the provided Example.hdf5, which was provided by

a colleague, and produced with Labview, and is only different that
it uses a compound type, can be read by my program with no problems
at all. All you have to do, is uncomment the part of the code that
has the file name Example.hdf5 in main.cpp, and see how attributes
get printed with no problems.

I really, really would appreciate your help. I'm so shocked because

my software was supposed to be ready today and I was supposed to
present it, and this problem suddenly happened. Now I’m giving
excuses to people on why it’s not done. I’ve been working on this
for more than 8 hours with no result.

Thank you, and I really appreciate any efforts.

All the best,

Samer

main.cpp (7.87 KB)

Example.hdf5 (475 KB)

···

On 23.07.2015 21:16, Gerd Heber wrote:


Is the string a (1) fixed-length or (2) variable-length string?
(What does h5dump say? Or programmatically via "htri_t H5Tis_variable_str( hid_t dtype_id )"...)
Your code should work fine for (2) and a scalar attribute, but won't work with (1).
You'd have to allocate the buffer for the string(s) in (1) yourself.
G.
-----Original Message-----
From: Hdf-forum [] On Behalf Of Sent: Thursday, July 23, 2015 11:55 AM
To: HDF Users Discussion List
Subject: [Hdf-forum] Weird H5Aread() error in HDF5 files - possibly a bug
Dear pros:
So a colleague of mine created an HDF5 for testing purposes using Labview, and I used my C++ program to read attributes from it, and that worked with the attached function with no problem.
Now I created my own HDF5 files with exactly the same structure, but with the difference that I'm using non-compound types. The attribute names are exactly the same. So I started my program and loaded my new files exactly like I do with the other files I got from my colleague, and got a crash. I pursued the issue with a debugger and found the situation attached in the image. It's a very weird situation that I can't understand. On the right, you'll find that all values are positive and read return is zero (so no error), and yet the assignment operator is throwing an error and the variable str, which is char*, is not accessible.
How could this happen? I could provide any additional information you require.
Thank you for any help you provide.
All the best,
Samer
_______________________________________________
Hdf-forum is for HDF software users discussion.
Twitter:

mailto:hdf-forum-bounces@lists.hdfgroup.orgwebmaster@afach.deHdf-forum@lists.hdfgroup.orghttp://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org https://twitter.com/hdf5

bmribler · July 27, 2015, 7:07pm

Hi Samer,

I think it has something to do with variable-length vs. fixed-length string for the attribute. I tried your program and got segfault too. Then I saw that you passed in an un-allocated "str" to HDF5Attribute<std::string>::readAttribute() for an attribute which seemed to be a fixed-length type, so I allocated "str" and the segfault went away. Here is the output. Does it look right?

Success read file into memory: 0

12:21:12.222

21:22:11.444

1000

Program ended normally

I removed the free too.

Binh-Minh

···

________________________________
From: Hdf-forum <hdf-forum-bounces@lists.hdfgroup.org> on behalf of Samer Afach <webmaster@afach.de>
Sent: Thursday, July 23, 2015 5:35 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] Weird H5Aread() error in HDF5 files - possibly a bug

Dear Gerd, Dear experts in hdf:

The problem is even beyond reading the string (the string I wrote is a scalar). Here I present the problem.

To make the problem clearer, I created a compilable minimal example by truncating parts of my program and resorting it, in order to show the problem's weird behavior. Please compile the main.cpp file with

g++ main.cpp -std=c++11 -lhdf5_hl -lhdf5 -lhdf5_hl_cpp -lhdf5_cpp -lszip -lz -ldl -o exe

The program creates a data file, writes it, reads it again to a string buffer, then opens it with H5LTopen_file_image(). This will cause the program to crash when trying to read attributes. Nothing works properly with image opening for some reason. I even introduced std::vector<char> to ensure contiguity in memory.

On the other hand, the provided Example.hdf5, which was provided by a colleague, and produced with Labview, and is only different that it uses a compound type, can be read by my program with no problems at all. All you have to do, is uncomment the part of the code that has the file name Example.hdf5 in main.cpp, and see how attributes get printed with no problems.

I really, really would appreciate your help. I'm so shocked because my software was supposed to be ready today and I was supposed to present it, and this problem suddenly happened. Now I'm giving excuses to people on why it's not done. I've been working on this for more than 8 hours with no result.

Thank you, and I really appreciate any efforts.

All the best,
Samer

On 23.07.2015 21:16, Gerd Heber wrote:

Is the string a (1) fixed-length or (2) variable-length string?
(What does h5dump say? Or programmatically via "htri_t H5Tis_variable_str( hid_t dtype_id )"...)
Your code should work fine for (2) and a scalar attribute, but won't work with (1).
You'd have to allocate the buffer for the string(s) in (1) yourself.

G.

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of webmaster@afach.de<mailto:webmaster@afach.de>
Sent: Thursday, July 23, 2015 11:55 AM
To: HDF Users Discussion List
Subject: [Hdf-forum] Weird H5Aread() error in HDF5 files - possibly a bug

Dear pros:

So a colleague of mine created an HDF5 for testing purposes using Labview, and I used my C++ program to read attributes from it, and that worked with the attached function with no problem.

Now I created my own HDF5 files with exactly the same structure, but with the difference that I'm using non-compound types. The attribute names are exactly the same. So I started my program and loaded my new files exactly like I do with the other files I got from my colleague, and got a crash. I pursued the issue with a debugger and found the situation attached in the image. It's a very weird situation that I can't understand. On the right, you'll find that all values are positive and read return is zero (so no error), and yet the assignment operator is throwing an error and the variable str, which is char*, is not accessible.

How could this happen? I could provide any additional information you require.

Thank you for any help you provide.

All the best,
Samer

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

bmribler · July 27, 2015, 7:08pm

One more thing, I also removed the '&' from "&str" in:

readError = H5Aread(attribHandler,attribType,&str);

Binh-Minh

···

________________________________
From: Hdf-forum <hdf-forum-bounces@lists.hdfgroup.org> on behalf of Binh-Minh Ribler <bmribler@hdfgroup.org>
Sent: Monday, July 27, 2015 3:07 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] Weird H5Aread() error in HDF5 files - possibly a bug

Hi Samer,

I think it has something to do with variable-length vs. fixed-length string for the attribute. I tried your program and got segfault too. Then I saw that you passed in an un-allocated "str" to HDF5Attribute<std::string>::readAttribute() for an attribute which seemed to be a fixed-length type, so I allocated "str" and the segfault went away. Here is the output. Does it look right?

Success read file into memory: 0

12:21:12.222

21:22:11.444

1000

Program ended normally

I removed the free too.

Binh-Minh

________________________________
From: Hdf-forum <hdf-forum-bounces@lists.hdfgroup.org> on behalf of Samer Afach <webmaster@afach.de>
Sent: Thursday, July 23, 2015 5:35 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] Weird H5Aread() error in HDF5 files - possibly a bug

Dear Gerd, Dear experts in hdf:

The problem is even beyond reading the string (the string I wrote is a scalar). Here I present the problem.

To make the problem clearer, I created a compilable minimal example by truncating parts of my program and resorting it, in order to show the problem's weird behavior. Please compile the main.cpp file with

g++ main.cpp -std=c++11 -lhdf5_hl -lhdf5 -lhdf5_hl_cpp -lhdf5_cpp -lszip -lz -ldl -o exe

The program creates a data file, writes it, reads it again to a string buffer, then opens it with H5LTopen_file_image(). This will cause the program to crash when trying to read attributes. Nothing works properly with image opening for some reason. I even introduced std::vector<char> to ensure contiguity in memory.

On the other hand, the provided Example.hdf5, which was provided by a colleague, and produced with Labview, and is only different that it uses a compound type, can be read by my program with no problems at all. All you have to do, is uncomment the part of the code that has the file name Example.hdf5 in main.cpp, and see how attributes get printed with no problems.

I really, really would appreciate your help. I'm so shocked because my software was supposed to be ready today and I was supposed to present it, and this problem suddenly happened. Now I'm giving excuses to people on why it's not done. I've been working on this for more than 8 hours with no result.

Thank you, and I really appreciate any efforts.

All the best,
Samer

On 23.07.2015 21:16, Gerd Heber wrote:

Is the string a (1) fixed-length or (2) variable-length string?
(What does h5dump say? Or programmatically via "htri_t H5Tis_variable_str( hid_t dtype_id )"...)
Your code should work fine for (2) and a scalar attribute, but won't work with (1).
You'd have to allocate the buffer for the string(s) in (1) yourself.

G.

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of webmaster@afach.de<mailto:webmaster@afach.de>
Sent: Thursday, July 23, 2015 11:55 AM
To: HDF Users Discussion List
Subject: [Hdf-forum] Weird H5Aread() error in HDF5 files - possibly a bug

Dear pros:

So a colleague of mine created an HDF5 for testing purposes using Labview, and I used my C++ program to read attributes from it, and that worked with the attached function with no problem.

Now I created my own HDF5 files with exactly the same structure, but with the difference that I'm using non-compound types. The attribute names are exactly the same. So I started my program and loaded my new files exactly like I do with the other files I got from my colleague, and got a crash. I pursued the issue with a debugger and found the situation attached in the image. It's a very weird situation that I can't understand. On the right, you'll find that all values are positive and read return is zero (so no error), and yet the assignment operator is throwing an error and the variable str, which is char*, is not accessible.

How could this happen? I could provide any additional information you require.

Thank you for any help you provide.

All the best,
Samer

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

Samer_Afach · July 27, 2015, 7:12pm

Dear Binh:

Thank you for your response. Yes you are right. This solved the

problem. There’s a major difference between variable and fixed
strings in many ways that I had never realized.

Thank you.

All the best,

Samer

···

On 27.07.2015 21:07, Binh-Minh Ribler
wrote:

Hi Samer,

      I think it has something to do with variable-length vs.
fixed-length string for the attribute. �I tried your program
and got segfault too. �Then I saw that you passed in an
un-allocated
�"str"�to�HDF5Attributestd::string::readAttribute()
for an attribute which seemed to be a fixed-length type, so I
allocated “str” and the segfault went away. �Here is the
output. �Does it look right?

Success read file into memory: 0

12:21:12.222

21:22:11.444

1000

Program ended normally

I removed the free too.

Binh-Minh

From:
Hdf-forum on
behalf of Samer Afach Thursday, July 23, 2015 5:35 PM
HDF Users Discussion List
Re: [Hdf-forum] Weird H5Aread() error in
HDF5 files - possibly a bug
�

Dear Gerd, Dear experts in hdf:
        The problem is even beyond reading the string (the string I
wrote is a scalar). Here I present the problem.
        To make the problem clearer, I created a compilable minimal
example by truncating parts of my program and resorting it,
in order to show the problem’s weird behavior. Please
compile the main.cpp file with
        �              g++ main.cpp
-std=c++11 -lhdf5_hl -lhdf5 -lhdf5_hl_cpp -lhdf5_cpp
-lszip -lz -ldl -o exe
        The program creates a data file, writes it, reads it again
to a string buffer, then opens it with
H5LTopen_file_image(). This will cause the program to crash
when trying to read attributes. Nothing works properly with
image opening for some reason. I even introduced
std::vector to ensure contiguity in memory.
        On the other hand, the provided Example.hdf5, which was
provided by a colleague, and produced with Labview, and is
only different that it uses a compound type, can be read by
my program with no problems at all. All you have to do, is
uncomment the part of the code that has the file name
Example.hdf5 in main.cpp, and see how attributes get printed
with no problems.
        I really, really would appreciate your help. I'm so shocked
because my software was supposed to be ready today and I was
supposed to present it, and this problem suddenly happened.
Now I’m giving excuses to people on why it’s not done. I’ve
been working on this for more than 8 hours with no result.
        Thank you, and I really appreciate any efforts.



        All the best,

        Samer



_______________________________________________
Hdf-forum is for HDF software users discussion.
Twitter:

hdf-forum-bounces@lists.hdfgroup.org webmaster@afach.de
Sent:
To:
Subject:
On 23.07.2015 21:16, Gerd Heber
wrote:


Is the string a (1) fixed-length or (2) variable-length string?
(What does h5dump say? Or programmatically via "htri_t H5Tis_variable_str( hid_t dtype_id )"...)
Your code should work fine for (2) and a scalar attribute, but won't work with (1).
You'd have to allocate the buffer for the string(s) in (1) yourself.
G.
-----Original Message-----
From: Hdf-forum [] On Behalf Of Sent: Thursday, July 23, 2015 11:55 AM
To: HDF Users Discussion List
Subject: [Hdf-forum] Weird H5Aread() error in HDF5 files - possibly a bug
Dear pros:
So a colleague of mine created an HDF5 for testing purposes using Labview, and I used my C++ program to read attributes from it, and that worked with the attached function with no problem.
Now I created my own HDF5 files with exactly the same structure, but with the difference that I'm using non-compound types. The attribute names are exactly the same. So I started my program and loaded my new files exactly like I do with the other files I got from my colleague, and got a crash. I pursued the issue with a debugger and found the situation attached in the image. It's a very weird situation that I can't understand. On the right, you'll find that all values are positive and read return is zero (so no error), and yet the assignment operator is throwing an error and the variable str, which is char*, is not accessible.
How could this happen? I could provide any additional information you require.
Thank you for any help you provide.
All the best,
Samer
_______________________________________________
Hdf-forum is for HDF software users discussion.
Twitter:

mailto:hdf-forum-bounces@lists.hdfgroup.orgwebmaster@afach.deHdf-forum@lists.hdfgroup.orghttp://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org https://twitter.com/hdf5
Hdf-forum@lists.hdfgroup.org http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org https://twitter.com/hdf5

bmribler · July 27, 2015, 7:13pm

You're welcome. Good luck!

Binh-Minh

···

________________________________
From: Hdf-forum <hdf-forum-bounces@lists.hdfgroup.org> on behalf of Samer Afach <webmaster@afach.de>
Sent: Monday, July 27, 2015 3:12 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] Weird H5Aread() error in HDF5 files - possibly a bug

Dear Binh:

Thank you for your response. Yes you are right. This solved the problem. There's a major difference between variable and fixed strings in many ways that I had never realized.

Thank you.

All the best,
Samer

On 27.07.2015 21:07, Binh-Minh Ribler wrote:

Hi Samer,

I think it has something to do with variable-length vs. fixed-length string for the attribute. I tried your program and got segfault too. Then I saw that you passed in an un-allocated "str" to HDF5Attribute<std::string>::readAttribute() for an attribute which seemed to be a fixed-length type, so I allocated "str" and the segfault went away. Here is the output. Does it look right?

Success read file into memory: 0

12:21:12.222

21:22:11.444

1000

Program ended normally

I removed the free too.

Binh-Minh

________________________________
From: Hdf-forum <hdf-forum-bounces@lists.hdfgroup.org><mailto:hdf-forum-bounces@lists.hdfgroup.org> on behalf of Samer Afach <webmaster@afach.de><mailto:webmaster@afach.de>
Sent: Thursday, July 23, 2015 5:35 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] Weird H5Aread() error in HDF5 files - possibly a bug

Dear Gerd, Dear experts in hdf:

The problem is even beyond reading the string (the string I wrote is a scalar). Here I present the problem.

To make the problem clearer, I created a compilable minimal example by truncating parts of my program and resorting it, in order to show the problem's weird behavior. Please compile the main.cpp file with

g++ main.cpp -std=c++11 -lhdf5_hl -lhdf5 -lhdf5_hl_cpp -lhdf5_cpp -lszip -lz -ldl -o exe

The program creates a data file, writes it, reads it again to a string buffer, then opens it with H5LTopen_file_image(). This will cause the program to crash when trying to read attributes. Nothing works properly with image opening for some reason. I even introduced std::vector<char> to ensure contiguity in memory.

On the other hand, the provided Example.hdf5, which was provided by a colleague, and produced with Labview, and is only different that it uses a compound type, can be read by my program with no problems at all. All you have to do, is uncomment the part of the code that has the file name Example.hdf5 in main.cpp, and see how attributes get printed with no problems.

I really, really would appreciate your help. I'm so shocked because my software was supposed to be ready today and I was supposed to present it, and this problem suddenly happened. Now I'm giving excuses to people on why it's not done. I've been working on this for more than 8 hours with no result.

Thank you, and I really appreciate any efforts.

All the best,
Samer

On 23.07.2015 21:16, Gerd Heber wrote:

Is the string a (1) fixed-length or (2) variable-length string?
(What does h5dump say? Or programmatically via "htri_t H5Tis_variable_str( hid_t dtype_id )"...)
Your code should work fine for (2) and a scalar attribute, but won't work with (1).
You'd have to allocate the buffer for the string(s) in (1) yourself.

G.

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of webmaster@afach.de<mailto:webmaster@afach.de>
Sent: Thursday, July 23, 2015 11:55 AM
To: HDF Users Discussion List
Subject: [Hdf-forum] Weird H5Aread() error in HDF5 files - possibly a bug

Dear pros:

So a colleague of mine created an HDF5 for testing purposes using Labview, and I used my C++ program to read attributes from it, and that worked with the attached function with no problem.

Now I created my own HDF5 files with exactly the same structure, but with the difference that I'm using non-compound types. The attribute names are exactly the same. So I started my program and loaded my new files exactly like I do with the other files I got from my colleague, and got a crash. I pursued the issue with a debugger and found the situation attached in the image. It's a very weird situation that I can't understand. On the right, you'll find that all values are positive and read return is zero (so no error), and yet the assignment operator is throwing an error and the variable str, which is char*, is not accessible.

How could this happen? I could provide any additional information you require.

Thank you for any help you provide.

All the best,
Samer

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5