I've run into the following problem while using HDF5: I can't create an attribute for an empty dataset when application is executed in parallel.
I have some dataset with total number of elements N. I'm writing it to a file as 1d dataset.
When application is executed in parallel, each process writes it's own piece of data.
After saving the dataset to the file, I want to attach some attributes to it.
And here is the problem emerges: if total number of elements N is zero, i.e. dataset is empty,
my attemp to create an attribute results in an error and program termination.
It doesn't happen if the dataset is nonempty.
Besides, serial version of the program works fine it both cases.
I must be doing something wrong, but I can't figure out what exactly.
What is the correct way to create an attribute for a dataset in parallel?
Attached is the source code for the parallel and the serial versions of the program.
I'm using Debian with libhdf5-1.8.13.
Here is the error message I get, after compiling and running 'attr_for_empty_dset_prll.c'
( see the attachment and also the previous letter ). Can somebody confirm it?
HDF5-DIAG: Error detected in HDF5 (1.8.13) MPI-process 0: #000: ../../../src/H5D.c line 426 in H5Dclose(): can't decrement count on dataset ID
major: Dataset
minor: Unable to decrement reference count #001: ../../../src/H5I.c line 1533 in H5I_dec_app_ref_always_close(): can't decrement ID ref count
major: Object atom
minor: Unable to decrement reference count #002: ../../../src/H5I.c line 1475 in H5I_dec_app_ref(): can't decrement ID ref count
major: Object atom
minor: Unable to decrement reference count #003: ../../../src/H5Dint.c line 1386 in H5D_close(): unable to flush cached dataset info
major: Dataset
minor: Write failed #004: ../../../src/H5Dint.c line 2316 in H5D__flush_real(): unable to update layout/pline/efl info
major: Dataset
minor: Write failed #005: ../../../src/H5Dlayout.c line 479 in H5D__layout_oh_write(): unable to update layout message
major: Dataset
minor: Write failed #006: ../../../src/H5Omessage.c line 335 in H5O_msg_write_oh(): unable to write object header message
major: Object header
minor: Write failed #007: ../../../src/H5Omessage.c line 389 in H5O_msg_write_real(): unable to modify constant message
major: Object header
minor: Write failed
On 11/24/2015 06:46 PM, A.B. wrote:
Hello.
I've run into the following problem while using HDF5: I can't create an attribute for an empty dataset when application is executed in parallel.
I have some dataset with total number of elements N. I'm writing it to a file as 1d dataset.
When application is executed in parallel, each process writes it's own piece of data.
After saving the dataset to the file, I want to attach some attributes to it.
And here is the problem emerges: if total number of elements N is zero, i.e. dataset is empty,
my attemp to create an attribute results in an error and program termination.
It doesn't happen if the dataset is nonempty.
Besides, serial version of the program works fine it both cases.
I must be doing something wrong, but I can't figure out what exactly.
What is the correct way to create an attribute for a dataset in parallel?
Attached is the source code for the parallel and the serial versions of the program.
I'm using Debian with libhdf5-1.8.13.
Here is the error message I get, after compiling and running
'attr_for_empty_dset_prll.c'
( see the attachment and also the previous letter ). Can somebody
confirm it?
Wow, I have never seen anyone else encounter "unable to modify constant message" before.
We never did figure out what was going on with my particular environment, but when I tried to reproduce it with HDF5-1.8.14 and newer, it seemed to fix the problem.
On the HDF5 side of things, this was ticket HDFFV-8762 , but perhaps your best move is to build the latest Hdf5. not as simple as using debians hdf5-1.8.13 but you'll pick up a couple years worth of bug fixes.
==rob
···
On 11/26/2015 03:59 AM, ab wrote:
HDF5-DIAG: Error detected in HDF5 (1.8.13) MPI-process 0: #000: ../../../src/H5D.c line 426 in H5Dclose(): can't decrement
count on dataset ID
major: Dataset
minor: Unable to decrement reference count #001: ../../../src/H5I.c line 1533 in
H5I_dec_app_ref_always_close(): can't decrement ID ref count
major: Object atom
minor: Unable to decrement reference count #002: ../../../src/H5I.c line 1475 in H5I_dec_app_ref(): can't
decrement ID ref count
major: Object atom
minor: Unable to decrement reference count #003: ../../../src/H5Dint.c line 1386 in H5D_close(): unable to
flush cached dataset info
major: Dataset
minor: Write failed #004: ../../../src/H5Dint.c line 2316 in H5D__flush_real(): unable
to update layout/pline/efl info
major: Dataset
minor: Write failed #005: ../../../src/H5Dlayout.c line 479 in H5D__layout_oh_write():
unable to update layout message
major: Dataset
minor: Write failed #006: ../../../src/H5Omessage.c line 335 in H5O_msg_write_oh():
unable to write object header message
major: Object header
minor: Write failed #007: ../../../src/H5Omessage.c line 389 in H5O_msg_write_real():
unable to modify constant message
major: Object header
minor: Write failed
On 11/24/2015 06:46 PM, A.B. wrote:
Hello.
I've run into the following problem while using HDF5: I can't create
an attribute for an empty dataset when application is executed in
parallel.
I have some dataset with total number of elements N. I'm writing it to
a file as 1d dataset.
When application is executed in parallel, each process writes it's own
piece of data.
After saving the dataset to the file, I want to attach some attributes
to it.
And here is the problem emerges: if total number of elements N is
zero, i.e. dataset is empty,
my attemp to create an attribute results in an error and program
termination.
It doesn't happen if the dataset is nonempty.
Besides, serial version of the program works fine it both cases.
I must be doing something wrong, but I can't figure out what exactly.
What is the correct way to create an attribute for a dataset in parallel?
Attached is the source code for the parallel and the serial versions
of the program.
I'm using Debian with libhdf5-1.8.13.
I thought I replied to this earlier, but apparently I did not.
I managed to replicate this with trunk.
I entered ticket HDFFV-9622
Rob, I see that the ticket you mentioned is marked as not an issue anymore in our system, which means that it was fixed at some point. But I believe that the issue here is very similar triggering the "unable to modify constant message". We'll investigate.
Thanks,
Mohamad
···
-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Rob Latham
Sent: Monday, December 07, 2015 2:08 PM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] in parallel program, attempt to create an attribute for an empty dataset results in error; how to avoid
On 11/26/2015 03:59 AM, ab wrote:
Here is the error message I get, after compiling and running
'attr_for_empty_dset_prll.c'
( see the attachment and also the previous letter ). Can somebody
confirm it?
Wow, I have never seen anyone else encounter "unable to modify constant message" before.
We never did figure out what was going on with my particular environment, but when I tried to reproduce it with HDF5-1.8.14 and newer, it seemed to fix the problem.
On the HDF5 side of things, this was ticket HDFFV-8762 , but perhaps your best move is to build the latest Hdf5. not as simple as using debians hdf5-1.8.13 but you'll pick up a couple years worth of bug fixes.
==rob
HDF5-DIAG: Error detected in HDF5 (1.8.13) MPI-process 0: #000: ../../../src/H5D.c line 426 in H5Dclose(): can't decrement
count on dataset ID
major: Dataset
minor: Unable to decrement reference count #001: ../../../src/H5I.c line 1533 in
H5I_dec_app_ref_always_close(): can't decrement ID ref count
major: Object atom
minor: Unable to decrement reference count #002: ../../../src/H5I.c line 1475 in H5I_dec_app_ref(): can't
decrement ID ref count
major: Object atom
minor: Unable to decrement reference count #003: ../../../src/H5Dint.c line 1386 in H5D_close(): unable to
flush cached dataset info
major: Dataset
minor: Write failed #004: ../../../src/H5Dint.c line 2316 in H5D__flush_real(): unable
to update layout/pline/efl info
major: Dataset
minor: Write failed #005: ../../../src/H5Dlayout.c line 479 in H5D__layout_oh_write():
unable to update layout message
major: Dataset
minor: Write failed #006: ../../../src/H5Omessage.c line 335 in H5O_msg_write_oh():
unable to write object header message
major: Object header
minor: Write failed #007: ../../../src/H5Omessage.c line 389 in H5O_msg_write_real():
unable to modify constant message
major: Object header
minor: Write failed
On 11/24/2015 06:46 PM, A.B. wrote:
Hello.
I've run into the following problem while using HDF5: I can't create
an attribute for an empty dataset when application is executed in
parallel.
I have some dataset with total number of elements N. I'm writing it
to a file as 1d dataset.
When application is executed in parallel, each process writes it's
own piece of data.
After saving the dataset to the file, I want to attach some
attributes to it.
And here is the problem emerges: if total number of elements N is
zero, i.e. dataset is empty, my attemp to create an attribute results
in an error and program termination.
It doesn't happen if the dataset is nonempty.
Besides, serial version of the program works fine it both cases.
I must be doing something wrong, but I can't figure out what exactly.
What is the correct way to create an attribute for a dataset in parallel?
Attached is the source code for the parallel and the serial versions
of the program.
I'm using Debian with libhdf5-1.8.13.
