strange problems reading chunked dataset

Hi,

I am getting a strange series of segfaults at this line of my code, using hdf5-1.6.7 on linux:

status = H5Dread(dataset, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT, (int*)data);

It reproducibly happens if only if the dataset is chunked (compressed or not). The moment exactly the same dataset is not chunked (and not compressed) a segfault does not occur.

The questions: should I somehow handle the two cases separately or is my 2-in-1 H5Dread call all right and the problem must be somewhere else in the code?

regards,
Dominik

···

--
Dominik Szczerba, Ph.D.
Biomedical Simulation Group
Computer Vision Lab CH-8092 Zurich
http://www.vision.ee.ethz.ch/~domi

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.

Oh, excuse me, the version is 1.6.5. I am not able to try 1.6.7 because it is built into a bigger application I am writing a part of a code for. Could this be an issue for 1.6.5?
regards,
Dominik

Dominik Szczerba wrote:

···

Hi,

I am getting a strange series of segfaults at this line of my code, using hdf5-1.6.7 on linux:

status = H5Dread(dataset, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT, (int*)data);

It reproducibly happens if only if the dataset is chunked (compressed or not). The moment exactly the same dataset is not chunked (and not compressed) a segfault does not occur.

The questions: should I somehow handle the two cases separately or is my 2-in-1 H5Dread call all right and the problem must be somewhere else in the code?

regards,
Dominik

--
Dominik Szczerba, Ph.D.
Biomedical Simulation Group
Computer Vision Lab CH-8092 Zurich
http://www.vision.ee.ethz.ch/~domi

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.

OK, the update: I managed to force 1.6.7 into the failing code and the problem seems gone... But I am still interested to hear if this could have been the problem, or it is another temporary luck and it will crash again some day...

regards,
Dominik

Dominik Szczerba wrote:

···

Oh, excuse me, the version is 1.6.5. I am not able to try 1.6.7 because it is built into a bigger application I am writing a part of a code for. Could this be an issue for 1.6.5?
regards,
Dominik

Dominik Szczerba wrote:

Hi,

I am getting a strange series of segfaults at this line of my code, using hdf5-1.6.7 on linux:

status = H5Dread(dataset, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT, (int*)data);

It reproducibly happens if only if the dataset is chunked (compressed or not). The moment exactly the same dataset is not chunked (and not compressed) a segfault does not occur.

The questions: should I somehow handle the two cases separately or is my 2-in-1 H5Dread call all right and the problem must be somewhere else in the code?

regards,
Dominik

--
Dominik Szczerba, Ph.D.
Biomedical Simulation Group
Computer Vision Lab CH-8092 Zurich
http://www.vision.ee.ethz.ch/~domi

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.

Hi Dominik,

How big are your chunks? Could it be your process is running out of memory?

With kind regards,

Pieter van der Meer

···

--
Pieter van der Meer
Software Engineer / Scientist
E-mail: P.van.der.Meer@sron.nl
Tel. +31 (0) 30 - 2535725

SRON Netherlands Institute for Space Research
http://www.sron.nl

Dominik Szczerba <domi@vision.ee.ethz.ch> 07/31/08 1:24 AM >>>

Oh, excuse me, the version is 1.6.5. I am not able to try 1.6.7 because
it is built into a bigger application I am writing a part of a code for.
Could this be an issue for 1.6.5?
regards,
Dominik

Dominik Szczerba wrote:

Hi,

I am getting a strange series of segfaults at this line of my code,
using hdf5-1.6.7 on linux:

status = H5Dread(dataset, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT,
(int*)data);

It reproducibly happens if only if the dataset is chunked (compressed or
not). The moment exactly the same dataset is not chunked (and not
compressed) a segfault does not occur.

The questions: should I somehow handle the two cases separately or is my
2-in-1 H5Dread call all right and the problem must be somewhere else in
the code?

regards,
Dominik

--
Dominik Szczerba, Ph.D.
Biomedical Simulation Group
Computer Vision Lab CH-8092 Zurich
http://www.vision.ee.ethz.ch/~domi

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.

Hi Pieter,

I wanted to mention it but forgot: no, there are small datasets. The chunk size was set to the dims of the whole dataset and was in order 3x400000, so the memory issue should not the case. BTW. That is how MATLAB wrote them with hdf5write.

regards,
Dominik

Pieter van der Meer wrote:

···

Hi Dominik,

How big are your chunks? Could it be your process is running out of memory?

With kind regards,

Pieter van der Meer

--
Dominik Szczerba, Ph.D.
Biomedical Simulation Group
Computer Vision Lab CH-8092 Zurich
http://www.vision.ee.ethz.ch/~domi

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.