Is this a bug in hdf5 1.8.6 ?

Hi all,

i have a problem with hdf5 version 1.8.6. (32 bits) and Visual c++ 2008.
i have several files which contain a 4x545869 double - matrix datasset. I can open/read/write these files on linux and mac os, but windows refuses to work with these files.

I get the following call stack:

HDF5-DIAG: Error detected in HDF5 (1.8.6) thread 0:
  #000: ..\..\..\src\H5Dio.c line 174 in H5Dread(): can't read data
    major: Dataset
    minor: Read failed
  #001: ..\..\..\src\H5Dio.c line 448 in H5D_read(): can't read data
    major: Dataset
    minor: Read failed
  #002: ..\..\..\src\H5Dcontig.c line 520 in H5D_contig_read(): contiguous read
failed
    major: Dataset
    minor: Read failed
  #003: ..\..\..\src\H5Dselect.c line 275 in H5D_select_read(): read error
    major: Dataspace
    minor: Read failed
  #004: ..\..\..\src\H5Dselect.c line 210 in H5D_select_io(): read error
    major: Dataspace
    minor: Read failed
  #005: ..\..\..\src\H5Dcontig.c line 692 in H5D_contig_readvv(): block read fai
led
    major: Low-level I/O
    minor: Read failed
  #006: ..\..\..\src\H5Fio.c line 113 in H5F_block_read(): read through metadata
accumulator failed
    major: Low-level I/O
    minor: Read failed
  #007: ..\..\..\src\H5Faccum.c line 254 in H5F_accum_read(): driver read reques
t failed
    major: Low-level I/O
    minor: Read failed
  #008: ..\..\..\src\H5FDint.c line 142 in H5FD_read(): driver read request fail
ed
    major: Virtual File Layer
    minor: Read failed
  #009: ..\..\..\src\H5FDwindows.c line 827 in H5FD_windows_read(): file read fa
iled, errno = 22, error message = 'Invalid argument'
    major: Low-level I/O
    minor: Read failed

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

Is there a way to detect whats wrong with the files? like an integrity checking tool?

regards

stefan

Hello again, i have played a bit more:

Also building files on Mac/Linux using Matlab

a=zeros(545869,4);
hdf5write('testfile','/testmatrix',a);

generates files wich are not readable on windows but work perfectly on Mac/Linux. Also i tried the new hdf5 1.8.7 which has a slightly more verbose output:

HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
  #000: ..\..\..\src\H5Dio.c line 174 in H5Dread(): can't read data
    major: Dataset
    minor: Read failed
  #001: ..\..\..\src\H5Dio.c line 448 in H5D_read(): can't read data
    major: Dataset
    minor: Read failed
  #002: ..\..\..\src\H5Dchunk.c line 1758 in H5D_chunk_read(): chunked read fail
ed
    major: Dataset
    minor: Read failed
  #003: ..\..\..\src\H5Dselect.c line 275 in H5D_select_read(): read error
    major: Dataspace
    minor: Read failed
  #004: ..\..\..\src\H5Dselect.c line 210 in H5D_select_io(): read error
    major: Dataspace
    minor: Read failed
  #005: ..\..\..\src\H5Dcontig.c line 890 in H5D_contig_readvv(): can't perform
vectorized sieve buffer read
    major: Dataset
    minor: Can't operate on object
  #006: ..\..\..\src\H5V.c line 1454 in H5V_opvv(): can't perform operation
    major: Internal error (too specific to document in detail)
    minor: Can't operate on object
  #007: ..\..\..\src\H5Dcontig.c line 691 in H5D_contig_readvv_sieve_cb(): block
read failed
    major: Dataset
    minor: Read failed
  #008: ..\..\..\src\H5Fio.c line 113 in H5F_block_read(): read through metadata
accumulator failed
    major: Low-level I/O
    minor: Read failed
  #009: ..\..\..\src\H5Faccum.c line 254 in H5F_accum_read(): driver read reques
t failed
    major: Low-level I/O
    minor: Read failed
  #010: ..\..\..\src\H5FDint.c line 142 in H5FD_read(): driver read request fail
ed
    major: Virtual File Layer
    minor: Read failed
  #011: ..\..\..\src\H5FDwindows.c line 827 in H5FD_windows_read(): file read fa
iled, errno = 22, error message = 'Invalid argument'
    major: Low-level I/O
    minor: Read failed

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

This looks like a HDF5 Bug to me.

regards
Stefan

···

Hi all,

i have a problem with hdf5 version 1.8.6. (32 bits) and Visual c++ 2008.
i have several files which contain a 4x545869 double - matrix datasset. I can open/read/write these files on linux and mac os, but windows refuses to work with these files.

I get the following call stack:

HDF5-DIAG: Error detected in HDF5 (1.8.6) thread 0:
#000: ..\..\..\src\H5Dio.c line 174 in H5Dread(): can't read data
   major: Dataset
   minor: Read failed
#001: ..\..\..\src\H5Dio.c line 448 in H5D_read(): can't read data
   major: Dataset
   minor: Read failed
#002: ..\..\..\src\H5Dcontig.c line 520 in H5D_contig_read(): contiguous read
failed
   major: Dataset
   minor: Read failed
#003: ..\..\..\src\H5Dselect.c line 275 in H5D_select_read(): read error
   major: Dataspace
   minor: Read failed
#004: ..\..\..\src\H5Dselect.c line 210 in H5D_select_io(): read error
   major: Dataspace
   minor: Read failed
#005: ..\..\..\src\H5Dcontig.c line 692 in H5D_contig_readvv(): block read fai
led
   major: Low-level I/O
   minor: Read failed
#006: ..\..\..\src\H5Fio.c line 113 in H5F_block_read(): read through metadata
accumulator failed
   major: Low-level I/O
   minor: Read failed
#007: ..\..\..\src\H5Faccum.c line 254 in H5F_accum_read(): driver read reques
t failed
   major: Low-level I/O
   minor: Read failed
#008: ..\..\..\src\H5FDint.c line 142 in H5FD_read(): driver read request fail
ed
   major: Virtual File Layer
   minor: Read failed
#009: ..\..\..\src\H5FDwindows.c line 827 in H5FD_windows_read(): file read fa
iled, errno = 22, error message = 'Invalid argument'
   major: Low-level I/O
   minor: Read failed

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

Is there a way to detect whats wrong with the files? like an integrity checking tool?

regards

stefan

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Stefan,

Could you please send to help@hdfgroup.org your file and the code?

It does look like a bug.

Thank you!

Elena

···

On Jun 20, 2011, at 4:53 PM, Stefan Schiffler wrote:

Hello again, i have played a bit more:

Also building files on Mac/Linux using Matlab

a=zeros(545869,4);
hdf5write('testfile','/testmatrix',a);

generates files wich are not readable on windows but work perfectly on Mac/Linux. Also i tried the new hdf5 1.8.7 which has a slightly more verbose output:

HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
#000: ..\..\..\src\H5Dio.c line 174 in H5Dread(): can't read data
   major: Dataset
   minor: Read failed
#001: ..\..\..\src\H5Dio.c line 448 in H5D_read(): can't read data
   major: Dataset
   minor: Read failed
#002: ..\..\..\src\H5Dchunk.c line 1758 in H5D_chunk_read(): chunked read fail
ed
   major: Dataset
   minor: Read failed
#003: ..\..\..\src\H5Dselect.c line 275 in H5D_select_read(): read error
   major: Dataspace
   minor: Read failed
#004: ..\..\..\src\H5Dselect.c line 210 in H5D_select_io(): read error
   major: Dataspace
   minor: Read failed
#005: ..\..\..\src\H5Dcontig.c line 890 in H5D_contig_readvv(): can't perform
vectorized sieve buffer read
   major: Dataset
   minor: Can't operate on object
#006: ..\..\..\src\H5V.c line 1454 in H5V_opvv(): can't perform operation
   major: Internal error (too specific to document in detail)
   minor: Can't operate on object
#007: ..\..\..\src\H5Dcontig.c line 691 in H5D_contig_readvv_sieve_cb(): block
read failed
   major: Dataset
   minor: Read failed
#008: ..\..\..\src\H5Fio.c line 113 in H5F_block_read(): read through metadata
accumulator failed
   major: Low-level I/O
   minor: Read failed
#009: ..\..\..\src\H5Faccum.c line 254 in H5F_accum_read(): driver read reques
t failed
   major: Low-level I/O
   minor: Read failed
#010: ..\..\..\src\H5FDint.c line 142 in H5FD_read(): driver read request fail
ed
   major: Virtual File Layer
   minor: Read failed
#011: ..\..\..\src\H5FDwindows.c line 827 in H5FD_windows_read(): file read fa
iled, errno = 22, error message = 'Invalid argument'
   major: Low-level I/O
   minor: Read failed

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

This looks like a HDF5 Bug to me.

regards
Stefan

Hi all,

i have a problem with hdf5 version 1.8.6. (32 bits) and Visual c++ 2008.
i have several files which contain a 4x545869 double - matrix datasset. I can open/read/write these files on linux and mac os, but windows refuses to work with these files.

I get the following call stack:

HDF5-DIAG: Error detected in HDF5 (1.8.6) thread 0:
#000: ..\..\..\src\H5Dio.c line 174 in H5Dread(): can't read data
  major: Dataset
  minor: Read failed
#001: ..\..\..\src\H5Dio.c line 448 in H5D_read(): can't read data
  major: Dataset
  minor: Read failed
#002: ..\..\..\src\H5Dcontig.c line 520 in H5D_contig_read(): contiguous read
failed
  major: Dataset
  minor: Read failed
#003: ..\..\..\src\H5Dselect.c line 275 in H5D_select_read(): read error
  major: Dataspace
  minor: Read failed
#004: ..\..\..\src\H5Dselect.c line 210 in H5D_select_io(): read error
  major: Dataspace
  minor: Read failed
#005: ..\..\..\src\H5Dcontig.c line 692 in H5D_contig_readvv(): block read fai
led
  major: Low-level I/O
  minor: Read failed
#006: ..\..\..\src\H5Fio.c line 113 in H5F_block_read(): read through metadata
accumulator failed
  major: Low-level I/O
  minor: Read failed
#007: ..\..\..\src\H5Faccum.c line 254 in H5F_accum_read(): driver read reques
t failed
  major: Low-level I/O
  minor: Read failed
#008: ..\..\..\src\H5FDint.c line 142 in H5FD_read(): driver read request fail
ed
  major: Virtual File Layer
  minor: Read failed
#009: ..\..\..\src\H5FDwindows.c line 827 in H5FD_windows_read(): file read fa
iled, errno = 22, error message = 'Invalid argument'
  major: Low-level I/O
  minor: Read failed

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

Is there a way to detect whats wrong with the files? like an integrity checking tool?

regards

stefan

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Stefan,

Actually... Could you please run h5dump on your files on windows and see if you still get an error?

If yes, then it is HDF5 problem and we will need to take a look.

Thank you!

Elena

···

On Jun 20, 2011, at 4:53 PM, Stefan Schiffler wrote:

Hello again, i have played a bit more:

Also building files on Mac/Linux using Matlab

a=zeros(545869,4);
hdf5write('testfile','/testmatrix',a);

generates files wich are not readable on windows but work perfectly on Mac/Linux. Also i tried the new hdf5 1.8.7 which has a slightly more verbose output:

HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
#000: ..\..\..\src\H5Dio.c line 174 in H5Dread(): can't read data
   major: Dataset
   minor: Read failed
#001: ..\..\..\src\H5Dio.c line 448 in H5D_read(): can't read data
   major: Dataset
   minor: Read failed
#002: ..\..\..\src\H5Dchunk.c line 1758 in H5D_chunk_read(): chunked read fail
ed
   major: Dataset
   minor: Read failed
#003: ..\..\..\src\H5Dselect.c line 275 in H5D_select_read(): read error
   major: Dataspace
   minor: Read failed
#004: ..\..\..\src\H5Dselect.c line 210 in H5D_select_io(): read error
   major: Dataspace
   minor: Read failed
#005: ..\..\..\src\H5Dcontig.c line 890 in H5D_contig_readvv(): can't perform
vectorized sieve buffer read
   major: Dataset
   minor: Can't operate on object
#006: ..\..\..\src\H5V.c line 1454 in H5V_opvv(): can't perform operation
   major: Internal error (too specific to document in detail)
   minor: Can't operate on object
#007: ..\..\..\src\H5Dcontig.c line 691 in H5D_contig_readvv_sieve_cb(): block
read failed
   major: Dataset
   minor: Read failed
#008: ..\..\..\src\H5Fio.c line 113 in H5F_block_read(): read through metadata
accumulator failed
   major: Low-level I/O
   minor: Read failed
#009: ..\..\..\src\H5Faccum.c line 254 in H5F_accum_read(): driver read reques
t failed
   major: Low-level I/O
   minor: Read failed
#010: ..\..\..\src\H5FDint.c line 142 in H5FD_read(): driver read request fail
ed
   major: Virtual File Layer
   minor: Read failed
#011: ..\..\..\src\H5FDwindows.c line 827 in H5FD_windows_read(): file read fa
iled, errno = 22, error message = 'Invalid argument'
   major: Low-level I/O
   minor: Read failed

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

This looks like a HDF5 Bug to me.

regards
Stefan

Hi all,

i have a problem with hdf5 version 1.8.6. (32 bits) and Visual c++ 2008.
i have several files which contain a 4x545869 double - matrix datasset. I can open/read/write these files on linux and mac os, but windows refuses to work with these files.

I get the following call stack:

HDF5-DIAG: Error detected in HDF5 (1.8.6) thread 0:
#000: ..\..\..\src\H5Dio.c line 174 in H5Dread(): can't read data
  major: Dataset
  minor: Read failed
#001: ..\..\..\src\H5Dio.c line 448 in H5D_read(): can't read data
  major: Dataset
  minor: Read failed
#002: ..\..\..\src\H5Dcontig.c line 520 in H5D_contig_read(): contiguous read
failed
  major: Dataset
  minor: Read failed
#003: ..\..\..\src\H5Dselect.c line 275 in H5D_select_read(): read error
  major: Dataspace
  minor: Read failed
#004: ..\..\..\src\H5Dselect.c line 210 in H5D_select_io(): read error
  major: Dataspace
  minor: Read failed
#005: ..\..\..\src\H5Dcontig.c line 692 in H5D_contig_readvv(): block read fai
led
  major: Low-level I/O
  minor: Read failed
#006: ..\..\..\src\H5Fio.c line 113 in H5F_block_read(): read through metadata
accumulator failed
  major: Low-level I/O
  minor: Read failed
#007: ..\..\..\src\H5Faccum.c line 254 in H5F_accum_read(): driver read reques
t failed
  major: Low-level I/O
  minor: Read failed
#008: ..\..\..\src\H5FDint.c line 142 in H5FD_read(): driver read request fail
ed
  major: Virtual File Layer
  minor: Read failed
#009: ..\..\..\src\H5FDwindows.c line 827 in H5FD_windows_read(): file read fa
iled, errno = 22, error message = 'Invalid argument'
  major: Low-level I/O
  minor: Read failed

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

Is there a way to detect whats wrong with the files? like an integrity checking tool?

regards

stefan

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Thanks for the hint,
h5dump is still giving output... i will see if it finishes. Anyway it looks like h5dump is not reading the full dataset at once! I think that is why it might finish. My code only blows up if i am not using compression, with compression everything is fine. Hence it seems that only reading the huge amount of numbers fails. Also HDF-View fails on Windows when reading the dataset....

regards Stefan

Dr. Stefan Schiffler

Zentrum für Technomathematik
Universität Bremen
Bibliothekstr. 1
D-28359 Bremen

Tel. +49 (0)421 218-63 822

www.scils.de

···

Am 21.06.2011 um 15:50 schrieb Elena Pourmal:

Stefan,

Actually... Could you please run h5dump on your files on windows and see if you still get an error?

If yes, then it is HDF5 problem and we will need to take a look.

Thank you!

Elena
On Jun 20, 2011, at 4:53 PM, Stefan Schiffler wrote:

Hello again, i have played a bit more:

Also building files on Mac/Linux using Matlab

a=zeros(545869,4);
hdf5write('testfile','/testmatrix',a);

generates files wich are not readable on windows but work perfectly on Mac/Linux. Also i tried the new hdf5 1.8.7 which has a slightly more verbose output:

HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
#000: ..\..\..\src\H5Dio.c line 174 in H5Dread(): can't read data
  major: Dataset
  minor: Read failed
#001: ..\..\..\src\H5Dio.c line 448 in H5D_read(): can't read data
  major: Dataset
  minor: Read failed
#002: ..\..\..\src\H5Dchunk.c line 1758 in H5D_chunk_read(): chunked read fail
ed
  major: Dataset
  minor: Read failed
#003: ..\..\..\src\H5Dselect.c line 275 in H5D_select_read(): read error
  major: Dataspace
  minor: Read failed
#004: ..\..\..\src\H5Dselect.c line 210 in H5D_select_io(): read error
  major: Dataspace
  minor: Read failed
#005: ..\..\..\src\H5Dcontig.c line 890 in H5D_contig_readvv(): can't perform
vectorized sieve buffer read
  major: Dataset
  minor: Can't operate on object
#006: ..\..\..\src\H5V.c line 1454 in H5V_opvv(): can't perform operation
  major: Internal error (too specific to document in detail)
  minor: Can't operate on object
#007: ..\..\..\src\H5Dcontig.c line 691 in H5D_contig_readvv_sieve_cb(): block
read failed
  major: Dataset
  minor: Read failed
#008: ..\..\..\src\H5Fio.c line 113 in H5F_block_read(): read through metadata
accumulator failed
  major: Low-level I/O
  minor: Read failed
#009: ..\..\..\src\H5Faccum.c line 254 in H5F_accum_read(): driver read reques
t failed
  major: Low-level I/O
  minor: Read failed
#010: ..\..\..\src\H5FDint.c line 142 in H5FD_read(): driver read request fail
ed
  major: Virtual File Layer
  minor: Read failed
#011: ..\..\..\src\H5FDwindows.c line 827 in H5FD_windows_read(): file read fa
iled, errno = 22, error message = 'Invalid argument'
  major: Low-level I/O
  minor: Read failed

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

This looks like a HDF5 Bug to me.

regards
Stefan

Hi all,

i have a problem with hdf5 version 1.8.6. (32 bits) and Visual c++ 2008.
i have several files which contain a 4x545869 double - matrix datasset. I can open/read/write these files on linux and mac os, but windows refuses to work with these files.

I get the following call stack:

HDF5-DIAG: Error detected in HDF5 (1.8.6) thread 0:
#000: ..\..\..\src\H5Dio.c line 174 in H5Dread(): can't read data
major: Dataset
minor: Read failed
#001: ..\..\..\src\H5Dio.c line 448 in H5D_read(): can't read data
major: Dataset
minor: Read failed
#002: ..\..\..\src\H5Dcontig.c line 520 in H5D_contig_read(): contiguous read
failed
major: Dataset
minor: Read failed
#003: ..\..\..\src\H5Dselect.c line 275 in H5D_select_read(): read error
major: Dataspace
minor: Read failed
#004: ..\..\..\src\H5Dselect.c line 210 in H5D_select_io(): read error
major: Dataspace
minor: Read failed
#005: ..\..\..\src\H5Dcontig.c line 692 in H5D_contig_readvv(): block read fai
led
major: Low-level I/O
minor: Read failed
#006: ..\..\..\src\H5Fio.c line 113 in H5F_block_read(): read through metadata
accumulator failed
major: Low-level I/O
minor: Read failed
#007: ..\..\..\src\H5Faccum.c line 254 in H5F_accum_read(): driver read reques
t failed
major: Low-level I/O
minor: Read failed
#008: ..\..\..\src\H5FDint.c line 142 in H5FD_read(): driver read request fail
ed
major: Virtual File Layer
minor: Read failed
#009: ..\..\..\src\H5FDwindows.c line 827 in H5FD_windows_read(): file read fa
iled, errno = 22, error message = 'Invalid argument'
major: Low-level I/O
minor: Read failed

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

Is there a way to detect whats wrong with the files? like an integrity checking tool?

regards

stefan

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org