Hi,
I am having some trouble reading in a HDF5 file that is ~5GB. I was
initially having some problems even writing out a file this large and
looked at some postings in the HDF5 forum related to this.
I tried using the serial version of the HDF5 library, v1.8.9 and was
able to write out a file of size ~5GB. However, when I try to read it
back in, I get the following error.
···
----------------
#000: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dio.c line
182 in H5Dread(): can't read data
major: Dataset
minor: Read failed
#001: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dio.c line
550 in H5D__read(): can't read data
major: Dataset
minor: Read failed
#002: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c
line 543 in H5D__contig_read(): contiguous read failed
major: Dataset
minor: Read failed
#003: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dselect.c
line 278 in H5D__select_read(): read error
major: Dataspace
minor: Read failed
#004: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dselect.c
line 213 in H5D__select_io(): read error
major: Dataspace
minor: Read failed
#005: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c
line 873 in H5D__contig_readvv(): can't perform vectorized sieve buffer read
major: Dataset
minor: Can't operate on object
#006: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5V.c line
1457 in H5V_opvv(): can't perform operation
major: Internal error (too specific to document in detail)
minor: Can't operate on object
#007: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c
line 674 in H5D__contig_readvv_sieve_cb(): block read failed
major: Dataset
minor: Read failed
#008: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Fio.c line
113 in H5F_block_read(): read through metadata accumulator failed
major: Low-level I/O
minor: Read failed
#009: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Faccum.c
line 258 in H5F_accum_read(): driver read request failed
major: Low-level I/O
minor: Read failed
#010: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5FDint.c
line 142 in H5FD_read(): driver read request failed
major: Virtual File Layer
minor: Read failed
#011: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5FDsec2.c
line 725 in H5FD_sec2_read(): file read failed: time = Mon Jul 1
20:21:57 2013
, filename = '/tmp/file.hdf5', file descriptor = 5, errno = 14, error
message = 'Bad address', buf = 0x2ae9619ea010, total read size =
4677466176, bytes this sub-read = 4677466176, bytes actually read =
18446744073709551615, offset = 744468544
-----------------
I looked at the forum again and saw this posting, which mentioned that
there was a bug fix done for POSIX issues.
http://mail.lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2012-December/006348.html
I was using v1.8.9, but thought of using the latest 1.8.11 to rule out
any issues with the HDF5 version I was using. However, I still get the
above error.
Please could someone provide some feedback on why this is happening. If
I am able to write out such a large file, I should be able to read it in
as well? Ofcourse, small sized files work perfectly fine in my application.
Since the library I am using, that in turn calls the HDF5 routines,
needs the interface of v1.6 of the HDF5 library, I use the flag
H5_USE_16_API while compiling. Would this in anyway cause the problem.
Thanks in advance.
--
Regards
Nitya