HDF5 - reading files larger than 2GB (Albert Cheng)

Hi Albert, Dana,

Thanks for your replies. I give the output of uname below

Linux 3.0.74-0.6.10-default #1 SMP x86_64 x86_64 x86_64 GNU/Linux

I was earlier using v1.8.9 and then tried using v1.8.11. I have given
the libhdf5.settings file for both versions below. I downloaded the
library files of v1.8.11 from the HDF5 website, so haven't compiled
anything. v1.8.9 has been compiled by the system administrators.

···

-------------------
SUMMARY OF THE HDF5 CONFIGURATION
            =================================

General Information:
-------------------
                   HDF5 Version: 1.8.9
                  Configured on: Thu Nov 29 11:11:42 CET 2012
                  Configured by: wwn@qa01
                 Configure mode: production
                    Host system: x86_64-unknown-linux-gnu
              Uname information: Linux qa01 3.0.42-0.7-default #1 SMP
Tue Oct 9 11:58:45 UTC 2012 (a8dc443) x86_64 x86_64 x86_64 GNU/Linux
                       Byte sex: little-endian
                      Libraries:
             Installation point:
/afs/rzg/common/soft/hdf5/1.8.9/amd64_sles11/intel/12.1/serial

Compiling Options:
------------------
               Compilation Mode: production
                     C Compiler:
/afs/@cell/common/soft/intel/ics/2011.0.013/12.1/bin/intel64/bin/icc (
Intel(R) C Intel(R) 64 Compiler Version 12.1 Build 20120928)
                         CFLAGS: -fPIC
                      H5_CFLAGS: -std=c99 -O3
                      AM_CFLAGS:
                       CPPFLAGS:
                    H5_CPPFLAGS: -D_POSIX_C_SOURCE=199506L -DNDEBUG
-UH5_DEBUG_API
                    AM_CPPFLAGS: -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -D_BSD_SOURCE
               Shared C Library: yes
               Static C Library: yes
  Statically Linked Executables: no
                        LDFLAGS:
                     H5_LDFLAGS:
                     AM_LDFLAGS:
                Extra libraries: -lz -lrt -lm
                       Archiver: ar
                         Ranlib: ranlib
              Debugged Packages:
                    API Tracing: no

Languages:
----------
                        Fortran: yes
               Fortran Compiler:
/afs/@cell/common/soft/intel/ics/2011.0.013/12.1/bin/intel64/bin/ifort (
Intel(R) Fortran Intel(R) 64 Compiler Version 12.1 Build 20120928)
          Fortran 2003 Compiler: yes
                  Fortran Flags: -fPIC
               H5 Fortran Flags: -O3
               AM Fortran Flags:
         Shared Fortran Library: yes
         Static Fortran Library: yes
                            C++: yes
                   C++ Compiler:
/afs/@cell/common/soft/intel/ics/2011.0.013/12.1/bin/intel64/bin/icpc (
Intel(R) C++ Intel(R) 64 Compiler Version 12.1 Build 20120928)
                      C++ Flags:
                   H5 C++ Flags:
                   AM C++ Flags:
             Shared C++ Library: yes
             Static C++ Library: yes

Features:
---------
                  Parallel HDF5: no
             High Level library: yes
                   Threadsafety: no
            Default API Mapping: v18
With Deprecated Public Symbols: yes
         I/O filters (external): deflate(zlib)
         I/O filters (internal): shuffle,fletcher32,nbit,scaleoffset
                            MPE: no
                     Direct VFD: no
                        dmalloc: no
Clear file buffers before write: yes
           Using memory checker: no
         Function Stack Tracing: no
                           GPFS: no
      Strict File Format Checks: no
   Optimization Instrumentation: no
       Large File Support (LFS): yes
-----------------------------------------------

v1.8.11
***************************************
            SUMMARY OF THE HDF5 CONFIGURATION
            =================================

General Information:
-------------------
                   HDF5 Version: 1.8.11
                  Configured on: Wed May 8 16:20:56 CDT 2013
                  Configured by: hdftest@koala
                 Configure mode: production
                    Host system: x86_64-unknown-linux-gnu
              Uname information: Linux koala 2.6.18-348.1.1.el5 #1 SMP
Tue Jan 22 16:19:19 EST 2013 x86_64 x86_64 x86_64 GNU/Linux
                       Byte sex: little-endian
                      Libraries: static, shared
             Installation point:
/mnt/scr1/pre-release/hdf5/v1811/thg-builds/koala

Compiling Options:
------------------
               Compilation Mode: production
                     C Compiler: /usr/bin/gcc ( gcc (GCC) 4.1.2 20080704 )
                         CFLAGS:
                      H5_CFLAGS: -std=c99 -pedantic -Wall -Wextra
-Wundef -Wshadow -Wpointer-arith -Wbad-function-cast -Wcast-qual
-Wcast-align -Wwrite-strings -Wconversion -Wag
gregate-return -Wstrict-prototypes -Wmissing-prototypes
-Wmissing-declarations -Wredundant-decls -Wnested-externs -Winline
-Wno-long-long -Wfloat-equal -Wmissing-format-attribut
e -Wmissing-noreturn -Wpacked -Wdisabled-optimization -Wformat=2
-Wunreachable-code -Wendif-labels -Wdeclaration-after-statement
-Wold-style-definition -Winvalid-pch -Wvariadic-
macros -Wnonnull -Winit-self -Wmissing-include-dirs -Wswitch-default
-Wswitch-enum -Wunused-macros -Wunsafe-loop-optimizations -Wc++-compat
-Wvolatile-register-var -O3 -fomit-fr
ame-pointer -finline-functions
                      AM_CFLAGS:
                       CPPFLAGS:
                    H5_CPPFLAGS: -D_POSIX_C_SOURCE=199506L -DNDEBUG
-UH5_DEBUG_API
                    AM_CPPFLAGS:
-I/mnt/hdf/packages/szip/shared/encoder/Linux2.6-x86_64-gcc/include
-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_BSD_SOURCE
               Shared C Library: yes
               Static C Library: yes
  Statically Linked Executables: yes
                        LDFLAGS:
                     H5_LDFLAGS:
                     AM_LDFLAGS:
-L/mnt/hdf/packages/szip/shared/encoder/Linux2.6-x86_64-gcc/lib
                Extra libraries: -lsz -lz -lrt -ldl -lm
                       Archiver: ar
                         Ranlib: ranlib
              Debugged Packages:
                    API Tracing: no
Languages:
----------
                        Fortran: yes
               Fortran Compiler: /usr/bin/gfortran ( GNU Fortran (GCC)
4.1.2 20080704 )
          Fortran 2003 Compiler: no
                  Fortran Flags:
               H5 Fortran Flags:
               AM Fortran Flags:
         Shared Fortran Library: yes
         Static Fortran Library: yes

                            C++: yes
                   C++ Compiler: /usr/bin/g++ ( g++ (GCC) 4.1.2 20080704 )
                      C++ Flags:
                   H5 C++ Flags:
                   AM C++ Flags:
             Shared C++ Library: yes
             Static C++ Library: yes

Features:
---------
                  Parallel HDF5: no
             High Level library: yes
                   Threadsafety: no
            Default API Mapping: v18
With Deprecated Public Symbols: yes
         I/O filters (external): deflate(zlib),szip(encoder)
         I/O filters (internal): shuffle,fletcher32,nbit,scaleoffset
                            MPE: no
                     Direct VFD: no
                        dmalloc: no
Clear file buffers before write: yes
           Using memory checker: no
         Function Stack Tracing: no
                           GPFS: no
      Strict File Format Checks: no
   Optimization Instrumentation: no
       Large File Support (LFS): yes
****************************************

I had another query. What value does H5P_DEFAULT contain and where is
the value for this variable set. I was not able to find it from the
v1.8.9 source, please let me know. I ask, since the code I am working
with, sets the file access to H5P_DEFAULT if MPI is not being used, else
its set to H5Pset_fapl_mpio.

Please let me know if you need more information.

Thanks

Nitya

-----------------------------

Message: 4
Date: Tue, 2 Jul 2013 14:33:07 -0500
From: Albert Cheng <acheng@hdfgroup.org>
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
Subject: Re: [Hdf-forum] HDF5 - reading files larger than 2GB
Message-ID: <4C52ED59-20DB-4F48-A642-17CF4202D3BB@hdfgroup.org>
Content-Type: text/plain; charset="iso-8859-1"

Hi Nitya,

Can you also give more information about the machine in which you ran your application
that got the failures? E.g.,

1. OS version ("uname -a" usually shows it.)
1.1. For Mac, try "sw_vers".

2. Did you build the HDF5 library from source?
2.1. if yes, please send a copy of the output of configure.
2.2. If the output of configure is not available, please send a copy of libhdf5.settings.
        You can usually find "libhdf5.settings" in the same directory where you find the
        HDF5 library, libhdf5.a or libhdf5.so.

This will help us to try to repeat your problems in the appropriate platform.
Thanks.

-Albert Change
THG Staff

On Jul 2, 2013, at 11:20 AM, Dana Robinson <derobins@hdfgroup.org> wrote:

Hi Nitya,

I'll take a look into this. It's a holiday weekend here, so I probably won't get to this until next week.

Dana

On Mon, Jul 1, 2013 at 1:37 PM, Nitya Hariharan <nty@mpa-garching.mpg.de> wrote:
Hi,

I am having some trouble reading in a HDF5 file that is ~5GB. I was
initially having some problems even writing out a file this large and
looked at some postings in the HDF5 forum related to this.

http://hdf-forum.184993.n3.nabble.com/Trouble-writing-2GB-dataset-from-single-task-with-HDF5-1-8-10-td4025821.html

I tried using the serial version of the HDF5 library, v1.8.9 and was
able to write out a file of size ~5GB. However, when I try to read it
back in, I get the following error.

----------------
#000: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dio.c line
182 in H5Dread(): can't read data
    major: Dataset
    minor: Read failed
  #001: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dio.c line
550 in H5D__read(): can't read data
    major: Dataset
    minor: Read failed
  #002: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c
line 543 in H5D__contig_read(): contiguous read failed
    major: Dataset
    minor: Read failed
  #003: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dselect.c
line 278 in H5D__select_read(): read error
    major: Dataspace
    minor: Read failed
  #004: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dselect.c
line 213 in H5D__select_io(): read error
    major: Dataspace
    minor: Read failed
  #005: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c
line 873 in H5D__contig_readvv(): can't perform vectorized sieve buffer read
    major: Dataset
    minor: Can't operate on object
  #006: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5V.c line
1457 in H5V_opvv(): can't perform operation
    major: Internal error (too specific to document in detail)
    minor: Can't operate on object
  #007: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c
line 674 in H5D__contig_readvv_sieve_cb(): block read failed
    major: Dataset
    minor: Read failed
  #008: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Fio.c line
113 in H5F_block_read(): read through metadata accumulator failed
    major: Low-level I/O
    minor: Read failed
  #009: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Faccum.c
line 258 in H5F_accum_read(): driver read request failed
    major: Low-level I/O
    minor: Read failed
  #010: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5FDint.c
line 142 in H5FD_read(): driver read request failed
    major: Virtual File Layer
    minor: Read failed
  #011: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5FDsec2.c
line 725 in H5FD_sec2_read(): file read failed: time = Mon Jul 1
20:21:57 2013
, filename = '/tmp/file.hdf5', file descriptor = 5, errno = 14, error
message = 'Bad address', buf = 0x2ae9619ea010, total read size =
4677466176, bytes this sub-read = 4677466176, bytes actually read =
18446744073709551615, offset = 744468544
-----------------

I looked at the forum again and saw this posting, which mentioned that
there was a bug fix done for POSIX issues.

http://mail.lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2012-December/006348.html

I was using v1.8.9, but thought of using the latest 1.8.11 to rule out
any issues with the HDF5 version I was using. However, I still get the
above error.

Please could someone provide some feedback on why this is happening. If
I am able to write out such a large file, I should be able to read it in
as well? Ofcourse, small sized files work perfectly fine in my application.

Since the library I am using, that in turn calls the HDF5 routines,
needs the interface of v1.6 of the HDF5 library, I use the flag
H5_USE_16_API while compiling. Would this in anyway cause the problem.

Thanks in advance.

--
Regards

Nitya

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

Hi Nitya,

Looks like you are in a Linux 3.0 system.
Can you find out what distribution it is? E.g., Redhat, Fedora, Suse, ….
There is no one way to find this information but you often can find a file like
/etc/<XYZ>-release. If you can locate such a file, please send its content.

-Albert Cheng

···

On Jul 4, 2013, at 4:31 AM, Nitya <nty@MPA-Garching.MPG.DE> wrote:

Hi Albert, Dana,

Thanks for your replies. I give the output of uname below

Linux 3.0.74-0.6.10-default #1 SMP x86_64 x86_64 x86_64 GNU/Linux

I was earlier using v1.8.9 and then tried using v1.8.11. I have given
the libhdf5.settings file for both versions below. I downloaded the
library files of v1.8.11 from the HDF5 website, so haven't compiled
anything. v1.8.9 has been compiled by the system administrators.

-------------------
SUMMARY OF THE HDF5 CONFIGURATION
           =================================

General Information:
-------------------
                  HDF5 Version: 1.8.9
                 Configured on: Thu Nov 29 11:11:42 CET 2012
                 Configured by: wwn@qa01
                Configure mode: production
                   Host system: x86_64-unknown-linux-gnu
             Uname information: Linux qa01 3.0.42-0.7-default #1 SMP
Tue Oct 9 11:58:45 UTC 2012 (a8dc443) x86_64 x86_64 x86_64 GNU/Linux
                      Byte sex: little-endian
                     Libraries:
            Installation point:
/afs/rzg/common/soft/hdf5/1.8.9/amd64_sles11/intel/12.1/serial

Compiling Options:
------------------
              Compilation Mode: production
                    C Compiler:
/afs/@cell/common/soft/intel/ics/2011.0.013/12.1/bin/intel64/bin/icc (
Intel(R) C Intel(R) 64 Compiler Version 12.1 Build 20120928)
                        CFLAGS: -fPIC
                     H5_CFLAGS: -std=c99 -O3
                     AM_CFLAGS:
                      CPPFLAGS:
                   H5_CPPFLAGS: -D_POSIX_C_SOURCE=199506L -DNDEBUG
-UH5_DEBUG_API
                   AM_CPPFLAGS: -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -D_BSD_SOURCE
              Shared C Library: yes
              Static C Library: yes
Statically Linked Executables: no
                       LDFLAGS:
                    H5_LDFLAGS:
                    AM_LDFLAGS:
               Extra libraries: -lz -lrt -lm
                      Archiver: ar
                        Ranlib: ranlib
             Debugged Packages:
                   API Tracing: no

Languages:
----------
                       Fortran: yes
              Fortran Compiler:
/afs/@cell/common/soft/intel/ics/2011.0.013/12.1/bin/intel64/bin/ifort (
Intel(R) Fortran Intel(R) 64 Compiler Version 12.1 Build 20120928)
         Fortran 2003 Compiler: yes
                 Fortran Flags: -fPIC
              H5 Fortran Flags: -O3
              AM Fortran Flags:
        Shared Fortran Library: yes
        Static Fortran Library: yes
                           C++: yes
                  C++ Compiler:
/afs/@cell/common/soft/intel/ics/2011.0.013/12.1/bin/intel64/bin/icpc (
Intel(R) C++ Intel(R) 64 Compiler Version 12.1 Build 20120928)
                     C++ Flags:
                  H5 C++ Flags:
                  AM C++ Flags:
            Shared C++ Library: yes
            Static C++ Library: yes

Features:
---------
                 Parallel HDF5: no
            High Level library: yes
                  Threadsafety: no
           Default API Mapping: v18
With Deprecated Public Symbols: yes
        I/O filters (external): deflate(zlib)
        I/O filters (internal): shuffle,fletcher32,nbit,scaleoffset
                           MPE: no
                    Direct VFD: no
                       dmalloc: no
Clear file buffers before write: yes
          Using memory checker: no
        Function Stack Tracing: no
                          GPFS: no
     Strict File Format Checks: no
  Optimization Instrumentation: no
      Large File Support (LFS): yes
-----------------------------------------------

v1.8.11
***************************************
           SUMMARY OF THE HDF5 CONFIGURATION
           =================================

General Information:
-------------------
                  HDF5 Version: 1.8.11
                 Configured on: Wed May 8 16:20:56 CDT 2013
                 Configured by: hdftest@koala
                Configure mode: production
                   Host system: x86_64-unknown-linux-gnu
             Uname information: Linux koala 2.6.18-348.1.1.el5 #1 SMP
Tue Jan 22 16:19:19 EST 2013 x86_64 x86_64 x86_64 GNU/Linux
                      Byte sex: little-endian
                     Libraries: static, shared
            Installation point:
/mnt/scr1/pre-release/hdf5/v1811/thg-builds/koala

Compiling Options:
------------------
              Compilation Mode: production
                    C Compiler: /usr/bin/gcc ( gcc (GCC) 4.1.2 20080704 )
                        CFLAGS:
                     H5_CFLAGS: -std=c99 -pedantic -Wall -Wextra
-Wundef -Wshadow -Wpointer-arith -Wbad-function-cast -Wcast-qual
-Wcast-align -Wwrite-strings -Wconversion -Wag
gregate-return -Wstrict-prototypes -Wmissing-prototypes
-Wmissing-declarations -Wredundant-decls -Wnested-externs -Winline
-Wno-long-long -Wfloat-equal -Wmissing-format-attribut
e -Wmissing-noreturn -Wpacked -Wdisabled-optimization -Wformat=2
-Wunreachable-code -Wendif-labels -Wdeclaration-after-statement
-Wold-style-definition -Winvalid-pch -Wvariadic-
macros -Wnonnull -Winit-self -Wmissing-include-dirs -Wswitch-default
-Wswitch-enum -Wunused-macros -Wunsafe-loop-optimizations -Wc++-compat
-Wvolatile-register-var -O3 -fomit-fr
ame-pointer -finline-functions
                     AM_CFLAGS:
                      CPPFLAGS:
                   H5_CPPFLAGS: -D_POSIX_C_SOURCE=199506L -DNDEBUG
-UH5_DEBUG_API
                   AM_CPPFLAGS:
-I/mnt/hdf/packages/szip/shared/encoder/Linux2.6-x86_64-gcc/include
-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_BSD_SOURCE
              Shared C Library: yes
              Static C Library: yes
Statically Linked Executables: yes
                       LDFLAGS:
                    H5_LDFLAGS:
                    AM_LDFLAGS:
-L/mnt/hdf/packages/szip/shared/encoder/Linux2.6-x86_64-gcc/lib
               Extra libraries: -lsz -lz -lrt -ldl -lm
                      Archiver: ar
                        Ranlib: ranlib
             Debugged Packages:
                   API Tracing: no
Languages:
----------
                       Fortran: yes
              Fortran Compiler: /usr/bin/gfortran ( GNU Fortran (GCC)
4.1.2 20080704 )
         Fortran 2003 Compiler: no
                 Fortran Flags:
              H5 Fortran Flags:
              AM Fortran Flags:
        Shared Fortran Library: yes
        Static Fortran Library: yes

                           C++: yes
                  C++ Compiler: /usr/bin/g++ ( g++ (GCC) 4.1.2 20080704 )
                     C++ Flags:
                  H5 C++ Flags:
                  AM C++ Flags:
            Shared C++ Library: yes
            Static C++ Library: yes

Features:
---------
                 Parallel HDF5: no
            High Level library: yes
                  Threadsafety: no
           Default API Mapping: v18
With Deprecated Public Symbols: yes
        I/O filters (external): deflate(zlib),szip(encoder)
        I/O filters (internal): shuffle,fletcher32,nbit,scaleoffset
                           MPE: no
                    Direct VFD: no
                       dmalloc: no
Clear file buffers before write: yes
          Using memory checker: no
        Function Stack Tracing: no
                          GPFS: no
     Strict File Format Checks: no
  Optimization Instrumentation: no
      Large File Support (LFS): yes
****************************************

I had another query. What value does H5P_DEFAULT contain and where is
the value for this variable set. I was not able to find it from the
v1.8.9 source, please let me know. I ask, since the code I am working
with, sets the file access to H5P_DEFAULT if MPI is not being used, else
its set to H5Pset_fapl_mpio.

Please let me know if you need more information.

Thanks

Nitya

-----------------------------

Message: 4
Date: Tue, 2 Jul 2013 14:33:07 -0500
From: Albert Cheng <acheng@hdfgroup.org>
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
Subject: Re: [Hdf-forum] HDF5 - reading files larger than 2GB
Message-ID: <4C52ED59-20DB-4F48-A642-17CF4202D3BB@hdfgroup.org>
Content-Type: text/plain; charset="iso-8859-1"

Hi Nitya,

Can you also give more information about the machine in which you ran your application
that got the failures? E.g.,

1. OS version ("uname -a" usually shows it.)
1.1. For Mac, try "sw_vers".

2. Did you build the HDF5 library from source?
2.1. if yes, please send a copy of the output of configure.
2.2. If the output of configure is not available, please send a copy of libhdf5.settings.
       You can usually find "libhdf5.settings" in the same directory where you find the
       HDF5 library, libhdf5.a or libhdf5.so.

This will help us to try to repeat your problems in the appropriate platform.
Thanks.

-Albert Change
THG Staff

On Jul 2, 2013, at 11:20 AM, Dana Robinson <derobins@hdfgroup.org> wrote:

Hi Nitya,

I'll take a look into this. It's a holiday weekend here, so I probably won't get to this until next week.

Dana

On Mon, Jul 1, 2013 at 1:37 PM, Nitya Hariharan <nty@mpa-garching.mpg.de> wrote:
Hi,

I am having some trouble reading in a HDF5 file that is ~5GB. I was
initially having some problems even writing out a file this large and
looked at some postings in the HDF5 forum related to this.

http://hdf-forum.184993.n3.nabble.com/Trouble-writing-2GB-dataset-from-single-task-with-HDF5-1-8-10-td4025821.html

I tried using the serial version of the HDF5 library, v1.8.9 and was
able to write out a file of size ~5GB. However, when I try to read it
back in, I get the following error.

----------------
#000: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dio.c line
182 in H5Dread(): can't read data
   major: Dataset
   minor: Read failed
#001: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dio.c line
550 in H5D__read(): can't read data
   major: Dataset
   minor: Read failed
#002: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c
line 543 in H5D__contig_read(): contiguous read failed
   major: Dataset
   minor: Read failed
#003: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dselect.c
line 278 in H5D__select_read(): read error
   major: Dataspace
   minor: Read failed
#004: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dselect.c
line 213 in H5D__select_io(): read error
   major: Dataspace
   minor: Read failed
#005: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c
line 873 in H5D__contig_readvv(): can't perform vectorized sieve buffer read
   major: Dataset
   minor: Can't operate on object
#006: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5V.c line
1457 in H5V_opvv(): can't perform operation
   major: Internal error (too specific to document in detail)
   minor: Can't operate on object
#007: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c
line 674 in H5D__contig_readvv_sieve_cb(): block read failed
   major: Dataset
   minor: Read failed
#008: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Fio.c line
113 in H5F_block_read(): read through metadata accumulator failed
   major: Low-level I/O
   minor: Read failed
#009: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Faccum.c
line 258 in H5F_accum_read(): driver read request failed
   major: Low-level I/O
   minor: Read failed
#010: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5FDint.c
line 142 in H5FD_read(): driver read request failed
   major: Virtual File Layer
   minor: Read failed
#011: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5FDsec2.c
line 725 in H5FD_sec2_read(): file read failed: time = Mon Jul 1
20:21:57 2013
, filename = '/tmp/file.hdf5', file descriptor = 5, errno = 14, error
message = 'Bad address', buf = 0x2ae9619ea010, total read size =
4677466176, bytes this sub-read = 4677466176, bytes actually read =
18446744073709551615, offset = 744468544
-----------------

I looked at the forum again and saw this posting, which mentioned that
there was a bug fix done for POSIX issues.

http://mail.lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2012-December/006348.html

I was using v1.8.9, but thought of using the latest 1.8.11 to rule out
any issues with the HDF5 version I was using. However, I still get the
above error.

Please could someone provide some feedback on why this is happening. If
I am able to write out such a large file, I should be able to read it in
as well? Ofcourse, small sized files work perfectly fine in my application.

Since the library I am using, that in turn calls the HDF5 routines,
needs the interface of v1.6 of the HDF5 library, I use the flag
H5_USE_16_API while compiling. Would this in anyway cause the problem.

Thanks in advance.

--
Regards

Nitya

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

Hi Nitya,

(Sorry that I missed your other question.)

H5P_DEFAULT is defined as 0 in H5public.h which is #include by hdf5.h.
It represents the default value of all Property Lists and its effect depends on
what function API you are calling.

In your case, the default file access property is the default VFL (Virtual File Layer).
And the binary you used has it set to using the sec2 VFL which is same as the posix
API of open/close/read/write/lseek/etc..

This VFL should be able and has been tested to handle files larger than 32bit size files.

But I will first try to repeat the failure.

···

On Jul 4, 2013, at 4:31 AM, Nitya <nty@MPA-Garching.MPG.DE> wrote:

I had another query. What value does H5P_DEFAULT contain and where is
the value for this variable set. I was not able to find it from the
v1.8.9 source, please let me know. I ask, since the code I am working
with, sets the file access to H5P_DEFAULT if MPI is not being used, else
its set to H5Pset_fapl_mpio.

Please let me know if you need more information.

Thanks

Nitya

Hi Albert,

The distribution is SuSE, the contents of the SuSE-release file are

SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 2

Thanks for looking into this.

Nitya

···

On 07/09/13 20:41, Albert Cheng wrote:

Hi Nitya,

Looks like you are in a Linux 3.0 system.
Can you find out what distribution it is? E.g., Redhat, Fedora, Suse, �.
There is no one way to find this information but you often can find a file like
/etc/<XYZ>-release. If you can locate such a file, please send its content.

-Albert Cheng

On Jul 4, 2013, at 4:31 AM, Nitya <nty@MPA-Garching.MPG.DE> wrote:

Hi Albert, Dana,

Thanks for your replies. I give the output of uname below

Linux 3.0.74-0.6.10-default #1 SMP x86_64 x86_64 x86_64 GNU/Linux

I was earlier using v1.8.9 and then tried using v1.8.11. I have given
the libhdf5.settings file for both versions below. I downloaded the
library files of v1.8.11 from the HDF5 website, so haven't compiled
anything. v1.8.9 has been compiled by the system administrators.

-------------------
SUMMARY OF THE HDF5 CONFIGURATION
           =================================

General Information:
-------------------
                  HDF5 Version: 1.8.9
                 Configured on: Thu Nov 29 11:11:42 CET 2012
                 Configured by: wwn@qa01
                Configure mode: production
                   Host system: x86_64-unknown-linux-gnu
             Uname information: Linux qa01 3.0.42-0.7-default #1 SMP
Tue Oct 9 11:58:45 UTC 2012 (a8dc443) x86_64 x86_64 x86_64 GNU/Linux
                      Byte sex: little-endian
                     Libraries:
            Installation point:
/afs/rzg/common/soft/hdf5/1.8.9/amd64_sles11/intel/12.1/serial

Compiling Options:
------------------
              Compilation Mode: production
                    C Compiler:
/afs/@cell/common/soft/intel/ics/2011.0.013/12.1/bin/intel64/bin/icc (
Intel(R) C Intel(R) 64 Compiler Version 12.1 Build 20120928)
                        CFLAGS: -fPIC
                     H5_CFLAGS: -std=c99 -O3
                     AM_CFLAGS:
                      CPPFLAGS:
                   H5_CPPFLAGS: -D_POSIX_C_SOURCE=199506L -DNDEBUG
-UH5_DEBUG_API
                   AM_CPPFLAGS: -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -D_BSD_SOURCE
              Shared C Library: yes
              Static C Library: yes
Statically Linked Executables: no
                       LDFLAGS:
                    H5_LDFLAGS:
                    AM_LDFLAGS:
               Extra libraries: -lz -lrt -lm
                      Archiver: ar
                        Ranlib: ranlib
             Debugged Packages:
                   API Tracing: no

Languages:
----------
                       Fortran: yes
              Fortran Compiler:
/afs/@cell/common/soft/intel/ics/2011.0.013/12.1/bin/intel64/bin/ifort (
Intel(R) Fortran Intel(R) 64 Compiler Version 12.1 Build 20120928)
         Fortran 2003 Compiler: yes
                 Fortran Flags: -fPIC
              H5 Fortran Flags: -O3
              AM Fortran Flags:
        Shared Fortran Library: yes
        Static Fortran Library: yes
                           C++: yes
                  C++ Compiler:
/afs/@cell/common/soft/intel/ics/2011.0.013/12.1/bin/intel64/bin/icpc (
Intel(R) C++ Intel(R) 64 Compiler Version 12.1 Build 20120928)
                     C++ Flags:
                  H5 C++ Flags:
                  AM C++ Flags:
            Shared C++ Library: yes
            Static C++ Library: yes

Features:
---------
                 Parallel HDF5: no
            High Level library: yes
                  Threadsafety: no
           Default API Mapping: v18
With Deprecated Public Symbols: yes
        I/O filters (external): deflate(zlib)
        I/O filters (internal): shuffle,fletcher32,nbit,scaleoffset
                           MPE: no
                    Direct VFD: no
                       dmalloc: no
Clear file buffers before write: yes
          Using memory checker: no
        Function Stack Tracing: no
                          GPFS: no
     Strict File Format Checks: no
  Optimization Instrumentation: no
      Large File Support (LFS): yes
-----------------------------------------------

v1.8.11
***************************************
           SUMMARY OF THE HDF5 CONFIGURATION
           =================================

General Information:
-------------------
                  HDF5 Version: 1.8.11
                 Configured on: Wed May 8 16:20:56 CDT 2013
                 Configured by: hdftest@koala
                Configure mode: production
                   Host system: x86_64-unknown-linux-gnu
             Uname information: Linux koala 2.6.18-348.1.1.el5 #1 SMP
Tue Jan 22 16:19:19 EST 2013 x86_64 x86_64 x86_64 GNU/Linux
                      Byte sex: little-endian
                     Libraries: static, shared
            Installation point:
/mnt/scr1/pre-release/hdf5/v1811/thg-builds/koala

Compiling Options:
------------------
              Compilation Mode: production
                    C Compiler: /usr/bin/gcc ( gcc (GCC) 4.1.2 20080704 )
                        CFLAGS:
                     H5_CFLAGS: -std=c99 -pedantic -Wall -Wextra
-Wundef -Wshadow -Wpointer-arith -Wbad-function-cast -Wcast-qual
-Wcast-align -Wwrite-strings -Wconversion -Wag
gregate-return -Wstrict-prototypes -Wmissing-prototypes
-Wmissing-declarations -Wredundant-decls -Wnested-externs -Winline
-Wno-long-long -Wfloat-equal -Wmissing-format-attribut
e -Wmissing-noreturn -Wpacked -Wdisabled-optimization -Wformat=2
-Wunreachable-code -Wendif-labels -Wdeclaration-after-statement
-Wold-style-definition -Winvalid-pch -Wvariadic-
macros -Wnonnull -Winit-self -Wmissing-include-dirs -Wswitch-default
-Wswitch-enum -Wunused-macros -Wunsafe-loop-optimizations -Wc++-compat
-Wvolatile-register-var -O3 -fomit-fr
ame-pointer -finline-functions
                     AM_CFLAGS:
                      CPPFLAGS:
                   H5_CPPFLAGS: -D_POSIX_C_SOURCE=199506L -DNDEBUG
-UH5_DEBUG_API
                   AM_CPPFLAGS:
-I/mnt/hdf/packages/szip/shared/encoder/Linux2.6-x86_64-gcc/include
-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_BSD_SOURCE
              Shared C Library: yes
              Static C Library: yes
Statically Linked Executables: yes
                       LDFLAGS:
                    H5_LDFLAGS:
                    AM_LDFLAGS:
-L/mnt/hdf/packages/szip/shared/encoder/Linux2.6-x86_64-gcc/lib
               Extra libraries: -lsz -lz -lrt -ldl -lm
                      Archiver: ar
                        Ranlib: ranlib
             Debugged Packages:
                   API Tracing: no
Languages:
----------
                       Fortran: yes
              Fortran Compiler: /usr/bin/gfortran ( GNU Fortran (GCC)
4.1.2 20080704 )
         Fortran 2003 Compiler: no
                 Fortran Flags:
              H5 Fortran Flags:
              AM Fortran Flags:
        Shared Fortran Library: yes
        Static Fortran Library: yes

                           C++: yes
                  C++ Compiler: /usr/bin/g++ ( g++ (GCC) 4.1.2 20080704 )
                     C++ Flags:
                  H5 C++ Flags:
                  AM C++ Flags:
            Shared C++ Library: yes
            Static C++ Library: yes

Features:
---------
                 Parallel HDF5: no
            High Level library: yes
                  Threadsafety: no
           Default API Mapping: v18
With Deprecated Public Symbols: yes
        I/O filters (external): deflate(zlib),szip(encoder)
        I/O filters (internal): shuffle,fletcher32,nbit,scaleoffset
                           MPE: no
                    Direct VFD: no
                       dmalloc: no
Clear file buffers before write: yes
          Using memory checker: no
        Function Stack Tracing: no
                          GPFS: no
     Strict File Format Checks: no
  Optimization Instrumentation: no
      Large File Support (LFS): yes
****************************************

I had another query. What value does H5P_DEFAULT contain and where is
the value for this variable set. I was not able to find it from the
v1.8.9 source, please let me know. I ask, since the code I am working
with, sets the file access to H5P_DEFAULT if MPI is not being used, else
its set to H5Pset_fapl_mpio.

Please let me know if you need more information.

Thanks

Nitya

-----------------------------

Message: 4
Date: Tue, 2 Jul 2013 14:33:07 -0500
From: Albert Cheng <acheng@hdfgroup.org>
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
Subject: Re: [Hdf-forum] HDF5 - reading files larger than 2GB
Message-ID: <4C52ED59-20DB-4F48-A642-17CF4202D3BB@hdfgroup.org>
Content-Type: text/plain; charset="iso-8859-1"

Hi Nitya,

Can you also give more information about the machine in which you ran your application
that got the failures? E.g.,

1. OS version ("uname -a" usually shows it.)
1.1. For Mac, try "sw_vers".

2. Did you build the HDF5 library from source?
2.1. if yes, please send a copy of the output of configure.
2.2. If the output of configure is not available, please send a copy of libhdf5.settings.
       You can usually find "libhdf5.settings" in the same directory where you find the
       HDF5 library, libhdf5.a or libhdf5.so.

This will help us to try to repeat your problems in the appropriate platform.
Thanks.

-Albert Change
THG Staff

On Jul 2, 2013, at 11:20 AM, Dana Robinson <derobins@hdfgroup.org> wrote:

Hi Nitya,

I'll take a look into this. It's a holiday weekend here, so I probably won't get to this until next week.

Dana

On Mon, Jul 1, 2013 at 1:37 PM, Nitya Hariharan <nty@mpa-garching.mpg.de> wrote:
Hi,

I am having some trouble reading in a HDF5 file that is ~5GB. I was
initially having some problems even writing out a file this large and
looked at some postings in the HDF5 forum related to this.

http://hdf-forum.184993.n3.nabble.com/Trouble-writing-2GB-dataset-from-single-task-with-HDF5-1-8-10-td4025821.html

I tried using the serial version of the HDF5 library, v1.8.9 and was
able to write out a file of size ~5GB. However, when I try to read it
back in, I get the following error.

----------------
#000: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dio.c line
182 in H5Dread(): can't read data
   major: Dataset
   minor: Read failed
#001: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dio.c line
550 in H5D__read(): can't read data
   major: Dataset
   minor: Read failed
#002: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c
line 543 in H5D__contig_read(): contiguous read failed
   major: Dataset
   minor: Read failed
#003: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dselect.c
line 278 in H5D__select_read(): read error
   major: Dataspace
   minor: Read failed
#004: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dselect.c
line 213 in H5D__select_io(): read error
   major: Dataspace
   minor: Read failed
#005: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c
line 873 in H5D__contig_readvv(): can't perform vectorized sieve buffer read
   major: Dataset
   minor: Can't operate on object
#006: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5V.c line
1457 in H5V_opvv(): can't perform operation
   major: Internal error (too specific to document in detail)
   minor: Can't operate on object
#007: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Dcontig.c
line 674 in H5D__contig_readvv_sieve_cb(): block read failed
   major: Dataset
   minor: Read failed
#008: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Fio.c line
113 in H5F_block_read(): read through metadata accumulator failed
   major: Low-level I/O
   minor: Read failed
#009: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5Faccum.c
line 258 in H5F_accum_read(): driver read request failed
   major: Low-level I/O
   minor: Read failed
#010: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5FDint.c
line 142 in H5FD_read(): driver read request failed
   major: Virtual File Layer
   minor: Read failed
#011: /home/hdftest/snapshots-bin-hdf5_1_8_11/current/src/H5FDsec2.c
line 725 in H5FD_sec2_read(): file read failed: time = Mon Jul 1
20:21:57 2013
, filename = '/tmp/file.hdf5', file descriptor = 5, errno = 14, error
message = 'Bad address', buf = 0x2ae9619ea010, total read size =
4677466176, bytes this sub-read = 4677466176, bytes actually read =
18446744073709551615, offset = 744468544
-----------------

I looked at the forum again and saw this posting, which mentioned that
there was a bug fix done for POSIX issues.

http://mail.lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/2012-December/006348.html

I was using v1.8.9, but thought of using the latest 1.8.11 to rule out
any issues with the HDF5 version I was using. However, I still get the
above error.

Please could someone provide some feedback on why this is happening. If
I am able to write out such a large file, I should be able to read it in
as well? Ofcourse, small sized files work perfectly fine in my application.

Since the library I am using, that in turn calls the HDF5 routines,
needs the interface of v1.6 of the HDF5 library, I use the flag
H5_USE_16_API while compiling. Would this in anyway cause the problem.

Thanks in advance.

--
Regards

Nitya

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

--
Regards

Nitya