# Reading from a HDF5 file using h5py fails

**URL:** <https://forum.hdfgroup.org/t/reading-from-a-hdf5-file-using-h5py-fails/9848>\
**Category:** HDF5 Library\
**Created:** [June 9, 2022, 11:32am UTC](https://forum.hdfgroup.org/t/reading-from-a-hdf5-file-using-h5py-fails/9848 "2022-06-09T11:32:45Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![vaclav.sisl](https://forum.hdfgroup.org/letter_avatar_proxy/v4/letter/v/73ab20/32.png) [@vaclav.sisl](https://forum.hdfgroup.org/u/vaclav.sisl)\
**Post date:** [June 9, 2022, 11:32am UTC](https://forum.hdfgroup.org/t/reading-from-a-hdf5-file-using-h5py-fails/9848/1 "2022-06-09T11:32:46Z")

</div>

Hello,

I suddenly started having troubles reading data from a relatively large HDF5 file (156 GB). In the past, I could read the file without issues but now I get the following error raised from h5py:

```auto
OSError: Can't read data (file read failed: 
time = Thu Jun 9 10:40:52 2022, 
filename = '../path/to/my_file.h5', 
file descriptor = 10, 
errno = 5, 
error message = 'Input/output error', 
buf = 0x7f77b1f422e0, 
total read size = 12139376, 
bytes this sub-read = 12139376, 
bytes actually read = 18446744073709551615, 
offset = 42224603136)

```

I’m only getting this error when I try to read certain datasets. Other datasets load fine.

I could find this [thread](https://github.com/h5py/h5py/issues/1610) where someone was facing a similar error but then they realised that it only affected them when they were trying to read a file from a network location. In my case, I have the file stored locally on an SSD (ext4 filesystem).

Is it possible that my file is corrupt? Do you have any suggestions on what I could check or try?

Thanks for help!

---

<div class="post-metadata">

**Author:** ![vaclav.sisl](https://forum.hdfgroup.org/letter_avatar_proxy/v4/letter/v/73ab20/32.png) [@vaclav.sisl](https://forum.hdfgroup.org/u/vaclav.sisl)\
**Post date:** [June 9, 2022, 12:07pm UTC](https://forum.hdfgroup.org/t/reading-from-a-hdf5-file-using-h5py-fails/9848/2 "2022-06-09T12:07:36Z")

</div>

Here’s another [case](https://github.com/h5py/h5py/issues/1592) but again only when using a network storage.

---

<div class="post-metadata">

**Author:** ![vaclav.sisl](https://forum.hdfgroup.org/letter_avatar_proxy/v4/letter/v/73ab20/32.png) [@vaclav.sisl](https://forum.hdfgroup.org/u/vaclav.sisl)\
**Post date:** [June 9, 2022, 12:09pm UTC](https://forum.hdfgroup.org/t/reading-from-a-hdf5-file-using-h5py-fails/9848/3 "2022-06-09T12:09:43Z")

</div>

It seems that in my case, the error was caused by a corrupt file. I re-wrote my file by a back-up version and now I’m not getting the error anymore. Hopefully it stays that way.

---

<div class="post-metadata">

**Author:** ![vaclav.sisl](https://forum.hdfgroup.org/letter_avatar_proxy/v4/letter/v/73ab20/32.png) [@vaclav.sisl](https://forum.hdfgroup.org/u/vaclav.sisl)\
**Post date:** [December 3, 2022, 9:07pm UTC](https://forum.hdfgroup.org/t/reading-from-a-hdf5-file-using-h5py-fails/9848/4 "2022-12-03T21:07:55Z")

</div>

Coming back after a few months…

Recently this error came back and now it seems to re-occur. I keep “solving” the problem by replacing the corrupt(?) HDF5 file by a backup version. But after a while the problem returns.

I observe the error on the same machine as before so it’s possible that my hard drive has some HW problem.

Previously I was only able to find posts online about the same problem but only when data was stored on network drives. [This post](https://github.com/h5py/h5py/issues/2084) talks about the same problem when data is stored on NTFS SSD (Windows system).

---

<div class="post-metadata">

**Author:** ![gheber](https://forum.hdfgroup.org/user_avatar/forum.hdfgroup.org/gheber/32/1919_2.png) [@gheber](https://forum.hdfgroup.org/u/gheber)\
**Post date:** [December 5, 2022, 1:19pm UTC](https://forum.hdfgroup.org/t/reading-from-a-hdf5-file-using-h5py-fails/9848/5 "2022-12-05T13:19:26Z")

</div>

Can you tell us a little more about the environment? OS? HDF5/h5py version?  
What’s the output of

```auto
h5stat --enable-error-stack <HDF5 file>

```

on one of your “troublemakers?” What happens if you run this in a loop (100 or 1000 times)? Does it fail randomly?

G.
