Help with broken file

HDF5 Library
1

Using h5py I somehow got an broken h5 file I can’t open. Below is the output of h5dump I get. Is there any chance to save the data?

h5dump --enable-error-stack=2 test.h5
HDF5-DIAG: Error detected in HDF5 (1.10.6) thread 140446101915456:
  #000: H5F.c line 509 in H5Fopen(): unable to open file
    major: File accessibilty
    minor: Unable to open file
  #001: H5Fint.c line 1661 in H5F_open(): unable to read root group
    major: File accessibilty
    minor: Unable to open file
  #002: H5Groot.c line 239 in H5G_mkroot(): can't check if symbol table message exists
    major: Symbol table
    minor: Can't get value
  #003: H5Omessage.c line 883 in H5O_msg_exists(): unable to protect object header
    major: Object header
    minor: Unable to protect metadata
  #004: H5Oint.c line 1066 in H5O_protect(): unable to load object header
    major: Object header
    minor: Unable to protect metadata
  #005: H5AC.c line 1352 in H5AC_protect(): H5C_protect() failed
    major: Object cache
    minor: Unable to protect metadata
  #006: H5C.c line 2345 in H5C_protect(): can't load entry
    major: Object cache
    minor: Unable to load metadata into cache
  #007: H5C.c line 6685 in H5C_load_entry(): incorrect metadatda checksum after all read attempts
    major: Object cache
    minor: Read failed
  #008: H5Ocache.c line 219 in H5O__cache_get_final_load_size(): can't deserialize object header prefix
    major: Object header
    minor: Unable to decode value
  #009: H5Ocache.c line 1231 in H5O__prefix_deserialize(): bad object header version number
    major: Object header
    minor: Wrong version number
HDF5-DIAG: Error detected in HDF5 (1.10.6) thread 140446101915456:
  #000: H5F.c line 509 in H5Fopen(): unable to open file
    major: File accessibilty
    minor: Unable to open file
  #001: H5Fint.c line 1500 in H5F_open(): unable to open file: name = 'test.h5', tent_flags = 0
    major: File accessibilty
    minor: Unable to open file
  #002: H5FD.c line 734 in H5FD_open(): open failed
    major: Virtual File Layer
    minor: Unable to initialize object
  #003: H5FDfamily.c line 693 in H5FD_family_open(): file names not unique
    major: File accessibilty
    minor: File already exists
HDF5-DIAG: Error detected in HDF5 (1.10.6) thread 140446101915456:
  #000: H5F.c line 509 in H5Fopen(): unable to open file
    major: File accessibilty
    minor: Unable to open file
  #001: H5Fint.c line 1500 in H5F_open(): unable to open file: name = 'test.h5', tent_flags = 0
    major: File accessibilty
    minor: Unable to open file
  #002: H5FD.c line 734 in H5FD_open(): open failed
    major: Virtual File Layer
    minor: Unable to initialize object
HDF5-DIAG: Error detected in HDF5 (1.10.6) thread 140446101915456:
  #000: H5F.c line 509 in H5Fopen(): unable to open file
    major: File accessibilty
    minor: Unable to open file
  #001: H5Fint.c line 1500 in H5F_open(): unable to open file: name = 'test.h5', tent_flags = 0
    major: File accessibilty
    minor: Unable to open file
  #002: H5FD.c line 734 in H5FD_open(): open failed
    major: Virtual File Layer
    minor: Unable to initialize object
h5dump error: unable to open file "test.h5"`
```
2

I don’t want to create false hopes, but there are two low-level tools, h5check and h5debug, that you could use to do some investigative work. h5dump can’t find the root group, which is bad. Do you know how many datasets were supposed to be in the file and how they were layed out (contiguous, compact, chunked) and what their type and dimensions were? Which file format specification version did you use? For starters, you can poke around for strings such as SNOD, TREE, OHDR, etc., to get clues for offsets to look at with h5debug.

In the simplest case, youd’be dealing with a contiguously (or compactly) laid out dataset of a standard scalar type, and you’d be able to locate that dataset’s object header. From there you might be able to work your way through the metadata to the contiguous region containing the values, which you then can read without the HDF5 library and assuming that that data wasn’t corrupted or only written partially. Everything else (chunked layout) is going to be harder and require familiarity with the file format specification. Not impossible, but much more involved.

FWIW, a small file format tutorial can be found here.

Best, G.