Pure Go HDF5 Library - HDF5 2.0.0 Format Support Announcement

News and Announcements
1

The HDF5 Go Library team is pleased to announce that our latest production releases achieve full compatibility with the HDF5 2.0.0 format specification.

Overview

This milestone represents comprehensive support for HDF5 2.0.0, including Format Specification v4.0 compliance, security hardening, and native AI/ML datatype handling—all delivered as a pure Go implementation without CGo dependencies.

Key Features

HDF5 2.0.0 Format Support

  • Superblock versions 0, 2, and 3: Complete read/write implementation
  • Format Specification v4.0 compliance: Implements all specified superblock formats
  • Checksum validation: CRC32 validation for modern superblocks
  • Full backward compatibility: Seamless support for HDF5 1.x and 2.0.0 files

Security Hardening

Fixed 4 CVEs (HIGH to MEDIUM severity):

  • CVE-2025-7067 (HIGH 7.8): Buffer overflow in chunk reading
  • CVE-2025-6269 (MEDIUM 6.5): Heap overflow in attribute reading
  • CVE-2025-2926 (MEDIUM 6.2): Stack overflow in string handling
  • CVE-2025-44905 (MEDIUM 5.9): Integer overflow in hyperslab selection

Advanced Features

  • 64-bit chunk dimensions: Support for chunks >4GB per dimension
  • AI/ML datatypes: Native FP8 E4M3, FP8 E5M2, bfloat16 support
  • Production-ready rebalancing: Automatic B-tree optimization API

Quality Metrics

  • :white_check_mark: 86.1% test coverage
  • :white_check_mark: 0 linter issues (34+ linters)
  • :white_check_mark: HDF Group reference tests: 98.2% pass rate (433 files)
  • :white_check_mark: Cross-platform: Linux, macOS, Windows

Development Phase Transition

With HDF5 2.0.0 format support now complete, the project is transitioning from rapid development to a stable maintenance phase:

  • Focus: Minor improvements, bug fixes, and performance optimizations
  • Monitoring: Active tracking of HDF Group’s reference library releases and announcements
  • Compatibility: Continued alignment with official HDF5 specification updates
  • Community-driven: Prioritizing user feedback and real-world use cases

Installation

go get github.com/scigolib/hdf5@latest

Requirements: Go 1.25+, no external dependencies (pure Go, no CGo)

Resources

Use Cases

This library is particularly well-suited for:

  • Scientific computing: Cross-platform HDF5 access without C library dependencies
  • Cloud deployments: Pure Go simplifies containerization and deployment
  • Large-scale data: 64-bit chunk support for massive datasets
  • Machine learning: Native AI/ML datatype support (FP8, bfloat16)
  • Security-critical applications: Comprehensive overflow protection

Technical Details

Architecture: Pure Go implementation with zero CGo dependencies
Performance: Optimized for Go runtime and garbage collector
Compatibility: Reads/writes files compatible with HDF5 C library
Standard Compliance: Implements HDF5 Format Specification v4.0

Acknowledgments

We extend our sincere gratitude to:

  • The HDF Group for their comprehensive format specification and excellent documentation
  • Community contributors who provided feedback, bug reports, and feature requests
  • Early adopters who tested the library in production environments
  • Scientific computing community for supporting pure Go implementations of scientific formats

This project demonstrates the viability and benefits of pure Go implementations for complex scientific data formats. Your continued support and feedback drive the project’s evolution and improvement.

Project: GitHub - scigolib/hdf5: Modern Pure Go implementation of the HDF5 file format
License: MIT
Status: Production-ready
Latest Release: See Releases · scigolib/hdf5 · GitHub

2 Likes