# Proposal: shape-aware filter plugins for HDF5 (cross-codec preprocessor with measured wins)

**URL:** <https://forum.hdfgroup.org/t/proposal-shape-aware-filter-plugins-for-hdf5-cross-codec-preprocessor-with-measured-wins/13782>\
**Category:** HDF5 Library\
**Created:** [May 3, 2026, 10:45pm UTC](https://forum.hdfgroup.org/t/proposal-shape-aware-filter-plugins-for-hdf5-cross-codec-preprocessor-with-measured-wins/13782 "2026-05-03T22:45:32Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![rjamesy](https://forum.hdfgroup.org/letter_avatar_proxy/v4/letter/r/d78d45/32.png) [@rjamesy](https://forum.hdfgroup.org/u/rjamesy)\
**Post date:** [May 3, 2026, 10:45pm UTC](https://forum.hdfgroup.org/t/proposal-shape-aware-filter-plugins-for-hdf5-cross-codec-preprocessor-with-measured-wins/13782/1 "2026-05-03T22:45:32Z")

</div>

````auto
## Question: would STRATA-style shape-aware filters be useful as HDF5 plugins?

Hi all,

I have built a small MIT-licensed lossless preprocessing library called **STRATA** :

https://github.com/rjamesy/strata

STRATA exposes a bank of reversible structural transforms:

- 2D predictors
- 3D axis deltas
- full octahedral cube rotation for 3D volumes, using all 48 orientations
- concentric-shell radial reordering
- YCoCg-R lossless colour decorrelation
- BWT
- an auto-selector that always includes `raw` as a candidate, so the floor is “no worse than the underlying codec alone”

The results are reproducible via:

```bash
python3 bench/preprocess_demo.py

````

That script writes a CSV covering the tested codec × dataset combinations.

Some current measurements:

| Dataset | Pipeline | Result |
| --- | --- | --- |
| Smooth 256×256 heightmap | `predict_2d_gradient` + `deflate-9` | **57.1% smaller than raw `deflate-9`** |
| 64³ CT-style volume | cube rotation + radial + `zstd-22` | **13.3–14.4% smaller than raw `zstd-22`** |
| 2252×4000 RGB photo | `YCoCg-R` + `zstd-22` | **35% smaller than raw `zstd-22`** |

The 3D volume case is the one I think most overlaps with HDF5 use cases.

The idea is simple: cube rotation can align the smoothest direction of a non-axis-aligned volume with the storage axis before delta-style encoding. For the `volume_64.raw` CT-style test, this turns a “roughly tied with raw bytes” case into a roughly 14% improvement on top of `zstd-22`.

This seems structurally aligned with HDF5’s chunk-filter model, so I wanted to ask before doing any plugin work.

### Questions

1. **Would these make sense as registered HDF5 filters?**

2. **Is an auto-select-with-floor meta-filter appropriate for HDF5?**

3. **What reference datasets should I benchmark?**

Relevant links:

- Repo: [https://github.com/rjamesy/strata](https://github.com/rjamesy/strata)

- Cross-codec results: [https://github.com/rjamesy/strata/blob/main/bench/results/F10\_PREPROCESS\_RESULTS.md](https://github.com/rjamesy/strata/blob/main/bench/results/F10_PREPROCESS_RESULTS.md)

- v0.4.0 release: [https://github.com/rjamesy/strata/releases/tag/v0.4.0](https://github.com/rjamesy/strata/releases/tag/v0.4.0)

No expectation that this belongs inside HDF5 itself. The cleaner home may simply be the filter registry. I mainly wanted to ask whether the approach is useful to the HDF5 ecosystem before packaging the transforms as H5Z plugins.

Best,  
Richard James

```auto

```

---

<div class="post-metadata">

**Author:** ![mlarson](https://forum.hdfgroup.org/letter_avatar_proxy/v4/letter/m/8c91f0/32.png) [@mlarson](https://forum.hdfgroup.org/u/mlarson)\
**Post date:** [May 5, 2026, 4:12pm UTC](https://forum.hdfgroup.org/t/proposal-shape-aware-filter-plugins-for-hdf5-cross-codec-preprocessor-with-measured-wins/13782/2 "2026-05-05T16:12:26Z")

</div>

1. If each filter performs independent operations, then it makes sense to package them separately.
2. The filters have enough flexibility in their operation that this should be fine - the mode tag can be included in the filtered output, and and decompression time, the mode tag can be checked to determine the decompression mode.
3. If you just want a quick test, the `chunk` benchmark executable (under `build/bin/chunk` if you build HDF5 from source) creates a small (~4MB) chunked dataset you can try your filters on. Most public scientific datasets are already chunked, so you’d need to use `h5repack` to decompress them before testing your own filters