New PyTables 3.8.0 leveraging Blosc2 as a second data partition

Hi there,

On behalf of the PyTables team, I am happy to announce that the new release of PyTables comes, among other improvements, with support for the Blosc2 compressor, not only as a regular HDF5 filter, but also as another data partition layer in addition to the existing data partition in HDF5: the chunks. This allows choosing partition sizes that better fit in the different cache levels of modern CPUs for improved performance.

See how this can accelerate different operations in our recent blog: https://www.blosc.org/posts/blosc2-pytables-perf/

Many thanks to Elena Pourmal for suggesting the direct chunking mechanism in HDF5 in order to implement the second partition.

Finally, PyTables recently reached its 20 anniversary; here it is a short history: https://www.blosc.org/posts/pytables-20years/

Cheers!

3 Likes