Allow other metadata hashes (crc32c) as an alternative to Bob Jenkins' lookup3

The HDF5 library uses Bob Jenkins’ lookup3 for metadata hashing:

https://burtleburtle.net/bob/c/lookup3.c

@koziol selected over crc-32 and fletcher-32 in 2006 for the following reason:

Add ‘loookup3’ checksum routine and switch to using it for metadata checksums - it’s just as “strong” as the CRC32 and about 40% faster in general (with some compiler optimizations, it’s nearly as fast as the fletcher-32 algorithm).

Recent interest in lookup3 seems mainly spurred by HDF5.

Since 2006, crc32 is now hardware accelerated:

crc32c is also commonly found as packages for many languages, spurred in part by the hardware acceleration:

Is lookup3 actually still much faster than crc-32c?

Given the hardware acceleration and availability of crc-32c, should we allow for crc-32c as an alternate hash other than Bob Jenkins’ lookup3 in HDF5 2.0?

1 Like