As part of a validation study related to computational storage with HDF5, I ended up writing an I/O filter for data (de)compression using GPUs. The I/O filter is based on Mohammad Dashti’s wonderful snappy-CUDA project.
Thanks Elena. Judging from the implementation of cuda-c’s uncompressor alone, the two implementations are equivalent. However, since we don’t have access to the source code of the existing Snappy I/O filter, I can’t tell if the two filters are compatible. I’d like to know if Snappy ID 32003 prepends other metadata to the beginning of each compressed block, for instance.
I sent an email to Michael Rissi asking him for directions to download the filter. If I don’t hear from him until the end of the week then I think it’s safer to use a new ID.
We can reuse most of the wording that describes Snappy already. Here’s a suggested description for the new filter. Please feel free to adjust as needed.
Filter ID: 32003
Snappy-CUDA is a compression/decompression library that leverages GPU processing power to compress/decompress data. The Snappy compression algorithm does not aim for maximum compression or compatibility with any other compression library; instead, it aims for very high speeds and reasonable compression. For instance, compared to the fastest mode of zlib, the reference implementation of Snappy on the CPU is an order of magnitude faster for most inputs, but the resulting compressed files are anywhere from 20% to 100% bigger.
Lucas C. Villa Real
Email: lucasvr at gmail dot com