HDF5-UDF 2.1 released -- with support for Windows and macOS!

[lucasvr]

Greetings, all!

I’m happy (and I mean it!) to announce the first release of User-Defined Functions for HDF5 supporting Windows and macOS in addition to Linux.

User-Defined Functions allow you to write routines in Python, C/C++, and Lua and attach them to HDF5 files so that each time the associated dataset is read, its contents are produced on-the-fly by those routines. It’s possible to create routines that reach out to the Internet, that take input from files in other formats, and even that play Doom on HDF5!

Since macOS ships with several development tools, installation of HDF5-UDF goes through a straightforward source code compilation process on that platform. The Windows port features a friendly installer that fetches the runtime dependencies from the web and installs them for you.

Please note that support for Windows and macOS is considered experimental. In special, sandboxing of UDF processes is not implemented yet. Nevertheless, they both enable you to prototype ideas and see HDF5 in a whole different way.

Please visit the project page on GitHub for download directions. Last, but not least, I’d like to thank @werner for his technical assistance that helped me take informed decisions while developing the Windows port.

Have fun!
Lucas

[werner]

Pretty cool! What do you think would be required to have this functionality available from HDFView? Just for reading data equipped with HDF5-UDF in readonly mode (of course, HDFView shouldn’t compile source code just when browsing files).

[lucasvr]

@werner

Pretty cool! What do you think would be required to have this functionality available from HDFView? Just for reading data equipped with HDF5-UDF in readonly mode (of course, HDFView shouldn’t compile source code just when browsing files).

Ah! HDF5-UDF uses a trick to overcome a limitation of the HDF5 I/O filter API: it attempts to open the HDF5 file that hosts the UDF so it can get a handle on that file. However, I just learned that HDFView takes a lock on the HDF5 file it opened, which causes HDF5-UDF’s attempt to open the same file to fail with “Resource temporarily unavailable” (as the same lock cannot be taken twice).

I have just fixed this problem by temporarily setting HDF5_USE_FILE_LOCKING=FALSE prior to opening the HDF5 file. The screenshot below shows the UserDefinedDataset being correctly displayed by HDFView now. Note that you can apply a local workaround by simply setting HDF5_USE_FILE_LOCKING=FALSE on the command line prior to launching HDFView.

Thanks for the bug report!
Lucas