Hi all,
We are pleased to announce the Alpha 2 pre-release of the HDF5 library with support for the new Single-Writer/Multiple Reader (SWMR) capability, called “VFD SWMR.” VFD SWMR is designed to be a more flexible, more modular, and better-performing replacement for the existing HDF5 SWMR feature.
The VFD SWMR pre-release can be obtained from this Github repo:
https://github.com/HDFGroup/hdf5/tree/feature/vfd_swmr_alpha_2
Fundamentally, the feature has not changed significantly since the alpha 1 release as we’ve been focused on integration and ensuring that the VFD SWMR branch builds and passes tests on a wide variety of platforms and compilers. We’d especially like to get feedback on any platforms where the software cannot build and pass make check or ctest checks.
Known issues in this release:
-
The mirror VFD tests will not compile due to changes in the “zoo” test functionality.
-
OS X 1.10.11 (circa 2015) and earlier do not have clock_gettime(3) and will fail to compile. OS X 1.10.12 and newer should be fine.
-
The Java tests fail on Windows due to internal changes to how H5Fis_accessible() works.
-
Windows, in general, is poorly tested. It will compile and pass all non-Java tests, but we are currently bypassing two tests that have file permission problems. This may just be a test issue and not a fundamental VFD SWMR issue.
-
CMake builds still do not run many of the VFD SWMR tests.
-
On Windows, Visual Studio 2015 or newer is required to build the branch.
If you come across any other build or test errors, please let us know so we can look into the issue.
Important changes in this release:
-
The demo programs have been moved to the examples directory of this repository, however they are not integrated into the build and must be built separately. A README file in that directory describes how to build them.
-
The VFD SWMR acceptance tests now use the HDF5TestExpress environment variable. The default is now to perform quicker, less exhaustive testing. Setting the environment variable to 0 will run the lengthy tests that consume significant disk space.
-
The branch should build and pass tests on OS X 1.10.12 and up.
-
The branch should build and pass all tests (aside from Java) on Windows. This platform has not been very well tested, though.
In the repository’s doc directory, you will find “vfd-swmr-user-guide.md” and “VFD_SWMR_RFC_200916.pdf.” The latter describes the architecture and implementation.
More about VFD SWMR. SWMR is a feature of the HDF5 library that lets a process write data to an HDF5 file while one or more processes reads from the file. Use cases range from monitoring data collection and/or steering experiments to financial applications.
VFD SWMR is designed to be a more flexible, more modular, better-performing replacement for the existing SWMR feature.
· VFD SWMR allows HDF5 objects (groups, datasets, attributes) to be created and destroyed in the course of a reader-writer session. Creating and destroying objects is not possible using the existing SWMR feature.
· It compartmentalizes much of the SWMR functionality in the bottom layers of the HDF5 library, which both eases the HDF Group’s software maintenance burden, and allows non-SWMR applications to avoid SWMR related overhead.
· And it makes guarantees for the maximum time from write to availability of data for read, provided that the reading and writing systems and their interconnections can keep up with the data flow.
For details about the VFD SWMR architecture, see the VFD SWMR RFC in the repository’s doc directory.
Regards,
The VFD SWMR Team