Hermes 0.9.0-beta has been released. This release features Data Staging, which enables external datasets to be loaded into Hermes for processing. Data Staging is frequently used for extracting information from large datasets during scientific simulations and workflows. This approach shifts the burden of synchronization, aggregation, validation, cleaning, and processing from the PFS and main memory to the high-performance staging area. I/O stalls are reduced by overlapping compute phases with asynchronous I/O, utilizing the high-performance compute network, and leveraging storage tiering.
2 Likes