The VFD interface provides a good abstraction for hardware-accelerated I/O routines. John Ravi’s GDS VFD is one such example: it uses NVIDIA’s GPUDirect Storage to initiate I/O to a
cudaMalloc'd buffer provided by the caller.
Different VFDs may deal with memory managed by other specialized allocators: a VFD optimized to read from Samsung’s SmartSSD would need to perform I/O to a buffer allocated via OpenCL’s
clCreateBuffer(), for instance.
However, I don’t want to bloat every application that needs to support said VFDs by introducing
#ifdefs and explicit calls to device-specific memory allocators. For this reason I started to look around to find which kind of abstraction the VFD interface provides for memory management.
I have noticed that the
H5FD_class_t provides two members named
free that I’ve initially presumed to be in charge of interfacing with specialized memory allocators (which would be just perfect):
H5FD__alloc_real(H5FD_t *file, ...)calls the VFD
H5FD__free_real(H5FD_t *file, ...)calls the VFD
However, code comments (“allocate space in the file”) and the APIs below suggest that
free relate to
H5FD_alloc(H5FD_t *file, ..., H5F_t *f, ...)marks the file
fend-of-allocated flag dirty
H5FDalloc(H5FD_t *file, ..., H5F_t *f, ...)modifies the dataset access property list
In the absence of memory-management APIs at the VFD layer I’d like to ask if this is something that’s captured somebody else’s attention before. Is there someone working on this topic already? If that’s not the case then I’d be happy to prototype an extension to
H5FD_class_t as long as that’s useful to the community.
Such an API would be useful for two primary reasons. First, applications could be as vendor/VFD-neutral as possible. Second, I remember having seen some code in the I/O filter path (
H5Z*.c) which assumed that the output buffers had been allocated with
malloc. If that code were to call into the VFD to reallocate buffers that wouldn’t be a problem anymore.