Create/open a file with SWMR

HDF5 Library
1

I am trying to use HDF5 to save log data which can be monitored by separate processes as my simulation is running.

The SWMR feature appears to be what I want, however when I try to pass H5F_ACC_RDWR | H5F_ACC_SWMR_WRITE for the flags to H5Fcreate it complains about an invalid flag.

I also tried creating the file initially, calling H5Fclose, then opening the file again with H5Fopen I think the file is still open somewhere in cache as I get a File accessibility error about unable to unpin superblock:

HDF5-DIAG: Error detected in HDF5 (1.12.2) MPI-process 0:
#000: H5F.c line 620 in H5Fopen(): unable to open file
major: File accessibility
minor: Unable to open file
#001: H5VLcallback.c line 3502 in H5VL_file_open(): failed to iterate over available VOL connector plugins
major: Virtual Object Layer
minor: Iteration failed
#002: H5PLpath.c line 579 in H5PL__path_table_iterate(): can’t iterate over plugins in plugin path ‘(null)’
major: Plugin for dynamically loaded library
minor: Iteration failed
#003: H5PLpath.c line 620 in H5PL__path_table_iterate_process_path(): can’t open directory: /usr/local/hdf5/lib/plugin
major: Plugin for dynamically loaded library
minor: Can’t open directory or file
#004: H5VLcallback.c line 3351 in H5VL__file_open(): open failed
major: Virtual Object Layer
minor: Can’t open object
#005: H5VLnative_file.c line 97 in H5VL__native_file_open(): unable to open file
major: File accessibility
minor: Unable to open file
#006: H5Fint.c line 1990 in H5F_open(): unable to read superblock
major: File accessibility
minor: Read failed
#007: H5Fsuper.c line 1053 in H5F__super_read(): unable to unpin superblock
major: File accessibility
minor: Unable to un-pin cache entry
#008: H5AC.c line 1590 in H5AC_unpin_entry(): can’t unpin entry
major: Object cache
minor: Unable to un-pin cache entry
#009: H5C.c line 3100 in H5C_unpin_entry(): Can’t unpin entry from client
major: Object cache
minor: Unable to un-pin cache entry
#010: H5C.c line 4231 in H5C__unpin_entry_from_client(): entry isn’t pinned
major: Object cache
minor: Unable to un-pin cache entry
#011: H5Fsuper.c line 477 in H5F__super_read(): superblock version for SWMR is less than 3
major: File accessibility
minor: Bad value

Is there some other way to create a file with SWMR access?

2

I too have a similar use. I have a simulation program where run times can extend over a week (and potentially more when needed). Periodically, we log the state (parts of the state) as frames for later use and/or failure analysis. While the simulation is running, we wish to have a different process to monitor the progress of the simulation.

The first attempt at doing this was to do this without SWMR (open, log new frame, close, simulate for some period, repeat). This somewhat works, excepting that, when the observer program’s use intersects with the simulator’s use.

The second attempt is to use SWMR.

This works better, but we are having issues that are hard to resolve by reading the reference manual. An explanation of the issue:
The simulator opens/creates the file with:

    outFile[unitOut] = H5File(fileName, H5F_ACC_SWMR_WRITE | H5F_ACC_TRUNC);

inserts some header, then issues:

    outFile[unitOut].flush(H5F_SCOPE_LOCAL);

Then periodically, appends a “frame” of new data (/Frames/nnn) where nnn increases each frame.
At end of new frame writes:

    outFile[unitOut].flush(H5F_SCOPE_LOCAL);

There is several hundreds of new data per frame.

This works without errors.

To emulate the monitor process, I use HDFView with Open As: SWMR Read Only

The open succeeds, and I can view the current data, and the simulator (the SW) continues to run

Now then the problem.

When the simulator enters the append next frame:

  1. the create new group succeeds
  2. the new attributes can be written
  3. but the new datasets abort

Write attribute that works:
// open the group
Group group(outFile[unitOut].openGroup(Place));
// Create the data space for the attribute.
DataSpace attr_dataspace = DataSpace(rank, dims);
// create attribute (with empty data)
Attribute attribute = group.createAttribute(Name, type, attr_dataspace);
// Write the attribute data.
attribute.write(type, base);
attribute.close();
attr_dataspace.close();
group.close();

Write DataSet that fails:

   // open the group
    Group group(outFile[unitOut].openGroup(Place));
    // Create the data space for the Dataset.
    DataSpace attr_dataspace = DataSpace(rank, dims);
    // create Dataset (with empty data)
    DataSet dataset = group.createDataSet(Name, type, attr_dataspace);
    // Write the Dataset data.
    dataset.write(base, type);
    dataset.close();
    attr_dataspace.close();
    group.close();

The failure report is:

major: Virtual File Layer
minor: Write failed
#016: C:\autotest\hdf5114-StdRelease-dist-10vs17\build\hdfsrc\src\H5FDsec2.c line 814 in H5FD__sec2_write(): file write failed: time = Sun Apr 7 11:07:57 2024
, filename = ‘GRDB65.h5’, file descriptor = 3, errno = 13, error message = ‘Permission denied’, buf = 00000143B2B5AEB8, total write size = 288, bytes this sub-write = 288, bytes actually written = 18446744073709551615, offset = 287341
major: Low-level I/O
minor: Write failed
Error writing attribute /Frames/2/Tethers/Tether 1/Beads/Positions RPIF

the 18446744073709551615 hex FFFF FFFF FFFF FFFF (-1)

So why does the write attribute work and the write DataSet fail?

Does this require some sort of:

while(dataset.AttemptLockFails())
wait()
dataset.write(…)
dataset.unlock()

While the dataset does not have lock and unlock, does something higher up (e.g. file) require this?

Jim

3

Jim, can you clarify something for us? Are you saying that you are creating groups and attributes as part of the same file open/close cycle during which you are doing the SWMR dataset write? If that were the case, that would be outside the scope of the current SWMR implementation.

The current implementation is not “full SWMR.” Under full SWMR, you could create arbitrary new objects and update or extend datasets as you like. The current implementation is limited to dataset updates/appends but does not support the creation of new items (groups, attributes, etc.).

The VFD SWMR branch supports full SWMR but is not yet part of the main codebase.

OK? G.

4

Thanks for the reply.

Yes, I am creating new groups, attributes and datasets as I append a frame (log of state). I am also updating one attribute (number of frames written) after updating the frame, then flushing the file.

This is performed in a simulator (Space Elevator), run times may be hours, days, weeks, or longer. We recently switched over to logging in HDF5. Prior to this I am using a predecessor: Intel Array Visualizer. This use .h4/.h5 but is not compatible with the new HDF5. This library does permit SWMR, though its implementation (my presumption) is different. I think it uses a memory mapped file. Updates are very fast and I can have multiple processes (observers) of the database, one producer) at very high frame rates.

I am in no rush to have full SWMR as I can produce both New HDF5 and use the older system side-by-side. At some point I wish to discard the older code.

I will check your provided link, and anticipate I will be an early adopter of VFD SWMR.

Jim Dempsey