Based on Richard D. Snyder FZMQ fortran package here I post a Fortran and H5CPP event processor, decoupled with 0ZMQ middle-ware. One advantage of this setup is that the framework allows various levels of robustness, as well as multiprocessing and multithreading. Another is the flexible interconnect allows prototyping in python/julia/R/matlab
before final implementation, or plug in HDF5 into existing software.
The events are 8bytes integers, a tiny fragment indeed, and the performance is 8'829'670 events/sec on a Lenovo X1 11th Gen Intel®;
program main
use, intrinsic :: iso_c_binding
use :: zmq
type(c_ptr) :: ctx, sock
integer(c_int) :: res, rc
integer(c_size_t), target :: i, size=8
! push - pull pattern
ctx = zmq_ctx_new()
sock = zmq_socket(ctx, ZMQ_PUSH)
rc = zmq_connect(sock, 'tcp://localhost:5555')
do i=1, 10**8
res = zmq_send(sock, c_loc(i), size, 0)
end do
! in our simple data exchange `0x0` represents end of stream
! send close sgnal to `recv`
i = 0
res = zmq_send(sock, c_loc(i), size, 0)
rc = zmq_close(sock)
rc = zmq_ctx_term(ctx)
end program main
and the receiver side:
int main() {
h5::fd_t fd = h5::create("collected-data.h5", H5F_ACC_TRUNC);
void *ctx = zmq_ctx_new ();
void *sock = zmq_socket (ctx, ZMQ_PULL);
int rc = zmq_bind (sock, "tcp://*:5555");
// we're roughly controlling IO caches with chunk size, technically you want it about
// to be the underlying buffer size: 1MB for jumbo ethernet frames, 64Kb for low latency interconnects
h5::pt_t pt = h5::create<int64_t>(fd, "some channel xyz",
h5::max_dims{H5S_UNLIMITED}, h5::chunk{1024});
int64_t buffer=1;
do
if( zmq_recv (sock, &buffer, sizeof(int64_t), 0) >= 0)
h5::append(pt, buffer);
while(buffer); // `0x0` terminates transmission
}
The project maybe downloaded from this github link
)
