I took a look at https://www.hdfgroup.org/2020/11/webinar-enabling-multithreading-concurrency-in-hdf5-community-discussion/
It got me looking at the lock implementation more closely.
Specifically made me think for the sec2 file io, to release the lock during those times like python GIL does with c extensions for file io. So there’s a recursive lock in the library though right (using a counter where only values transitioning to 1 or 0 change thread lock state)… so what about this algorithm:
another locking routine - suspend and corresponding restore, parameter is suspend_context
grabs the atomic_lock
asserts this thread is the owner
saves to a struct the lock_count
sets lock_count to 0, performs pthread_cond_signal on cond_var
unlock mutex
restore - takes the context
grabs the atomic_lock, waits with while lock_count on cond_var
sets lock_count back from suspend context
unlock mutex
Anything horrifically broken?
I figure this could result in caches getting updated twice in the case a user touches the same chunks in parallel but I also figure if a user is using different files and different datasets such risks are minimized or non-existent.
Quincey Koziol hang out here?