500 - An internal server error occurred

HSDS
1

I am using the vol-rest driver to talk to an instance of HSDS running in AWS from some C code. I am able to read datasets that are relatively small (16KB, 2K elements), but when I try to read a dataset that is larger (~0.5MB, ~70K elements), I get the following error:

HDF5 REST VOL-DIAG: Error detected in HDF5 REST VOL (1.0.0) thread 0:
#000: /home/ubuntu/dependencies/vol-rest/src/rest_vol_dataset.c line 592 in RV_dataset_read(): 500 - An internal server error occurred
major: Dataset
minor: Read failed
HDF5-DIAG: Error detected in HDF5 (1.12.0) thread 0:
#000: …/…/hdf5-1.12.0/src/H5Dio.c line 192 in H5Dread(): can’t read data
major: Dataset
minor: Read failed
#001: …/…/hdf5-1.12.0/src/H5VLcallback.c line 2080 in H5VL_dataset_read(): dataset read failed
major: Virtual Object Layer
minor: Read failed
#002: …/…/hdf5-1.12.0/src/H5VLcallback.c line 2046 in H5VL__dataset_read(): dataset read failed
major: Virtual Object Layer
minor: Read failed

I have the following configuration setup for HSDS:
min_chunk_size: 10m
max_chunk_size: 40m
max_request_size: 256m
metadata_mem_cache_size: 128m
chunk_mem_cache_size: 256m

2

Just as an update for Forum readers, it turned out this was due to a bug on the server side for datasets with one-byte types that use the shuffle filter (which ironically is a nop!). Anyway, a fix has been pushed to the HSDS master branch on github.