H5pyd authentication issue with Keycloak

HDF5 Library
1

Hi,
I am encountering issues when I try to use Keycloak for authentication. I am using h5pyd library to interface with the HSDS server.
I created the .hscfg file in my home dir as indicated in the installation procedure hsds/docs/keycloak_setup.md at master · HDFGroup/hsds · GitHub

But when I try to use commands like hsls, I am getting the error below:

Traceback (most recent call last):
File “/mnt/c/Users/User/Desktop/scripts/venv/bin/hsls”, line 5, in
from h5pyd._apps.hsls import main
File “/mnt/c/Users/User/Desktop/scripts/venv/lib/python3.12/site-packages/h5pyd/_apps/hsls.py”, line 17, in
cfg = Config()
^^^^^^^^
File “/mnt/c/Users/User/Desktop/scripts/venv/lib/python3.12/site-packages/h5pyd/_apps/config.py”, line 130, in init
raise ValueError(f"undefined option: {name}")
ValueError: undefined option: ignore

For the HSDS configuration, I created the override.yml file in hsds/admin/config. The configuration seems correct since when I launch runall.sh command, the containers start as usual.
Also, when I am using the default authentication method which is to use a passwd.txt file, there is no issue.

Below are the versions of the packages I am using:

  • Python 3.12.3
  • HSDS v0.8.4
  • Docker version 26.1.3, build 26.1.3-0ubuntu1~24.04.1
  • docker-compose version 1.29.2, build unknown
  • keycloak-15.0.2
  • h5pyd 0.18.0

I launch the HSDS server from WSL2 on Windows 11 (dist: Ubuntu 24.04.1 LTS)
The storage is Azure Blob Storage.

Thank you for your help.

2

Thanks for the detailed issue report!
I haven’t tested with Keycloak recently, but I’ll give it a try and get back to you.

BTW, how do you like WSL2? I was thinking of switching my Linux laptop to Windows and using WSL2 to cover testing with Linux.

3

After a bit of time re-acquainting myself with Keycloak, I have some updates to HSDS and h5pyd that should fix the issue.

You’ll need to get the latest HSDS on master and rebuild (or fetch the latest hsds:master from DockerHub). Also you need to fetch and build h5pyd from master (or do a pip install git+https://github.com/HDFGroup/h5pyd).

Finally, note that the openidurl has been updated in: hsds/docs/keycloak_setup.md at master · HDFGroup/hsds · GitHub.

One thing to be aware of is that if you are running HSDS and Keycloak in Docker, HSDS won’t be able by default to access the Keycloak url. I got around this by running Keycloak in Docker and HSDS with the ./runall.sh --no-docker-tcp option.

Anyway, let us know if these updates work for you.

4

Hi, I have just tested to run HSDS without docker as you said and so far it seems to work fine! I will let you know if I come across other problems
Thank you!

5

Great!

FYI - if you do want to run HSDS in Docker, you’ll need to either use a KeyCloak endpoint that’s accessible via DNS, or modify your docker compose to include a KeyCloak container. It’s just Docker doesn’t allow to access other containers that aren’t part of the same docker-compose yaml.