HSDS on prem via docker

HSDS
1

Hi there,

I’m trying to get HSDS on premise to run, and since I haven’t seen a lot of feedback I wanted to share my troubles. I’m running docker on an LXC container but I can always go to a full VM.

To get and build HSDS, I followed this guide on Github
Docker_install_posix

  • Could be a bit more detailed, in the guide for python install the commands for Environment Variables are given ('export = ')

Go to http://hsds.hdf.test/about and verify that “cluster_state” is “READY” (might need to give it a minute or two)

Checking ‘docker ps’ I could find out that none of the containers had a port mapped to 80, so I had to figure out which containers port might work (dn_1 and sn_1 returned some information on the /about under ports 49161 or 5101). The guide should either tell us that or mention where to check the ports.

{"start_time": 1643738699, "state": "READY", "hsds_version": "0.7.0beta", "name": "Highly Scalable Data Service (HSDS)", "greeting": "Welcome to HSDS!", "about": "HSDS is a webservice for HDF data", "node_count": 1, "dn_urls": ["http://172.21.0.5:6101"], "dn_ids": ["dn-aa3c2"], "username": "anonymous", "isadmin": false}
It only says “state”, not cluster state? So something could have gone wrong here already.

But that was the easy part.
Post-Install steps following this guide.
Post Install

Setting up the environment with pip hdf5 h5py and h5pyd (I used another VM with ubuntu and anaconda for this)

  1. Run $ hsconfigure. Answer prompts for HSDS endpoint, username, and password. These will become the defaults for hs commands and the h5pyd package

I could specify the endpoint (dns / port the docker listens on), user and password were specified in the passwd.txt, but then it asked for API KEY, no mention of that until now, so I left it empty.

Using the port 49161 - “Couldn’t connect to server” :-/
Using the port 5101 - “Unauthorized, username/password or api key not valid”
I tried the admin and normal user I specified, but both didnt work.

And that is where my adventure ended (for now). I could test some steps further down the guide, but I guess I’m missing crucial information already…
Please enlighten me :slight_smile:

Edit:
I’ve gone and redone all the steps on a fresh conda environment on an Ubuntu VM - got the same unauthorized message

Then I looked at the other docs for example the one on dockerhub which suggested I could ignore some “no access”…

I managed to continue the post_install …

Edit 2:
FOUND THE MAIN PROBLEM
I used a password that contains a ‘$’, cause… well I used it in all kinds of “need a quick pass that everybody involved can remember, but is still kind of safe”-situations before…
And most commands that are in the guide just go

-u user -p without any

without any " or ’ … and probably soime of the testcode (hsconfigure?`) is neither?

2

Hi Martin,

Thanks for your feedback! We are working to make the getting started guide as fool proof as possible, but it can be a challenge. Sometimes a code change or new deployment type will require an update in the documentation, but that can be missed.

You are correct that the port should be 5101 and that it’s the “ready” property not the “cluster_state” property that is returned from /about. So from what we can see so far, it looks like the server is up and running.

The password with ‘$’ causes some problem when used at the shell prompt. It should be ok if you put quotes around the password. After you run hsconfigure, check the contents of the .hscfg file. You can just edit it if you need to fixup anything. It should look something like this:

# Any line starting with a pound sign is a comment
hs_endpoint = http://hsds.hdf.test:5101
hs_username = martin
hs_password = $1234
hs_api_key = None

Don’t use quotes in the .hscfg file. (The server will think the quotes are part of the password or username value)

I’ve updated the posix setup and post install guides based on your feedback. Let us know how things go with the setup!