I'm a new user of h5 server. On my test server, I installed and ran the h5serv docker container. From a remote browser, I could get response by sending http://128.XXX.XXX.XXX:5000/ to the server. However, when I tried to use curl-based request, such as "curl --header "Host: dp.h5_store.hdfgroup.org" http://128.XXX. XXX. XXX:5000/", an error "tornado.web.HTTPError: HTTP 404: Not Found" occurred. The h5 files I want the server to serve is under "/data/h5_store". Because the h5 server is installed on a machine without domain name (e.g. hdfgroup.org), how could I set up a valid HTTP header so that the curl command could be used to send the request?
For curl, always use the dns name of the server the container is running on, i.e. the http://128… address you have.
To address files, the pseudo-DNS name is created by taking the “domain” config value (in h5serv/server/config.py), and combining with the reverse path of the file relative to the data directory. There’s another config value: “datapath” that gives the location to the data directory. So in your case you should set this to: /data/h5_store. After this (and restarting the server), if you have “myfile.h5” in the h5_store directory, the h5serv host path to that file would be: myfile.hdfgroup.org.
I’m a new user of h5 server. On my test server, I installed and ran the h5serv docker container. From a remote browser, I could get response by sending http://128.XXX.XXX.XXX:5000/ to the server. However, when I tried to use curl-based request, such as “curl --header "Host: dp.h5_store.hdfgroup.org" http://128.XXX. XXX. XXX:5000/”, an error “tornado.web.HTTPError: HTTP 404: Not Found” occurred. The h5 files I want the server to serve is under “/data/h5_store”. Because the h5 server is installed on a machine without domain name (e.g. hdfgroup.org), how could I set up a valid HTTP header so that the curl command could be used to send the request?
I've just discovered that when I create a dataset with one dimension which has a size = 0, then I cannot properly open such dataset - H5Dopen() returns -1.
But when I create similar dataset with one dimension which has a size at least 1 (even without writing data to such dataset), then I can properly open such dataset.
Is this a bug or a feature in the HDF5 library?
I think I should be able to open 0-size dimension dataset, just to create an attribute inside such dataset - and that was my intension - no data inside (thus dimension size = 0), but write some attributes to such dataset...
Rafal, this might be a bug. However, if you don't intend to store
data there's a cheaper and maybe less contorted option: You can
create a dataset with a dataspace of class H5S_NULL.
Think of that as a dataset whose dataspace is an empty set.
(Likewise you'd use H5S_SCALAR for a singleton.)
Best, G.
···
-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Rafal Lichwala
Sent: Thursday, October 26, 2017 3:00 AM
To: hdf-forum@lists.hdfgroup.org
Subject: [Hdf-forum] HDF5 library bug or feature?
Hi All,
I've just discovered that when I create a dataset with one dimension which has a size = 0, then I cannot properly open such dataset -
H5Dopen() returns -1.
But when I create similar dataset with one dimension which has a size at least 1 (even without writing data to such dataset), then I can properly open such dataset.
Is this a bug or a feature in the HDF5 library?
I think I should be able to open 0-size dimension dataset, just to create an attribute inside such dataset - and that was my intension - no data inside (thus dimension size = 0), but write some attributes to such dataset...
However, if you don't intend to store
data there's a cheaper and maybe less contorted option: You can
create a dataset with a dataspace of class H5S_NULL.
Good point - I will remember about that possibility in the future - thank you.
For now I've just moved all attributes level up - to the parent group - works fine.
Rafal, this might be a bug. However, if you don't intend to store
data there's a cheaper and maybe less contorted option: You can
create a dataset with a dataspace of class H5S_NULL.
Think of that as a dataset whose dataspace is an empty set.
(Likewise you'd use H5S_SCALAR for a singleton.)
A common desire is to store non-uniform sparse data in HDF5 (consider XML attributes as example). I tend to use empty groups which only serve as container of attributes. What do you think: would empty datasets (H5S_NULL) with attributes be more efficient (introduce less overhead)?
Best wishes,
Andrey Paramonov
···
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
Andrey, the experts should chime in on the storage trade-offs.
(I imagine the null dataset is stored with compact layout, but in
newer versions of the file format an empty group is very "compact"
as well.) At the conceptual level, the two are very different and
that might tip the balance in either direction.
A group leaves the door open for other descendants.
A dataset is a terminal node and, despite being empty, is nevertheless
typed. (A null dataset has an element type!) Of course, you could
"type" a group with additional attributes, but it would be a very
different kind of variable.
This might be a case where there're no right or wrong choices
as far as HDF5 is concerned, and users will have to decide what
makes it right for them.
Best, G.
···
-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of ??? ???
Sent: Thursday, October 26, 2017 6:52 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] HDF5 library bug or feature?
Hi Gerd!
26.10.2017 14:31, Gerd Heber пишет:
Rafal, this might be a bug. However, if you don't intend to store data
there's a cheaper and maybe less contorted option: You can create a
dataset with a dataspace of class H5S_NULL.
Think of that as a dataset whose dataspace is an empty set.
(Likewise you'd use H5S_SCALAR for a singleton.)
A common desire is to store non-uniform sparse data in HDF5 (consider XML attributes as example). I tend to use empty groups which only serve as container of attributes. What do you think: would empty datasets
(H5S_NULL) with attributes be more efficient (introduce less overhead)?
Best wishes,
Andrey Paramonov
--
This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
A choice consideration might be that the number of attributes has a
limit.
I think HDF5 should support empty N-dim datasets. Being empty also
conveys information.
Note that in any language it would be strange if you could not have an
empty sequence.
Andrey, the experts should chime in on the storage trade-offs.
(I imagine the null dataset is stored with compact layout, but in
newer versions of the file format an empty group is very "compact"
as well.) At the conceptual level, the two are very different and
that might tip the balance in either direction.
A group leaves the door open for other descendants.
A dataset is a terminal node and, despite being empty, is nevertheless
typed. (A null dataset has an element type!) Of course, you could
"type" a group with additional attributes, but it would be a very
different kind of variable.
This might be a case where there're no right or wrong choices
as far as HDF5 is concerned, and users will have to decide what
makes it right for them.
Best, G.
···
-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf
Of ??? ???
Sent: Thursday, October 26, 2017 6:52 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] HDF5 library bug or feature?
Hi Gerd!
26.10.2017 14:31, Gerd Heber пишет:
Rafal, this might be a bug. However, if you don't intend to store data
there's a cheaper and maybe less contorted option: You can create a
dataset with a dataspace of class H5S_NULL.
Think of that as a dataset whose dataspace is an empty set.
(Likewise you'd use H5S_SCALAR for a singleton.)
A common desire is to store non-uniform sparse data in HDF5 (consider
XML attributes as example). I tend to use empty groups which only serve
as container of attributes. What do you think: would empty datasets
(H5S_NULL) with attributes be more efficient (introduce less overhead)?
Best wishes,
Andrey Paramonov
--
This message has been scanned for viruses and dangerous content by
MailScanner, and is believed to be clean.
I am able to create a dataset with zero dimensions, close it, and re-open. (See attached).
Can you send us an example of what you are trying that fails?
Please send it to the helpdesk: help@hdfgroup.org
-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Gerd Heber
Sent: Thursday, October 26, 2017 6:49 AM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] HDF5 library bug or feature?
Rafal, this might be a bug.
Is there any way to record/track this issue?
Not sure. Helpdesk will check. (It's still early morning here...)
A choice consideration might be that the number of attributes has a limit.
Hmm, do I understand correctly that this limit is different for a group and (an empty) dataset?
Best wishes,
Andrey Paramonov
···
I think HDF5 should support empty N-dim datasets. Being empty also conveys information.
Note that in any language it would be strange if you could not have an empty sequence.
Andrey, the experts should chime in on the storage trade-offs.
(I imagine the null dataset is stored with compact layout, but in
newer versions of the file format an empty group is very "compact"
as well.) At the conceptual level, the two are very different and
that might tip the balance in either direction.
A group leaves the door open for other descendants.
A dataset is a terminal node and, despite being empty, is nevertheless
typed. (A null dataset has an element type!) Of course, you could
"type" a group with additional attributes, but it would be a very
different kind of variable.
This might be a case where there're no right or wrong choices
as far as HDF5 is concerned, and users will have to decide what
makes it right for them.
26.10.2017 14:31, Gerd Heber пишет:
> Rafal, this might be a bug. However, if you don't intend to store data
> there's a cheaper and maybe less contorted option: You can create a
> dataset with a dataspace of class H5S_NULL.
> Think of that as a dataset whose dataspace is an empty set.
> (Likewise you'd use H5S_SCALAR for a singleton.)
A common desire is to store non-uniform sparse data in HDF5 (consider XML attributes as example). I tend to use empty groups which only serve as container of attributes. What do you think: would empty datasets
(H5S_NULL) with attributes be more efficient (introduce less overhead)?
Best wishes,
Andrey Paramonov
--
This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Thank you for an answer!
Yes, your code also works fine on my side, BUT...
when I try to set some custom H5P_FILE_ACCESS by call:
H5Pset_fapl_mpio(plist_id, comm, info)
(so running this example under OpenMPI - even for 1 process only)
it DOES NOT work any more.
In that case (see attached slightly modified your code) I've got the following output:
2: H5Pcreate: OK
2: H5Pset_fapl_mpio: OK
2: H5Fcreate: OK
2: H5Screate_simple: OK
2: H5Dcreate2: OK
2: H5Dclose: OK
2: H5Sclose: OK
2: H5Pclose: OK
2: H5Fclose: OK
2: H5Pcreate: OK
2: H5Pset_fapl_mpio: OK
2: H5Fopen: OK
2: H5Dopen2 Failed!
2: H5Dget_space Failed!
2: H5Dclose Failed!
2: H5Sclose Failed!
2: H5Pclose: OK
2: H5Fclose: OK
HDF5 library also reports some error messages in that case:
unable to open dataset
unable to initialize file storage
unable to mark dataspace as dirty
unable to update layout info
unable to write object header message
unable to modify constant message
The most interesting thing is (which possibly means it's a bug), that when in the same code I just change dims definition to:
I am able to create a dataset with zero dimensions, close it, and re-open. (See attached).
Can you send us an example of what you are trying that fails?
Please send it to the helpdesk: help@hdfgroup.org
Thanks!
-Barbara
help@hdfgroup.org
-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Gerd Heber
Sent: Thursday, October 26, 2017 6:49 AM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] HDF5 library bug or feature?
Rafal, this might be a bug.
Is there any way to record/track this issue?
Not sure. Helpdesk will check. (It's still early morning here...)
Thank you for an answer!
Yes, your code also works fine on my side, BUT...
when I try to set some custom H5P_FILE_ACCESS by call:
H5Pset_fapl_mpio(plist_id, comm, info)
(so running this example under OpenMPI - even for 1 process only) it DOES NOT work any more.
In that case (see attached slightly modified your code) I've got the following output:
2: H5Pcreate: OK
2: H5Pset_fapl_mpio: OK
2: H5Fcreate: OK
2: H5Screate_simple: OK
2: H5Dcreate2: OK
2: H5Dclose: OK
2: H5Sclose: OK
2: H5Pclose: OK
2: H5Fclose: OK
2: H5Pcreate: OK
2: H5Pset_fapl_mpio: OK
2: H5Fopen: OK
2: H5Dopen2 Failed!
2: H5Dget_space Failed!
2: H5Dclose Failed!
2: H5Sclose Failed!
2: H5Pclose: OK
2: H5Fclose: OK
HDF5 library also reports some error messages in that case:
unable to open dataset
unable to initialize file storage
unable to mark dataspace as dirty
unable to update layout info
unable to write object header message
unable to modify constant message
The most interesting thing is (which possibly means it's a bug), that when in the same code I just change dims definition to:
dims[0] = 1;
dims[1] = 1;
everything WORKS FINE now!
Best regards,
Rafal
W dniu 2017-10-26 o 17:53, Barbara Jones pisze:
Hi Rafal,
I am able to create a dataset with zero dimensions, close it, and re-open. (See attached).
Can you send us an example of what you are trying that fails?
Please send it to the helpdesk: help@hdfgroup.org
-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On
Behalf Of Gerd Heber
Sent: Thursday, October 26, 2017 6:49 AM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] HDF5 library bug or feature?
Rafal, this might be a bug.
Is there any way to record/track this issue?
Not sure. Helpdesk will check. (It's still early morning here...)
