this has been an interesting discussion about cache, but I am going to change the subject a little bit with some information regarding HDF5 Dimension Scales (DS), that we just released, that I thought it might be useful for everybody.
We received some DS usage questions from users from the Royal Netherlands Meteorological Institute (KNMI), which I reproduce below, with their kind permission.
I prepared a follow up that I will send soon, but this email has the original question
SCI_IMLM_lv2_20030911_08001_2741_v6.4.h5 (166 KB)
test_ds6.h5 (144 KB)
Date: Fri, 21 Mar 2008 15:54:37 -0500
From: Pedro Vicente Nunes <firstname.lastname@example.org>
Subject: Re: requested feedback on the usage of HDF5 dimension scales and CF metadata conventions (fwd)
Cc: email@example.com, firstname.lastname@example.org, email@example.com, hDF Helpdesk <firstname.lastname@example.org>
Dear Richard and others
Barbara forwarded me your questions, since I implemented the Dimension Scales (DS) API.
By taking a look at your binary file I guess that lat_bnds and lon_bnds contain 4 sets of 1826 latitude or longitude values.
We did not preview this case of having several lat lon sets in one dataset. That would be of course possible to do in the future but in the meantime and
if my assumption that those values are indeed 4 sets of lat and lon, you will have to modify, if possible, the way you save these datasets
For your program to "work" correctly with our DSs you will have to write each set of these 4 sets on a different dataset (that is 4 datasets for lat and 4 for lon) , and then "link" each one of them with your data dataset
You are right that there is no user's manual for this. I was hoping to write such a guide in the near future by this was delayed for a couple months. It will be done tough.
In the meantime we only have the reference manual
and a test program for the API in the hdf5 distribution under
by looking at this C program you'll probably have a guess about the "correct" way of using it.
Amongst other things, the program reads and writes realistic lat and lon data from the North Atlantic
(I used this data from an earlier position I had in a numeric hydrodinamics institute, IST , in Portugal)
Also, do download this HDF5 vizualization program called HDF Explorer (if you happen to use Windows)
go the left menu and click download
it handles dimension scales , you can add several and choose the ones you want to be displayed in the map
here's a snapshot and the HDF5 file that test program produces with the Atlantic data is attached
let us know if you have more questions and thanks for using the DS API
what are the CF metadata conventions ?
---------- Forwarded message ----------
Date: Thu, 20 Mar 2008 12:10:03 +0100
From: Richard van Hees <R.M.van.Hees@sron.nl>
To: email@example.com, firstname.lastname@example.org, email@example.com
Cc: John van de Vegte <firstname.lastname@example.org>
Subject: requested feedback on the usage of HDF5 dimension scales and CF
Dear Olga, Ben and HDF5 helpdesk,
For the project ADAGUC, I try to convert satellite observations written in various formats to a structured HDF5 file using the CF metadata conventions. In addition, I use the newly released HDF5 dimension scale API. Eventually, the goal is to use these HDF5 datasets as input for software to convert it to different formats (netCDF, geoTiff, kml, etc.) for users and applications which are not familiar with HDF5.
What I kindly ask you is to take a look at a small dataset with satellite data: Sciamachy level 2 retrievals of CH4, CO and H2O. The main dimension of the data is time, which is associated with spatial locations, given by (lon,lat) and bounds with 4-sided cells. This is all together a quite complicated case, however, typical for satellite observations.
Dear HDF5 helpdesk, I have tried to use the dimension scale API, however, due to the lack of documentation I am not sure that I use the API correctly. For example, would you suggest me to use the lat_bnds and lon_bnds as dimensions scales attached to the datasets lat and lon?
Any remark is welcome.
Richard van Hees (SRON, the Netherlands)