Hello
I succeeded to read it with HDF.Pinvoke library
but it needs 15 lines of code to extract a simple string in H5 files !!
Ty
Hi @benoitbeausir1,
You can have your code simpler and less verbose through the usage of HDFql, which supports P/Invoke. Example in C#:
// read dataset 'my_string'
HDFql.Execute("SELECT FROM my_string");
// set HDFql cursor to first element
HDFql.CursorFirst();
// get cursor element as a string and assign it to variable 'str'
string str = HDFql.CursorGetChar();
// print value of variable 'str'
System.Console.WriteLine("The string value is " + str);
Hope it helps!
hello thank you
I have tried to use HDFql, unfortunatly I could not import it in my VS project.
when I tried to import in my references it says the dll could not be added, verify it is a valid component… (sorry I translate from french)
also sorry I am not informaticien, I am not aware or familiar with all that
So finaly I had to create my own “CursorGetChar” function.
the thing was I thought h5 format was much more easy to read
Hi @benoitbeausir1,
Thanks for the screenshot.
It seems that you are trying to use HDFql library as an assembly or COM component. Instead, HDFql should be used as a normal shared library by your program - please see this post on this forum as well as this post on Stack Overflow for additional information.
Hope it helps!
@benoitbeausir1, hdf5dotnet has been deprecated long ago in favor of HDF.PInvoke and community libraries. I created the HDF.PInvoke.1.10 library (https://www.nuget.org/packages/HDF.PInvoke.1.10) to ease the usage of HDF.PInvoke in .NET. But using the C API directly is always a bit cumbersome as you have already experienced. You could try using HDF5-CSharp (https://www.nuget.org/packages/HDF5-CSharp) or the read-only PureHDF package (https://www.nuget.org/packages/PureHDF) which both offer a high level API to work with HDF5 files.
thank you apollo3zehn-h5
I will try that next time I need to read H5 files, I passed about 4 days to create my functions to extract arrays, string, single… and up to now it seems sufficient for what I need
And also thank you for the HDF.PInvoke
1 Like
Hello mike
you are right, I cannot read the unprocessed patterns. it returns zeros.
Do you know if there is a parameter in the H5 read function, or something to say that the data are LZH compressed ? the compression is automatically detected, I don’t find any help.
Benoit
Not sure. I have been working/helping a few other individuals who are trying to read those files and they ran into the same problem. They were trying to compile HDF5 with all of the necessary plugins so that they could read the data from the .h5oina file.
I was going to add support into DREAM.3D and EBSDLib to read H5OINA files but this compression part has stopped me attempting to add the readers.
h5dump -Hp should give you information about the compression and its parameters even when compression library is not available. Also, you may try to use h5debug or l5ls (see section 3.4.2 in this document) to troubleshoot.
Elena
1 Like
There are APIs to get information about a filter. One can use H5Pget_nfilters to get number of filters and then iterate over the number of filters using H5Pget_filter.
yes I also stopped. Impossible to know how to read that. pfffff
A simple binary file containing the patterns would be much much easier to read.
hello Elena, thank you for your reply. I know it is LZF compression. what I don’t know is the function or whatever to read it (and uncompressed it). I use HDF5.Pinvoke and I program in VB.NET
I forgot to add that to use the filter function you should have LZF compression library installed on your system.
The easiest way to bypass the plugin problem (as a quick hack) is to register the filter function with your application. For example, see section 5 in this document. Compile and link the filter function and the LZF library with your application. I am not sure from where to get the LZF library, but may be tar file from the THG GitHub plugins repo will be useful.
I quickly read waht you send, I will try next week thank you
I have zlib.dll and szip.dll
and I read the dataset like that
H5D.read(dsID, typeID, H5S.ALL, H5S.ALL, H5P.DEFAULT, pinnedArray.AddrOfPinnedObject())
what is not clear for me is how and when the data are uncompressed ?
Information about the applied filter(s) is stored in the header of a dataset, therefore HDF5 library knows which filter(s) to apply on write and read operations. HDF5 will search for the corresponding filter function and use it to encode/decode the data.
There are three mechanism in HDF5 to invoke filters: via plugins (the filter is loaded at run time), built-in with HDF5 (e.g., gzip, szip, n-bit, scale+offset), and built-in with HDF5 app (I pointed you to this one as the simplest to implement and debug). In all those cases decoding is applied automatically, so your current read call should work without modifications.
OK I see, ty. I currently use HDF5.Pinvoke, I just have downloaded hdf5-1.14.0-Std-win10_64-vs16.
For HDF5.Pinvoke I have
HDF.Pinvoke.dll (the one I import in my project), along with I have hdf5.dll, hdf5_hl.dll, zlid and szip.dll
When I read the array compressed I have no errors and the size seems also correct, but the array contain only zeros.
I installed hdf5-1.14.0-Std-win10_64-vs16 and I see in plugin directory libh5lzf.dll but I cannot include as a reference, should I do something with this ? Now I am quite lost with all the dll versions…
ty for your patience and support
Agree… It is confusing…But if everything is installed correctly, it should work without any work on your part 
I am on macOS system and downloaded hdf5-1.14.0-Std-macos11_64-clang.tar.gz to check binary packaging. It looks like that libh5lzf.so (equivalent to Windows libh5lzf.dll) contains the filter function and the LZF library. If your HDF5_PLUGIN_PATH environment variable points to the plugin directory, it should work.
I would try a few things to troubleshoot the problem:
-
Make sure you are using HDF5.Pinvoke based on HDF5 1.14.0
-
Try
h5dumpfrom this installation to dump the content of the file and see if it prints data. Use--enable-error-stackoption for h5dump to catch any errors. If the tool cannot see theplugindirectory then it will print an error message. If it prints zeros without displaying an error, I would suspect that the dataset was not written at all. When you use-poption, check theSIZEvalue for the LZF compressed dataset’s output. It should not be 0 if data was written. -
Try a simple C example (see section 3.1 in this document, and use 32000 for the filter ID to set LZF filter in the H5Pset_filter function. You should be able to see a filter parameter used for your file from the
h5dump -pHoutput. Also, section 3.2 shows the error you should be getting on read if the filter is not available.
Where I can get the file in questions? I will try it on my system in the next few days.
Elena