Hello again, Ahmed,
The h4map displays the offset and length of the data linked-blocks similar to this sample:
<h4:byteStream offset="2905156" nBytes="196608"/>
<h4:byteStream offset="3573137" nBytes="131072"/>
<h4:byteStream offset="4113809" nBytes="131072"/>
It displays the offset and length of some of the data chunks in your file like this:
<h4:byteStream offset="37500" nBytes="35" chunkPositionInArray="[0,0]"/>
<h4:byteStream offset="42377" nBytes="35" chunkPositionInArray="[1,0]"/>
<h4:byteStream offset="88582" nBytes="35" chunkPositionInArray="[2,0]"/>
<h4:byteStream offset="89181" nBytes="35" chunkPositionInArray="[3,0]"/>
Do you think this kind of information is useful in your project?
From: Hdf-forum <email@example.com> on behalf of Ahmed Eldawy <firstname.lastname@example.org>
Sent: Thursday, July 02, 2015 5:52 AM
Subject: Re: [Hdf-forum] Linked block in HDF4 files
I've seen your previous comment about the HDF4 Mapping Tool. However, I couldn't find any instructions on how to install or use it. If the generated XML file looks like this
then I don't think it will be of much help to me.
On Sun, Jun 28, 2015 at 11:50 PM, Ahmed Eldawy <email@example.com<mailto:firstname.lastname@example.org>> wrote:
Hi Binh-Minh Ribler,
Thanks for your response. I've been looking into the issue myself and I think I found the problem. I had to check the source code of the HDF library and I found some useful comments in the file hblocks.c. According to this file, the correct header format is
ext_tag_desc - SPECIAL_LINKED(16 bit constant), identifies this as a linked block description record
elem_tot_len - Length of the entire element(32 bit field)
blk_length - Length of successive data blocks(32 bit field) after first block,
first block is calculated.
num_blk - Number of blocks per block table(32 bit field)
link_ref - Reference number of the first block table(16 bit field)
As I mentioned earlier, the ext_tag_desc is 16-bits only and the constant is SPECIAL_LINKED not EXT_LINKED. In addition, the value first_blk_len is not actually stored in the header. It is calculated based on number of blocks, the size of each block, and the total element length.
After I fixed these issues, the numbers started to make more sense. Do you know how I can reach out to the people who write the documentation so that they can solve these problems? It seems that the documentation is outdated.
On Thu, Jun 18, 2015 at 3:04 PM, Ahmed Eldawy <email@example.com<mailto:firstname.lastname@example.org>> wrote:
To give some background context, I'm working on a Java project that deals with HDF files. I used to use the Java-interface to the native HDF library, however, I had some issues with it and I wanted to use a pure Java library. I didn't find any pure Java HDF libraries, so I decided to make my own. It's going very fine and it fits very well in my project. It's not complete yet, but I'm extending it bit-by-bit. All my work is open source. Currently, it's part of a bigger SpatialHadoop project [https://github.com/aseldawy/spatialhadoop2]. However, I'm planning to ship it is a standalone library which will be available for the community.
Currently, I'm working on the Linked Block feature of HDF files. However, I'm having problems with this file
The file opens fine in HDF viewer, which means that the file is correct. I'm following the HDF specs at
Linked block elements are described in Section 10.3 on page 149 (160 in the PDF). First, there seems to be a problem in the documentation as it says that EXT_LINKED constant is 32-bits. This seems to be error because other extension identifiers are 16-bits (e.g., see SPECIAL_EXT in 10.4).
Anyway, I fixed it myself and worked with a 16-bit identifier. However, I started to see some problems and I don't know if my fix of 32-bits to 16-bits was right or wrong.
The file I mentioned above has an extended data block with tagID 1963 (DFTAG_VS) and reference no 49. The data of this object is located in offset 38215 and length 16 bytes. When I parse this block according to the specs, I get the following numbers which are inconsistent.
Now these numbers do not make sense because if the length of the first block is 4096 and then we have 131072 blocks, each of 16-bytes length, the total length would be 2101248 which is much larger than 57600.
I'm having more troubles when parsing the linked block table but let's resolve this issue first as the later might be related.