dataset with absolute addresses

Hdf-forum archives
1

Hi all,

I'm trying to generate HDF5 bindings from a Matlab Simulink/RTW C API.
Their API has several datastructures with the names, addresses, and types
of variables in memory.

Since their API exposes absolute addresses in memory, I thought I might
tell HDF5 about "the object starting at address 0". The essence of my
code follows.

hid_t memtype=H5Tcreate(H5T_COMPOUND, SIZE_MAX);
for field
  H5Tinsert(memtype, name, (size_t)(address), htype);
...
H5Dwrite(dataset, memtype, memspace, filespace, H5P_DEFAULT, 0);

This fails with a runtime error.
HDF5-DIAG: Error detected in HDF5 (1.8.6) thread 0:
  #000: H5Dio.c line 255 in H5Dwrite(): no output buffer
    major: Invalid arguments to routine
    minor: Bad value

Inspecting the code shows somebody put a check for my 0 to protect a file
transport I don't use... So try a different small number. New code.

hid_t memtype=H5Tcreate(H5T_COMPOUND, SIZE_MAX-4);
for field
  H5Tinsert(memtype, name, (size_t)(((char *)address)-4), htype);
...
H5Dwrite(dataset, memtype, memspace, filespace, H5P_DEFAULT, 4);

This segfaults deep in a memcpy in H5D_gather_mem. Yeah, can't memcpy the
whole memory space...

Any suggestions on how to describe a dataset when you only know absolute
addresses that may cover a wide chunk of memory? Do I really have to do
my own copies to a smaller data structure?

Thanks,
Daniel

2

I think this would qualify as abuse of the HDF API :wink: And obviously it
doesn't work so well.

The Compound type in HDF is defined specifically to handle reading C-style
structs. You build up the compound type by inserting compound members,
based off the address into the structure. Then you write structures into
the HDFCompound by passing the address of an instance of the structure.
Conversely, if you read the compound from an HDF file, you pass a pointer
to a structure and it copies from the disk into the structure.

This involves some intelligent processing inside the HDF5 library -- when
you read out the file, it converts from the on-disk type to the
corresponding native type for each field. And since you build up the
compound description to read with at runtime, it can handle differences
between platforms. In other words, you can write on a 32-bit platform and
read on 64-bit (or vice versa) even though structure sizes & offsets would
be different.

You have 2 options: 1. Copy to a struct mirroring the Compound type you
want to define 2. Abandon Compound, and write out a series of Datasets in
a Group.

Which choice to use depends on the size of the data. For a smaller set of
data, choice #1 is fine, as memory use and copying time will be minimal.
For a collection of large data objects, #2 is probably better. Choice #2
also allows -- in effect -- for variable-sized data collections -- since
you write/read a series of datasets, and access them by path string, they
can be any size you want.

···

On 4/11/11 5:20 PM, "dherring@tentpost.com" <dherring@tentpost.com> wrote:

Hi all,

I'm trying to generate HDF5 bindings from a Matlab Simulink/RTW C API.
Their API has several datastructures with the names, addresses, and types
of variables in memory.

Since their API exposes absolute addresses in memory, I thought I might
tell HDF5 about "the object starting at address 0". The essence of my
code follows.

hid_t memtype=H5Tcreate(H5T_COMPOUND, SIZE_MAX);
for field
H5Tinsert(memtype, name, (size_t)(address), htype);
...
H5Dwrite(dataset, memtype, memspace, filespace, H5P_DEFAULT, 0);

This fails with a runtime error.
HDF5-DIAG: Error detected in HDF5 (1.8.6) thread 0:
#000: H5Dio.c line 255 in H5Dwrite(): no output buffer
   major: Invalid arguments to routine
   minor: Bad value

Inspecting the code shows somebody put a check for my 0 to protect a file
transport I don't use... So try a different small number. New code.

hid_t memtype=H5Tcreate(H5T_COMPOUND, SIZE_MAX-4);
for field
H5Tinsert(memtype, name, (size_t)(((char *)address)-4), htype);
...
H5Dwrite(dataset, memtype, memspace, filespace, H5P_DEFAULT, 4);

This segfaults deep in a memcpy in H5D_gather_mem. Yeah, can't memcpy the
whole memory space...

Any suggestions on how to describe a dataset when you only know absolute
addresses that may cover a wide chunk of memory? Do I really have to do
my own copies to a smaller data structure?

Thanks,
Daniel

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

________________________________
Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error, then delete it. Thank you.
________________________________