How to treat a VL data type

Hi all,
   I am not very experienced with HDF5 and I need some advice for the
following problem.

Consider the fortran code

type :: t_data
integer :: i
real, allocatable :: vals(:slight_smile:
end type t_data

type(t_data) :: data(100) ! the array which I want to save

where the vals component has a different dimension for each element:

allocate( data(1)%vals(3) )
allocate( data(2)%vals(1) )
allocate( data(3)%vals(0) )
allocate( data(4)%vals(10) )
...

How could I save the array data in an HDF5 file? I read that there are
"ragged arrays" as well as "fractal arrays", but I can't find any
example about how to use them. Can anybody give my any reference?

Thank you!
Marco

Hi Marco,

I think the example program h5ex_t_vlen_F03.f90 at,

https://www.hdfgroup.org/HDF5/examples/api18-fortran.html

shows how to handle this.

Scot

···

On Jun 9, 2015, at 5:42 AM, marco restelli <mrestelli@gmail.com<mailto:mrestelli@gmail.com>> wrote:

Hi all,
  I am not very experienced with HDF5 and I need some advice for the
following problem.

Consider the fortran code

type :: t_data
integer :: i
real, allocatable :: vals(:slight_smile:
end type t_data

type(t_data) :: data(100) ! the array which I want to save

where the vals component has a different dimension for each element:

allocate( data(1)%vals(3) )
allocate( data(2)%vals(1) )
allocate( data(3)%vals(0) )
allocate( data(4)%vals(10) )
...

How could I save the array data in an HDF5 file? I read that there are
"ragged arrays" as well as "fractal arrays", but I can't find any
example about how to use them. Can anybody give my any reference?

Thank you!
Marco

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

Hi Scot,
   thank you for the pointer!

I see from the example how to write a ragged array, when I try to use
the same idea for my derived type I am not sure how to handle the
various fields. Elaborating on my previous example, let us assume for
instance

type :: t_data
  logical :: l
  integer :: i
  real, allocatable :: vals(:slight_smile:
  complex, allocatable :: xyz(:,:slight_smile:
end type t_data

type(t_data) :: data(100) ! the array which I want to save

One possibility would be treating each field separately, creating four
arrays, which in this specific case would be two ordinary arrays and
two ragged arrays. Unless I am missing something, this could be done
as in the example.

However, I would prefer writing one single HDF5 array keeping the same
structure as in the fortran derived type. Is it possible?

My main concern about isolating each field in a dedicated array is
that it seems to become awkward when derived data types are nested,
such as

type :: t_otherdata
  integer :: n
  type(t_data) :: loc_data
end tyep t_otherdata

type(t_otherdata) :: odata(1000) ! array to be saved

since I should extract all the components of the nested structures. In
practice, I often have ~5 or more nesting levels, and I would like to
be able to treat each of them independently.

Marco

···

2015-06-09 16:36 GMT+0200, Scot Breitenfeld <brtnfld@hdfgroup.org>:

Hi Marco,

I think the example program h5ex_t_vlen_F03.f90 at,

https://www.hdfgroup.org/HDF5/examples/api18-fortran.html

shows how to handle this.

Scot

On Jun 9, 2015, at 5:42 AM, marco restelli > <mrestelli@gmail.com<mailto:mrestelli@gmail.com>> wrote:

Hi all,
  I am not very experienced with HDF5 and I need some advice for the
following problem.

Consider the fortran code

type :: t_data
integer :: i
real, allocatable :: vals(:slight_smile:
end type t_data

type(t_data) :: data(100) ! the array which I want to save

where the vals component has a different dimension for each element:

allocate( data(1)%vals(3) )
allocate( data(2)%vals(1) )
allocate( data(3)%vals(0) )
allocate( data(4)%vals(10) )
...

How could I save the array data in an HDF5 file? I read that there are
"ragged arrays" as well as "fractal arrays", but I can't find any
example about how to use them. Can anybody give my any reference?

Thank you!
Marco

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

Hi Scot,

I am attaching a fortran test program showing what we are doing now
with ASCII output. I am including a few modules so that the program
can be compiled and run, but the only relevant part for the discussion
here is test.f90 and specifically the three subroutines
write_t_a_vec, write_t_b_vec and write_t_c_vec.

Ideally, I would like to translate this code to have HDF5 output
instead of ASCII, using the HDF5 library instead of the included
mod_octave_io. Of course, I understand that this might not be so easy!
For instance, maybe one would need more than one subroutine for each
data type, one to (recursively) collect beforehand the size of all the
nested components, and then a second pass to actually write the data.

The way I see it, everything is fine as long as each type can be dealt
with on its own, so that the overall complexity scales linearly with
the number of nested types. With this test code, this is the case: for
instance, if we decide to add a new field in t_a, the lowest type in
the nesting chain, we only have to change the corresponding subroutine
write_t_a_vec.

Well, I hope this clarifies my problem, if not please let me know if I
can provide additional information. In the meanwhile, I will also look
at nested_derived_type.f90, thank you a lot!

Marco

HDF5-test.tar.gz (17.4 KB)

···

2015-06-10 16:38 GMT+0200, Scot Breitenfeld <brtnfld@hdfgroup.org>:

On Jun 9, 2015, at 3:12 PM, marco restelli <mrestelli@gmail.com> wrote:

Hi Scot,
  thank you for the pointer!

I see from the example how to write a ragged array, when I try to use
the same idea for my derived type I am not sure how to handle the
various fields. Elaborating on my previous example, let us assume for
instance

type :: t_data
logical :: l
integer :: i
real, allocatable :: vals(:slight_smile:
complex, allocatable :: xyz(:,:slight_smile:
end type t_data

type(t_data) :: data(100) ! the array which I want to save

One possibility would be treating each field separately, creating four
arrays, which in this specific case would be two ordinary arrays and
two ragged arrays. Unless I am missing something, this could be done
as in the example.

However, I would prefer writing one single HDF5 array keeping the same
structure as in the fortran derived type. Is it possible?

In theory this should be possible, but we don’t have a test or an example
for doing this. We have a nested
example in the HDF5 source (fortran/examples/nested_derived_type.f90) but we
don’t have an
example of a VL type with nested data types. This would be a good example to
have.

Can you provide example code that mimics your code for the nested derived
type and VL length that you would like to write so that I make sure I
understand exactly what you are wanting to do? I’m thinking at most 3
levels
of nests would suffice for an example.