Error when selecting and writing an element dataset hdf5 1.10.4 [SOLVED]

HDF5 Library
1

Dear all,
I am new to hdf5. I’d like to select an element from a dataset and write a real value in it. To do this, I use the following fortran code (it’s a minimal working example extracted from the main code):

program test
use hdf5
implicit none
integer :: err, cnt
real(8) :: fmd
integer(HID_T) :: file_ifmd, dset_ifmd, dspace_ifmd
integer(HSIZE_T) :: dim_fmd(1)
integer(HSIZE_T), dimension(1) :: dim_buf(1)=1
integer(SIZE_T), parameter :: nel=1
integer(HSSIZE_T) :: fmd_coord(1,nel)

dim_fmd(1) = 54

call h5fcreate_f('fmd.hdf5', H5F_ACC_TRUNC_F, file_ifmd, err)
call h5screate_simple_f(1, dim_fmd, dspace_ifmd, err)
call h5dcreate_f(file_ifmd, 'fmd', H5T_NATIVE_DOUBLE, dspace_ifmd, dset_ifmd, err)
call h5dclose_f(dset_ifmd, err)
call h5sclose_f(dspace_ifmd, err)
call h5fclose_f(file_ifmd,err)

call h5fopen_f('fmd.hdf5', H5F_ACC_RDWR_F, file_ifmd, err)
call h5dopen_f(file_ifmd, 'fmd', dset_ifmd, err)

do cnt = 1, 1 ! just one loop for testing purposes
  fmd_coord(1,nel) = cnt
  call h5sselect_elements_f(dset_ifmd, H5S_SELECT_SET_F, 1, nel, fmd_coord, err)
  fmd = 0.5d0
  call h5dwrite_f(dset_ifmd, H5T_NATIVE_DOUBLE, fmd, dim_buf, err)
end do

call h5dclose_f(dset_ifmd, err)
call h5sclose_f(dspace_ifmd, err)
call h5fclose_f(file_ifmd,err)
end program test

I create the executable on ubuntu 20.04.3 LTS with the latest updates, gcc 9.3.0, HDF5 1.10.4 using the command
h5fc -ffree-line-length-none -Wall -Wextra -Wconversion -pedantic -O -fcheck=all -g -fbacktrace -ffpe-trap=zero,overflow,underflow,invalid test2_hdf5.f90

where h5fc -show gives

gfortran -I/usr/include/hdf5/serial -L/usr/lib/x86_64-linux-gnu/hdf5/serial /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5hl_fortran.a /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5_hl.a /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5_fortran.a /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.a -lpthread -lsz -lz -ldl -lm -Wl,-rpath -Wl,/usr/lib/x86_64-linux-gnu/hdf5/serial

The code compiles successfully without warnings. When I run the executable I get

HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 139755212736320:
#000: …/…/…/src/H5D.c line 121 in H5Dcreate2(): not a datatype ID
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 139755212736320:
#000: …/…/…/src/H5D.c line 331 in H5Dclose(): not a dataset
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 139755212736320:
#000: …/…/…/src/H5D.c line 292 in H5Dopen2(): unable to open dataset
major: Dataset
minor: Can’t open object
#001: …/…/…/src/H5Dint.c line 1205 in H5D__open_name(): not found
major: Dataset
minor: Object not found
#002: …/…/…/src/H5Gloc.c line 422 in H5G_loc_find(): can’t find object
major: Symbol table
minor: Object not found
#003: …/…/…/src/H5Gtraverse.c line 851 in H5G_traverse(): internal path traversal failed
major: Symbol table
minor: Object not found
#004: …/…/…/src/H5Gtraverse.c line 627 in H5G__traverse_real(): traversal operator failed
major: Symbol table
minor: Callback failed
#005: …/…/…/src/H5Gloc.c line 378 in H5G__loc_find_cb(): object ‘fmd’ doesn’t exist
major: Symbol table
minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 139755212736320:
#000: …/…/…/src/H5S.c line 921 in H5Sget_simple_extent_ndims(): not a dataspace
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 139755212736320:
#000: …/…/…/src/H5Dio.c line 314 in H5Dwrite(): dset_id is not a dataset ID
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 139755212736320:
#000: …/…/…/src/H5D.c line 331 in H5Dclose(): not a dataset
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 139755212736320:
#000: …/…/…/src/H5S.c line 489 in H5Sclose(): not a dataspace
major: Invalid arguments to routine
minor: Inappropriate type

I browsed through the forum and in the web but I could not get a solution suitable for my environment.

Thanks a lot in advance for your help!
Antonio

2

Antonio, the first error message

says that the dataset creation has already failed. It complains that H5T_NATIVE_DOUBLE is not a dataype handle, which is not true. I suspect that there’s something else going on in your envrionment. Can you try a shorter version of your program?

program test
use hdf5
implicit none
integer :: err, cnt
real(8) :: fmd
integer(HID_T) :: file_ifmd, dset_ifmd, dspace_ifmd
integer(HSIZE_T) :: dim_fmd(1)
integer(HSIZE_T), dimension(1) :: dim_buf(1)=1
integer(SIZE_T), parameter :: nel=1
integer(HSSIZE_T) :: fmd_coord(1,nel)

dim_fmd(1) = 54

call h5fcreate_f('fmd.hdf5', H5F_ACC_TRUNC_F, file_ifmd, err)
call h5screate_simple_f(1, dim_fmd, dspace_ifmd, err)
call h5dcreate_f(file_ifmd, 'fmd', H5T_NATIVE_DOUBLE, dspace_ifmd, dset_ifmd, err)
call h5dclose_f(dset_ifmd, err)
call h5sclose_f(dspace_ifmd, err)
call h5fclose_f(file_ifmd,err)
end program test

That should fail with the same first error message. Just checking…

G.

3

Dear Gerd,
thanks a lot for your reply. I compiled your short example and the output is this

HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 140490861344576:
#000: …/…/…/src/H5D.c line 121 in H5Dcreate2(): not a datatype ID
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 140490861344576:
#000: …/…/…/src/H5D.c line 331 in H5Dclose(): not a dataset
major: Invalid arguments to routine
minor: Inappropriate type

The file is created but, of course, with no dataset:

$ h5dump fmd.hdf5

HDF5 “fmd.hdf5” {
GROUP “/” {
}
}

I don’t really know what the problem can be. Thanks a lot for your help.
Antonio

4

I realized what is the first problem of my example. I forgot to specify

call h5open_f(err)

at the beginning. Now, with this fix, the short code as you suggested is

program test
use hdf5
implicit none
integer :: err, cnt
real(8) :: fmd
integer(HID_T) :: file_ifmd, dset_ifmd, dspace_ifmd
integer(HSIZE_T) :: dim_fmd(1)
integer(HSIZE_T), dimension(1) :: dim_buf(1)=1
integer(SIZE_T), parameter :: nel=1
integer(HSSIZE_T) :: fmd_coord(1,nel)

dim_fmd(1) = 54

call h5open_f(err)
call h5fcreate_f(‘fmd.hdf5’, H5F_ACC_TRUNC_F, file_ifmd, err)
call h5screate_simple_f(1, dim_fmd, dspace_ifmd, err)
call h5dcreate_f(file_ifmd, ‘fmd’, H5T_NATIVE_DOUBLE, dspace_ifmd, dset_ifmd, err)
call h5dclose_f(dset_ifmd, err)
call h5sclose_f(dspace_ifmd, err)
call h5fclose_f(file_ifmd,err)
end program test

and it correctly produces the output file

h5dump fmd.hdf5
HDF5 “fmd.hdf5” {
GROUP “/” {
DATASET “fmd” {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 54 ) / ( 54 ) }
DATA {
(0): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
(22): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
(44): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
}
}
}
}

Now, going back to the original minimal working example (where I added the line “call h5open_f(err)”)

program test
use hdf5
implicit none
integer :: err, cnt
real(8) :: fmd
integer(HID_T) :: file_ifmd, dset_ifmd, dspace_ifmd
integer(HSIZE_T) :: dim_fmd(1)
integer(HSIZE_T), dimension(1) :: dim_buf(1)=1
integer(SIZE_T), parameter :: nel=1
integer(HSSIZE_T) :: fmd_coord(1,nel)

dim_fmd(1) = 54

call h5open_f(err)
call h5fcreate_f('fmd.hdf5', H5F_ACC_TRUNC_F, file_ifmd, err)
call h5screate_simple_f(1, dim_fmd, dspace_ifmd, err)
call h5dcreate_f(file_ifmd, 'fmd', H5T_NATIVE_DOUBLE, dspace_ifmd, dset_ifmd, err)
call h5dclose_f(dset_ifmd, err)
call h5sclose_f(dspace_ifmd, err)
call h5fclose_f(file_ifmd,err)

call h5fopen_f('fmd.hdf5', H5F_ACC_RDWR_F, file_ifmd, err)
call h5dopen_f(file_ifmd, 'fmd', dset_ifmd, err)

do cnt = 1, 1
fmd_coord(1,nel) = cnt
call h5sselect_elements_f(dset_ifmd, H5S_SELECT_SET_F, 1, nel, fmd_coord, err)
fmd = 0.5d0
call h5dwrite_f(dset_ifmd, H5T_NATIVE_DOUBLE, fmd, dim_buf, err)
end do

call h5dclose_f(dset_ifmd, err)
call h5sclose_f(dspace_ifmd, err)
call h5fclose_f(file_ifmd,err)
end program test

after compiling and executing I get these errors:

HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 140638998615872:
#000: …/…/…/src/H5S.c line 921 in H5Sget_simple_extent_ndims(): not a dataspace
major: Invalid arguments to routine
minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 140638998615872:
#000: …/…/…/src/H5S.c line 489 in H5Sclose(): not a dataspace
major: Invalid arguments to routine
minor: Inappropriate type

Thanks a lot again for your help.
Antonio

5

I solved the problem. Apparently, I have to define also a memory data space and pass the relative arguments to the h5dwrite_f interface. It is not clear to me why these optional arguments are indeed mandatory to have the result that I want; for sure, this is because I’m new to hdf5 and some mechanisms behind are still unknown to me. Here is the working code:

program test

use hdf5
implicit none
integer :: err, cnt
real(8) :: fmd
integer :: memrank = 1 ! Rank of the dataset in memory
integer(HSIZE_T), dimension(1) :: dimsm(1) = 1 ! Memory dataspace dimensions
integer(HID_T) :: file_ifmd, dset_ifmd, dspace_ifmd, memspace
integer(HSIZE_T) :: dim_fmd(1)
integer(HSIZE_T), dimension(1) :: dim_buf(1) = 1
integer(SIZE_T), parameter :: nel=1
integer(HSSIZE_T) :: fmd_coord(1,nel)

dim_fmd(1) = 54

call h5open_f(err)
call h5fcreate_f(‘fmd.hdf5’, H5F_ACC_TRUNC_F, file_ifmd, err)
call h5screate_simple_f(1, dim_fmd, dspace_ifmd, err)
call h5dcreate_f(file_ifmd, ‘fmd’, H5T_NATIVE_DOUBLE, dspace_ifmd, dset_ifmd, err)
call h5dclose_f(dset_ifmd, err)
call h5sclose_f(dspace_ifmd, err)
call h5fclose_f(file_ifmd,err)

call h5fopen_f(‘fmd.hdf5’, H5F_ACC_RDWR_F, file_ifmd, err)
call h5dopen_f(file_ifmd, ‘fmd’, dset_ifmd, err)

call h5dget_space_f(dset_ifmd, dspace_ifmd, err)
CALL h5screate_simple_f(memrank, dimsm, memspace, err)

do cnt = 1, 54
fmd_coord(1,nel) = cnt
call h5sselect_elements_f(dspace_ifmd, H5S_SELECT_SET_F, 1, nel, fmd_coord, err)
fmd = 0.5d0 + dble(cnt)
call h5dwrite_f(dset_ifmd, H5T_NATIVE_DOUBLE, fmd, dim_buf, err, mem_space_id=memspace, file_space_id=dspace_ifmd)
end do

call h5dclose_f(dset_ifmd, err)
call h5sclose_f(dspace_ifmd, err)
call h5fclose_f(file_ifmd,err)

end program test

Please, notice that I modified the loop extrema to “1” to “54”, and the value to be written in the position “cnt” is “fmd = 0.5d0 + dble(cnt)”. This is the content of the fmd.hdf5 file:

$ h5dump fmd.hdf5
HDF5 “fmd.hdf5” {
GROUP “/” {
DATASET “fmd” {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 54 ) / ( 54 ) }
DATA {
(0): 1.5, 2.5, 3.5, 4.5, 5.5, 6.5, 7.5, 8.5, 9.5, 10.5, 11.5, 12.5,
(12): 13.5, 14.5, 15.5, 16.5, 17.5, 18.5, 19.5, 20.5, 21.5, 22.5, 23.5,
(23): 24.5, 25.5, 26.5, 27.5, 28.5, 29.5, 30.5, 31.5, 32.5, 33.5, 34.5,
(34): 35.5, 36.5, 37.5, 38.5, 39.5, 40.5, 41.5, 42.5, 43.5, 44.5, 45.5,
(45): 46.5, 47.5, 48.5, 49.5, 50.5, 51.5, 52.5, 53.5, 54.5
}
}
}
}

Please, could you give me some link to learn more about this topic?
Thanks a lot for your effort. I will mark the post as solved once the pending replies will be accepted and published by the moderator.

All the best

Antonio