MPI-IO and Dense Attribute Storage !

Hi all,

  The work I am doing needs to store big attribute in a file. So I try to use the following code !

···

==============
  MPI_Comm comm = MPI_COMM_WORLD;
  MPI_Info info = MPI_INFO_NULL;
  MPI_Init(&argc, &argv);
  
  fapid = H5Pcreate(H5P_FILE_ACCESS);
  H5Pset_libver_bounds(fapid, H5F_LIBVER_LATEST, H5F_LIBVER_LATEST);
  H5Pset_fapl_mpio(fapid, comm, info);
  file_id = H5Fcreate("big_attribute.h5", H5F_ACC_TRUNC, H5P_DEFAULT, fapid);
  .........
   hid_t e_dpid;
  e_dpid = H5Pcreate(H5P_DATASET_CREATE);
  H5Pset_attr_phase_change (e_dpid, 0, 0);
  dset_id = H5Dcreate(group_id, "/Step#0/Energy", H5T_IEEE_F32LE, dset_space_id, H5P_DEFAULT, e_dpid, H5P_DEFAULT);

   The compiling is OK, but running program always gives errors, such as:

    HDF5-DIAG: Error detected in HDF5 (1.9.149) MPI-process 0:
  #000: H5A.c line 276 in H5Acreate2(): unable to create attribute
    major: Symbol table
    minor: Unable to initialize object
  #001: H5VLint.c line 481 in H5VL_attr_create(): create failed
    major: Virtual Object Layer
    minor: Unable to initialize object
  #002: H5VLnative.c line 469 in H5VL_native_attr_create(): unable to create attribute
    major: Attribute
    minor: Unable to initialize object
  #003: H5Aint.c line 280 in H5A_create(): unable to create attribute in object header
    major: Attribute
    minor: Unable to insert object
  #004: H5Oattribute.c line 338 in H5O_attr_create(): unable to add to dense storage
    major: Attribute
    minor: Unable to insert object
  #005: H5Adense.c line 527 in H5A_dense_insert(): unable to insert attribute into fractal heap
    major: Attribute
    minor: Unable to insert object
  #006: H5HF.c line 376 in H5HF_insert(): can't store 'huge' object in fractal heap
    major: Heap
    minor: Unable to insert object
  #007: H5HFhuge.c line 383 in H5HF_huge_insert(): writing 'huge' object to file failed

  It looks like we can't use H5Pset_libver_bounds and H5Pset_fapl_mpio at the same time !

  Anyone can give some help about this !

Cheers !
Bin

--

Hi Bin,

This should be fixed in the 1.8.11 release (May 15th).
You can actually get a copy of the "unofficial" source code now: http://www.hdfgroup.org/ftp/HDF5/releases/hdf5-1.8.11/src/

Thanks,
Mohamad

···

On 5/10/2013 10:26 AM, goon83 wrote:

Hi all,
  The work I am doing needs to store big attribute in a file. So I try to use the following code !

  MPI_Comm comm = MPI_COMM_WORLD;
  MPI_Info info = MPI_INFO_NULL;
MPI_Init(&argc, &argv);
  fapid = H5Pcreate(H5P_FILE_ACCESS);
  H5Pset_libver_bounds(fapid, H5F_LIBVER_LATEST, H5F_LIBVER_LATEST);
  H5Pset_fapl_mpio(fapid, comm, info);
file_id = H5Fcreate("big_attribute.h5", H5F_ACC_TRUNC, H5P_DEFAULT, fapid);
  .........
hid_t e_dpid;
  e_dpid = H5Pcreate(H5P_DATASET_CREATE);
  H5Pset_attr_phase_change (e_dpid, 0, 0);
  dset_id = H5Dcreate(group_id, "/Step#0/Energy", H5T_IEEE_F32LE, dset_space_id, H5P_DEFAULT, e_dpid, H5P_DEFAULT);

   The compiling is OK, but running program always gives errors, such as:

HDF5-DIAG: Error detected in HDF5 (1.9.149) MPI-process 0:
  #000: H5A.c line 276 in H5Acreate2(): unable to create attribute
    major: Symbol table
    minor: Unable to initialize object
  #001: H5VLint.c line 481 in H5VL_attr_create(): create failed
    major: Virtual Object Layer
    minor: Unable to initialize object
  #002: H5VLnative.c line 469 in H5VL_native_attr_create(): unable to create attribute
    major: Attribute
    minor: Unable to initialize object
  #003: H5Aint.c line 280 in H5A_create(): unable to create attribute in object header
    major: Attribute
    minor: Unable to insert object
  #004: H5Oattribute.c line 338 in H5O_attr_create(): unable to add to dense storage
    major: Attribute
    minor: Unable to insert object
  #005: H5Adense.c line 527 in H5A_dense_insert(): unable to insert attribute into fractal heap
    major: Attribute
    minor: Unable to insert object
  #006: H5HF.c line 376 in H5HF_insert(): can't store 'huge' object in fractal heap
    major: Heap
    minor: Unable to insert object
  #007: H5HFhuge.c line 383 in H5HF_huge_insert(): writing 'huge' object to file failed

  It looks like we can't use H5Pset_libver_bounds and H5Pset_fapl_mpio at the same time !

  Anyone can give some help about this !

Cheers !
Bin


365??? <http://gongyi.163.com/love365>

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Hi Mohamad,
    Thanks for your information ! Does the HDF5 1.8.1 support VOL ? Because we need to stay with the version with VOL support.
Cheers,
Bin

···

--

At 2013-05-10 23:31:55,"Mohamad Chaarawi" <chaarawi@hdfgroup.org> wrote:

Hi Bin,

This should be fixed in the 1.8.11 release (May 15th).
You can actually get a copy of the "unofficial" source code now: http://www.hdfgroup.org/ftp/HDF5/releases/hdf5-1.8.11/src/

Thanks,
Mohamad

On 5/10/2013 10:26 AM, goon83 wrote:

Hi all,

  The work I am doing needs to store big attribute in a file. So I try to use the following code !

  MPI_Comm comm = MPI_COMM_WORLD;
  MPI_Info info = MPI_INFO_NULL;
  MPI_Init(&argc, &argv);
  
  fapid = H5Pcreate(H5P_FILE_ACCESS);
  H5Pset_libver_bounds(fapid, H5F_LIBVER_LATEST, H5F_LIBVER_LATEST);
  H5Pset_fapl_mpio(fapid, comm, info);
  file_id = H5Fcreate("big_attribute.h5", H5F_ACC_TRUNC, H5P_DEFAULT, fapid);
  .........
   hid_t e_dpid;
  e_dpid = H5Pcreate(H5P_DATASET_CREATE);
  H5Pset_attr_phase_change (e_dpid, 0, 0);
  dset_id = H5Dcreate(group_id, "/Step#0/Energy", H5T_IEEE_F32LE, dset_space_id, H5P_DEFAULT, e_dpid, H5P_DEFAULT);

   The compiling is OK, but running program always gives errors, such as:

    HDF5-DIAG: Error detected in HDF5 (1.9.149) MPI-process 0:
  #000: H5A.c line 276 in H5Acreate2(): unable to create attribute
    major: Symbol table
    minor: Unable to initialize object
  #001: H5VLint.c line 481 in H5VL_attr_create(): create failed
    major: Virtual Object Layer
    minor: Unable to initialize object
  #002: H5VLnative.c line 469 in H5VL_native_attr_create(): unable to create attribute
    major: Attribute
    minor: Unable to initialize object
  #003: H5Aint.c line 280 in H5A_create(): unable to create attribute in object header
    major: Attribute
    minor: Unable to insert object
  #004: H5Oattribute.c line 338 in H5O_attr_create(): unable to add to dense storage
    major: Attribute
    minor: Unable to insert object
  #005: H5Adense.c line 527 in H5A_dense_insert(): unable to insert attribute into fractal heap
    major: Attribute
    minor: Unable to insert object
  #006: H5HF.c line 376 in H5HF_insert(): can't store 'huge' object in fractal heap
    major: Heap
    minor: Unable to insert object
  #007: H5HFhuge.c line 383 in H5HF_huge_insert(): writing 'huge' object to file failed

  It looks like we can't use H5Pset_libver_bounds and H5Pset_fapl_mpio at the same time !

  Anyone can give some help about this !

Cheers !
Bin

--

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.orghttp://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

Hi Mohamad,
    Thanks for your information ! Does the HDF5 1.8.1 support VOL ? Because we need to stay with the version with VOL support.

Ah. No the VOL branch is not part of any release branch.
It is just a branch off the trunk that I maintain. I'm not certain when was the last time I updated the branch itself from the development branch, but I'll try and do that sometime today, which should take care of the issue with dense attributes that you are seeing.
I'll ping later today individually to let you know when I update the VOL branch.

Thanks,
Mohamad

···

On 5/10/2013 10:39 AM, goon83 wrote:

Cheers,
Bin

Thanks Mohamad.

Best wishes !

Bin

···

--

At 2013-05-10 23:54:45,"Mohamad Chaarawi" <chaarawi@hdfgroup.org> wrote:

On 5/10/2013 10:39 AM, goon83 wrote:

Hi Mohamad,
    Thanks for your information ! Does the HDF5 1.8.1 support VOL ? Because we need to stay with the version with VOL support.

Ah. No the VOL branch is not part of any release branch.
It is just a branch off the trunk that I maintain. I'm not certain when was the last time I updated the branch itself from the development branch, but I'll try and do that sometime today, which should take care of the issue with dense attributes that you are seeing.
I'll ping later today individually to let you know when I update the VOL branch.

Thanks,
Mohamad

Cheers,
Bin