Requesting feedback on the use of the background buffer in type conversions

Hi all,

In order to simplify the HDF5 library code and allow us to implement user-defined vlen conversions, we are considering removing the background buffer from type conversion functions. This would cause the library to issue an error if the need_bkg field in H5T_cdata_t were set to anything other than H5T_BKG_NO, always pass NULL for the bkg argument to type conversion functions, and ignore the bkg parameter for H5Pset_buffer. 1.10 may see these types and functions versioned with the aforementioned arguments removed from the new versions.

Would anyone would be affected by removing the background buffer? If so, are you supplying your own buffer to H5Pset_buffer, using the background buffer as a temporary buffer in a type conversion function (H5T_BKG_TEMP), or do you require the original destination data to be placed in the background buffer (H5T_BKG_YES)? It should be possible to preserve only the temporary buffer facility (H5T_BKG_TEMP) while gaining most of the benefit of removing the background buffer, if people are currently using the background buffer as a temporary buffer.

Thanks,

Neil Fortner

The HDF Group

Hi Neil,

The h5py project uses the backing buffer in several conversion paths:

Unfortunately this was written years ago and I'm not 100% sure why
this is needed. As I recall the converter crashes without it.

Andrew

Hi Neil,
I use H5T_BKG_TEMP as part of several conversion paths in *h5labview;*
specifically
in conversion paths where the object size changes - such as converting
between native (LabVIEW's "handle" based memory) and fixed length strings.
I would be keen to not have to rewrite that functionality as it's fairly
involved and has been running without error for some considerable time.

Cheers,
Martijn

···

On 21 January 2015 at 10:37, Neil Fortner <nfortne2@hdfgroup.org> wrote:

Hi all,

In order to simplify the HDF5 library code and allow us to implement
user-defined vlen conversions, we are considering removing the background
buffer from type conversion functions. This would cause the library to
issue an error if the need_bkg field in H5T_cdata_t were set to anything
other than H5T_BKG_NO, always pass NULL for the bkg argument to type
conversion functions, and ignore the bkg parameter for H5Pset_buffer. 1.10
may see these types and functions versioned with the aforementioned
arguments removed from the new versions.

Would anyone would be affected by removing the background buffer? If so,
are you supplying your own buffer to H5Pset_buffer, using the background
buffer as a temporary buffer in a type conversion function (H5T_BKG_TEMP),
or do you require the original destination data to be placed in the
background buffer (H5T_BKG_YES)? It should be possible to preserve only the
temporary buffer facility (H5T_BKG_TEMP) while gaining most of the benefit
of removing the background buffer, if people are currently using the
background buffer as a temporary buffer.

Thanks,

Neil Fortner

The HDF Group

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org

http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

Hi Neil,

Sorry, I'm late to the party. I use H5T_BKG_YES in Live HDF5 (also an interface to LabVIEW). IIRC, H5T_BKG_YES allows me to use re-use existing LabVIEW memory handles when reading from HDF5 datasets into memory that has already been allocated in LabVIEW. Without it, I would need to reallocate handles and may also leak LabVIEW memory.

Jason

···

On 1/20/2015 6:04 PM, Martijn Jasperse wrote:

Hi Neil,
I use H5T_BKG_TEMP as part of several conversion paths in /h5labview;/ specifically in conversion paths where the object size changes - such as converting between native (LabVIEW's "handle" based memory) and fixed length strings. I would be keen to not have to rewrite that functionality as it's fairly involved and has been running without error for some considerable time.

Cheers,
Martijn

On 21 January 2015 at 10:37, Neil Fortner <nfortne2@hdfgroup.org > <mailto:nfortne2@hdfgroup.org>> wrote:

    Hi all,

    In order to simplify the HDF5 library code and allow us to
    implement user-defined vlen conversions, we are considering
    removing the background buffer from type conversion functions.
    This would cause the library to issue an error if the need_bkg
    field in H5T_cdata_t were set to anything other than H5T_BKG_NO,
    always pass NULL for the bkg argument to type conversion
    functions, and ignore the bkg parameter for H5Pset_buffer. 1.10
    may see these types and functions versioned with the
    aforementioned arguments removed from the new versions.

    Would anyone would be affected by removing the background buffer?
    If so, are you supplying your own buffer to H5Pset_buffer, using
    the background buffer as a temporary buffer in a type conversion
    function (H5T_BKG_TEMP), or do you require the original
    destination data to be placed in the background buffer
    (H5T_BKG_YES)? It should be possible to preserve only the
    temporary buffer facility (H5T_BKG_TEMP) while gaining most of the
    benefit of removing the background buffer, if people are currently
    using the background buffer as a temporary buffer.

    Thanks,

    Neil Fortner

    The HDF Group

    _______________________________________________
    Hdf-forum is for HDF software users discussion.
    Hdf-forum@lists.hdfgroup.org <mailto:Hdf-forum@lists.hdfgroup.org>
    http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
    Twitter: https://twitter.com/hdf5

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5