RFC: Freeing Memory Allocated by the HDF Library

Hdf-forum archives
1

Hi all,

This is a brief RFC that outlines a new HDF5 API function, H5free_memory(), which can be used to free memory returned from the HDF5 library (e.g., via functions like H5Tget_member_name()). This will be of particular interest to Windows users, since this will help mitigate the problems of linking to multiple versions of the C runtime.

The new function will appear in HDF5 1.8.13, which will be released in May.

If anyone has any comments or concerns, please let us know.

Cheers,

Dana

RFC H5free_memory v2.pdf (233 KB)

2

Looks all good to me.

As a possible alternative to the two-function call version to allocate memory, options might also be to define something like a "memory manager" which contains a structure of function calls that do allocation/freeing, or just passing such a memory allocation function to those HDF5 functions that need to allocate memory. That way you could have a single HDF5 function call using a user-defined function to allocate memory, for instance a function allocating via the C++ new operator or putting data into a C++ std::vector<> object. This however comes with some danger of C++ exceptions being thrown in such an allocation function, so it at least needs to be documented how to deal with such situations.

         Werner

···

On 24.03.2014 19:34, Dana Robinson wrote:

Hi all,

This is a brief RFC that outlines a new HDF5 API function, H5free_memory(), which can be used to free memory returned from the HDF5 library (e.g., via functions like H5Tget_member_name()). This will be of particular interest to Windows users, since this will help mitigate the problems of linking to multiple versions of the C runtime.

The new function will appear in HDF5 1.8.13, which will be released in May.

If anyone has any comments or concerns, please let us know.

Cheers,

Dana

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

3

Hi Dana,

This is a brief RFC that outlines a new HDF5 API function, H5free_memory(),
which can be used to free memory returned from the HDF5 library (e.g., via
functions like H5Tget_member_name()). This will be of particular interest to
Windows users, since this will help mitigate the problems of linking to
multiple versions of the C runtime.

For Python in particular, this looks very interesting. Currently we
have to build HDF5 using the same compiler as the Python.org releases;
this means VS2008 for Python 2.6, 2.7 and 3.2. The HDF Group can't
support old compiler versions forever, so this might be a good way to
support future HDF5 releases without building/testing the HDF5 source
ourselves.

Andrew

4

This is a C library issue, not C++, so we don't have new operators or std::vectors.

While that's an interesting idea, I'd argue that providing a simple free function to users is a much easier and more direct solution.

Dana

···

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Werner Benger
Sent: Monday, March 24, 2014 5:02 PM
To: HDF Users Discussion List
Subject: Re: [Hdf-forum] RFC: Freeing Memory Allocated by the HDF Library

Looks all good to me.

As a possible alternative to the two-function call version to allocate memory, options might also be to define something like a "memory manager" which contains a structure of function calls that do allocation/freeing, or just passing such a memory allocation function to those HDF5 functions that need to allocate memory. That way you could have a single HDF5 function call using a user-defined function to allocate memory, for instance a function allocating via the C++ new operator or putting data into a C++ std::vector<> object. This however comes with some danger of C++ exceptions being thrown in such an allocation function, so it at least needs to be documented how to deal with such situations.

        Werner

On 24.03.2014 19:34, Dana Robinson wrote:
Hi all,

This is a brief RFC that outlines a new HDF5 API function, H5free_memory(), which can be used to free memory returned from the HDF5 library (e.g., via functions like H5Tget_member_name()). This will be of particular interest to Windows users, since this will help mitigate the problems of linking to multiple versions of the C runtime.

The new function will appear in HDF5 1.8.13, which will be released in May.

If anyone has any comments or concerns, please let us know.

Cheers,

Dana

_______________________________________________

Hdf-forum is for HDF software users discussion.

Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>

http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

--

___________________________________________________________________________

Dr. Werner Benger Visualization Research

Center for Computation & Technology at Louisiana State University (CCT/LSU)

2019 Digital Media Center, Baton Rouge, Louisiana 70803

Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

5

The advantage of allowing C++ allocation methods via callbacks would be that you can have one piece of code which retrieves data from an HDF5 file and then forget about the HDF5 file (and library) while still operating on the data, without need to copy C-allocated memory into C++-allocated memory. In general, anything that allows to avoid copying data would be good, though for these kind of data here it's probably not going to be a performance bottleneck. It could rather be some advanced functionality for the rare situations where it would be of relevance, maybe something that can be embedded into some property lists.

          Werner

···

On 25.03.2014 15:09, Dana Robinson wrote:

This is a C library issue, not C++, so we don't have new operators or std::vectors.

While that's an interesting idea, I'd argue that providing a simple free function to users is a much easier and more direct solution.

Dana

*From:*Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] *On Behalf Of *Werner Benger
*Sent:* Monday, March 24, 2014 5:02 PM
*To:* HDF Users Discussion List
*Subject:* Re: [Hdf-forum] RFC: Freeing Memory Allocated by the HDF Library

Looks all good to me.

As a possible alternative to the two-function call version to allocate memory, options might also be to define something like a "memory manager" which contains a structure of function calls that do allocation/freeing, or just passing such a memory allocation function to those HDF5 functions that need to allocate memory. That way you could have a single HDF5 function call using a user-defined function to allocate memory, for instance a function allocating via the C++ new operator or putting data into a C++ std::vector<> object. This however comes with some danger of C++ exceptions being thrown in such an allocation function, so it at least needs to be documented how to deal with such situations.

        Werner

On 24.03.2014 19:34, Dana Robinson wrote:

    Hi all,

    This is a brief RFC that outlines a new HDF5 API function,
    H5free_memory(), which can be used to free memory returned from
    the HDF5 library (e.g., via functions like H5Tget_member_name()).
    This will be of particular interest to Windows users, since this
    will help mitigate the problems of linking to multiple versions of
    the C runtime.

    The new function will appear in HDF5 1.8.13, which will be
    released in May.

    If anyone has any comments or concerns, please let us know.

    Cheers,

    Dana

    _______________________________________________

    Hdf-forum is for HDF software users discussion.

    Hdf-forum@lists.hdfgroup.org <mailto:Hdf-forum@lists.hdfgroup.org>

    http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Center for Computation & Technology at Louisiana State University (CCT/LSU)
2019 Digital Media Center, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362