I've place a datafile A2003061013500.L2_LAC_AT110 on the ftp server as well as a routine compress_memleak_multifiles.c that I hope will demonstrate the problem.
It is actually a netcdf4 file but the HDF5 library is called to do the reads and the example program uses only HDF5 calls.
It simulates multiple files by opening the same one five times. The allocated memory leak is about 15M.
We are using hdf5-1.8.10-patch1.
Hi Joel,
Which version of HDF5 are you using? Would you be able to send me a small program or code snippet and a sample file to reproduce the issue? A similar issue came up a while back (pre-HDF5-1.8.5) with a netCDF-4 user, but we never determined what the issue was.
You can place the files on our local ftp server. Let me know (at help@hdfgroup.org) when they are there.
Here are the instructions:
---------------
ftp ftp.hdfgroup.uiuc.edu
Login as "anonymous"
Go to the pub/incoming/help/ directory:
cd pub/incoming/help
Type 'pwd' to see if you are in that location. (You will not be able
to see the contents of the directory.):
pwd
Type 'bin' to put in binary mode:
bin
Place file in directory after you are in binary mode:
put xxx
Get out of ftp:
quit
--------------
Thanks!
-Barbara
=================
Barbara Jones
The HDF Helpdesk
The HDF Group
help@hdfgroup.org
-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On
Behalf Of Joel Gales
Sent: Sunday, February 23, 2014 2:01 PM
To: hdf-forum@lists.hdfgroup.org
Cc: Bryan Franz; sean bailey
Subject: [Hdf-forum] memory leak reading compressed files
I am reading a large number (186) of HDF5 compressed datafiles.
Eventually the allocated memory exceeds 20g and the program fails.
I've traced the increasing memory allocation to the H5Z_filter_deflate() function in H5Zdeflate.c, specifically the lines:
nalloc *= 2;
if(NULL == (new_outbuf = H5MM_realloc(outbuf, nalloc))) {
(void)inflateEnd(&z_strm);
HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, 0, "memory
allocation failed for deflate uncompression")
It appears as if this memory is never freed causing the memory leak.
Is this the desired behaviour or could there be a bug that prevents this memory from being freed?
=========================
Joel Gales
Futuretech Corp.
SIMBIOS Code 616.0
Phone: (301) 286-1403
FAX: (301) 286-0268
Bin Globally Map Locally
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgro
up.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgro
up.org
