Dear all,
I wrote a little C-program to experiment with the HDF5 packet table API. I am specially interested in speed and according to the documentation this should be a fast API. In my example program, I am interested to see if the API slows down when I open many tables each in a different group, appending many (> 10000) rows. (In real-life I would not open and close the HDF5 file in a loop, but open a file with measurements, reading data from it, open the HDF5 file and append this infomation to the tables. But the HDF5 file would contain many groups and each groups several tables (not all of the same size and shape)
I have compiled the code agains HDF5-1.8.1 on a linux platform (dual-core - 2 GB RAM):
# gcc -O3 -W -Wall -I/usr/local/include -L/usr/local/lib -lhdf5_hl -lhdf5 -lz test_h5pt.c
When I write more than 5000 lines to each table, the program uses more than 1 GB of memory! Even when I commented out the line where I append data to the tables (H5PTappend), this problem still occurs. The problem disappears when I also comment out the lines where I open and close the tables (H5PTopen and H5PTclose in write_h5pt_data). I have checked the source code of H5PTopen and H5PTclose, but the code looks fine to me. Any suggestions?
Best regards, Richard van Hees
test_h5pt.c (3.61 KB)
Dear all,
I found the memory leak in H5PTopen: a identifier for a copy of the
dataspace for a dataset (= space_id) was not closed. Attached you find
the patch — against hdf5-1.8.1-snap2/hl/src/H5PT.c — solving this
issue.
Best regards, Richard
Richard van Hees wrote:
···
hdf-forum-subscribe@hdfgroup.orghdf-forum-unsubscribe@hdfgroup.org
Dear all,
I found the memory leak in H5PTopen: a identifier for a copy of the
dataspace for a dataset (= space_id) was not closed. Attached you find
the patch — against hdf5-1.8.1-snap2/hl/src/H5PT.c — solving this
issue.
Best regards, Richard
Richard van Hees wrote:
patch_H5PT.c (1.28 KB)
···
hdf-forum-subscribe@hdfgroup.orghdf-forum-unsubscribe@hdfgroup.org
Richard,
Thank you very much for the patch. I entered a bug report, and we will try to address it in our next release.
Elena
···
On Sep 3, 2008, at 4:56 AM, Richard van Hees wrote:
Dear all,
I found the memory leak in H5PTopen: a identifier for a copy of the dataspace for a dataset (= space_id) was not closed. Attached you find the patch --- against hdf5-1.8.1-snap2/hl/src/H5PT.c --- solving this issue.
Best regards, Richard
Richard van Hees wrote:
Dear all,
I wrote a little C-program to experiment with the HDF5 packet table API. I am specially interested in speed and according to the documentation this should be a fast API. In my example program, I am interested to see if the API slows down when I open many tables each in a different group, appending many (> 10000) rows. (In real-life I would not open and close the HDF5 file in a loop, but open a file with measurements, reading data from it, open the HDF5 file and append this infomation to the tables. But the HDF5 file would contain many groups and each groups several tables (not all of the same size and shape)
I have compiled the code agains HDF5-1.8.1 on a linux platform (dual-core - 2 GB RAM):
# gcc -O3 -W -Wall -I/usr/local/include -L/usr/local/lib -lhdf5_hl -lhdf5 -lz test_h5pt.c
When I write more than 5000 lines to each table, the program uses more than 1 GB of memory! Even when I commented out the line where I append data to the tables (H5PTappend), this problem still occurs. The problem disappears when I also comment out the lines where I open and close the tables (H5PTopen and H5PTclose in write_h5pt_data). I have checked the source code of H5PTopen and H5PTclose, but the code looks fine to me. Any suggestions?
Best regards, Richard van Hees
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
233c233,234
< table = (htbl_t *)malloc(sizeof(htbl_t));
---
if ( (table = (htbl_t *)malloc(sizeof(htbl_t))) == NULL )
goto out;
237,239c238
< goto out;
< if(table->dset_id < 0)
< goto out;
---
goto out;
243c242
< goto out;
---
goto out;
247c246
< goto out;
---
goto out;
250c249
< goto out;
---
goto out;
254c253
< goto out;
---
goto out;
258c257
< goto out;
---
goto out;
260c259,260
< goto out;
---
goto out;
(void) H5Sclose( space_id );
264,267c264,265
< ret_value = H5Iregister(H5PT_ptable_id_type, table);
<
< if(ret_value != H5I_INVALID_HID)
< H5PT_ptable_count++;
---
if( (ret_value = H5Iregister(H5PT_ptable_id_type, table)) != H5I_INVALID_HID)
H5PT_ptable_count++;
269c267
< H5PT_close(table);
---
H5PT_close(table);
276c274,275
< if(table)
---
H5Sclose( space_id );
if(table != NULL)
309c308
< goto out;
---
goto out;
313c312
< goto out;
---
goto out;
317c316
< goto out;
---
goto out;
324c323
< if(table)
---
if(table != NULL)
359c358
< if((table = H5Iremove_verify(table_id, H5PT_ptable_id_type)) ==NULL)
---
if((table = H5Iremove_verify(table_id, H5PT_ptable_id_type)) == NULL)
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe@hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe@hdfgroup.org.
