In our application, we are writing simple HDF5 files to store images. On
Windows platform, the operation system tries to buffer the files in memory,
and it becomes inefficient when writing a lot of HDF5 files sequentially.
Looking at the HDF5 documentation, it seems like that we would be better off
using the unbuffered drivers such as H5FD_DIRECT or H5FD_SEC2
(http://www.hdfgroup.org/HDF5/doc1.6/UG/08_TheFile.html).
We tried to set up the calls for SEC2 using C++ API, reference
(http://www.hdfgroup.org/HDF5/doc/cpplus_RM/classH5_1_1FileAccPropList.html).
However, it does not seem to have any effect, when setSec2() was called. Am
I understanding things right?
Can I avoid operation system buffering at all with HDF5 lib? How can I
achieve the best sequential write speed to disk? Thanks a lot!
···
--
View this message in context: http://hdf-forum.184993.n3.nabble.com/Setting-up-unbuffered-I-O-using-HDF5-C-API-tp3599999p3599999.html
Sent from the hdf-forum mailing list archive at Nabble.com.
Hi ljin,
Please make sure that the application calls the driver function before the
files are created or opened.
Binh-Minh
···
-----Original Message-----
From: hdf-forum-bounces@hdfgroup.org [mailto:hdf-forum-bounces@hdfgroup.org]
On Behalf Of ljin
Sent: Monday, December 19, 2011 7:46 PM
To: hdf-forum@hdfgroup.org
Subject: [Hdf-forum] Setting up unbuffered I/O using HDF5 C++ API
In our application, we are writing simple HDF5 files to store images. On
Windows platform, the operation system tries to buffer the files in memory,
and it becomes inefficient when writing a lot of HDF5 files sequentially.
Looking at the HDF5 documentation, it seems like that we would be better off
using the unbuffered drivers such as H5FD_DIRECT or H5FD_SEC2
(http://www.hdfgroup.org/HDF5/doc1.6/UG/08_TheFile.html).
We tried to set up the calls for SEC2 using C++ API, reference
(http://www.hdfgroup.org/HDF5/doc/cpplus_RM/classH5_1_1FileAccPropList.html)
.
However, it does not seem to have any effect, when setSec2() was called. Am
I understanding things right?
Can I avoid operation system buffering at all with HDF5 lib? How can I
achieve the best sequential write speed to disk? Thanks a lot!
--
View this message in context:
http://hdf-forum.184993.n3.nabble.com/Setting-up-unbuffered-I-O-using-HDF5-C
-API-tp3599999p3599999.html
Sent from the hdf-forum mailing list archive at Nabble.com.
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
Internal Virus Database is out-of-date.
Checked by AVG.
Version: 7.5.549 / Virus Database: 270.9.0/1778 - Release Date: 11/9/2008
2:14 PM
