Hi All:
I'm very new to HDF, so please be patient with a dumb question!
I'm sure there is a proper HDF way to do this...I just don't know where
to start!
My goal is to have a (relatively) small buffer in memory that is written
out over time to the HDF dataset on disk, something like:
numPages = 100
Page = 0
Allocate an HDF dataset of size (numPages * bufferSize)
For I = 0 to numPages
fill the MemoryBuffer
write contents of MemoryBuffer to Dataset at locations Page*bufferSize
to (Page*bufferSize)+(bufferSize-1)
Page++
What is the best way to do this in HDF?
What is the largest dataset I can declare in my environment?
I'm using HDF 5 version 1.8.3 on a Windows XP PC using Visual C++ 2005
Express.
Thanks for all the help!
amb
Hi Amrose
I’m very new to HDF, so please be patient with a dumb question!
Sometimes answers are dump, but questions never, if all act in good faith
IMHO 
I think for what you're trying to do it is better to go with unbuffered
temporary flat binary files and not hdf5. I have done the same and my
experience is that the performance difference is significant.
HTH
-- dimitris
···
I’m sure there is a proper HDF way to do this…I just don’t know where to
start!
My goal is to have a (relatively) small buffer in memory that is written
out over time to the HDF dataset on disk, something like:
numPages = 100
Page = 0
Allocate an HDF dataset of size (numPages * bufferSize)
For I = 0 to numPages
fill the MemoryBuffer
write contents of MemoryBuffer to Dataset at locations Page*bufferSize to
(Page*bufferSize)+(bufferSize-1)
Page++
What is the best way to do this in HDF?
What is the largest dataset I can declare in my environment?
I’m using HDF 5 version 1.8.3 on a Windows XP PC using Visual C++ 2005
Express.
Thanks for all the help!
amb
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Well, if you're interested in learning about HDF5, then this is a good
example.
So, first off, welcome to HDF!
There are several examples available on the HDF website which cover a
variety of circumstances. They're available here:
http://www.hdfgroup.org/ftp/HDF5/examples/examples-by-api/api18-c.html
Specifically, the example file that directly applies to the problem you've
stated can be found here:
http://www.hdfgroup.org/ftp/HDF5/examples/examples-by-api/hdf5-examples/1_8/C/H5D/h5ex_d_hyper.c
It shows how to select a portion (aka, hyperslab) of a given dataset and how
to read/write to it. What you're going to end up doing is iterating around
the H5Sselect_hyperslab() function and updating your 'start' array to update
small portions of the dataset with the write buffer that has your data.
Other than that, if you're having trouble let me know and I'll help you out
a bit. Make sure to take a look at the User's Guide:
http://www.hdfgroup.org/HDF5/doc/UG/UG_frame10Datasets.html
-Stefan
···
On Wed, Aug 12, 2009 at 11:45 AM, Dimitris Servis <servisster@gmail.com>wrote:
Hi Amrose
I’m very new to HDF, so please be patient with a dumb question!
Sometimes answers are dump, but questions never, if all act in good faith
IMHO
I think for what you're trying to do it is better to go with unbuffered
temporary flat binary files and not hdf5. I have done the same and my
experience is that the performance difference is significant.
HTH
-- dimitris
I’m sure there is a proper HDF way to do this…I just don’t know where to
start!
My goal is to have a (relatively) small buffer in memory that is written
out over time to the HDF dataset on disk, something like:
numPages = 100
Page = 0
Allocate an HDF dataset of size (numPages * bufferSize)
For I = 0 to numPages
fill the MemoryBuffer
write contents of MemoryBuffer to Dataset at locations Page*bufferSize to
(Page*bufferSize)+(bufferSize-1)
Page++
What is the best way to do this in HDF?
What is the largest dataset I can declare in my environment?
I’m using HDF 5 version 1.8.3 on a Windows XP PC using Visual C++ 2005
Express.
Thanks for all the help!
amb
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
Stefan Novak
Sent from Greenbelt, Maryland, United States
