Hi All,
I am seeing some behavior in some tests that I am doing that suggests
that on the sec2 vfd (or any vfd I've tried) the small data block size
property (H5Pset_small_data_block_size) is having no effect in changing
I/O requests to underlying system I/O interface.
Does anyone have experience using this file access property such that
they KNOW FOR CERTAIN it is taking effect and indeed resulting in
different (lower) I/O request frequency to underlying I/O interface?
Mark
···
--
Mark C. Miller, Lawrence Livermore National Laboratory
================!!LLNL BUSINESS ONLY!!================
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-851
Am I missing something here?
I thought the intent was to perform sub-allocations out of a larger
contiguous block so that you could write all the data associated with
the datasets that went into a 'small block' in a single I/O request.
Mark
···
On Wed, 2010-03-03 at 13:04, Quincey Koziol wrote:
Hi Mark,
On Mar 3, 2010, at 12:51 PM, Mark Miller wrote:
> Hi All,
>
> I am seeing some behavior in some tests that I am doing that suggests
> that on the sec2 vfd (or any vfd I've tried) the small data block size
> property (H5Pset_small_data_block_size) is having no effect in changing
> I/O requests to underlying system I/O interface.
>
> Does anyone have experience using this file access property such that
> they KNOW FOR CERTAIN it is taking effect and indeed resulting in
> different (lower) I/O request frequency to underlying I/O interface?
This API routine is only used to "push" allocation of the space for storing small dataset raw data together in the file. It has no effect on I/O requests to the underlying system. H5Pset_sieve_buf_size may help you out though...
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://*mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
Mark C. Miller, Lawrence Livermore National Laboratory
================!!LLNL BUSINESS ONLY!!================
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-851
Hi Mark,
Am I missing something here?
I thought the intent was to perform sub-allocations out of a larger
contiguous block so that you could write all the data associated with
the datasets that went into a 'small block' in a single I/O request.
No, that's not what occurs currently. (Although it might be a logical outgrowth of the current feature) Currently, it is only trying to reduce fragmentation of space allocation in the file.
Quincey
···
On Mar 3, 2010, at 3:47 PM, Mark Miller wrote:
Mark
On Wed, 2010-03-03 at 13:04, Quincey Koziol wrote:
Hi Mark,
On Mar 3, 2010, at 12:51 PM, Mark Miller wrote:
Hi All,
I am seeing some behavior in some tests that I am doing that suggests
that on the sec2 vfd (or any vfd I've tried) the small data block size
property (H5Pset_small_data_block_size) is having no effect in changing
I/O requests to underlying system I/O interface.
Does anyone have experience using this file access property such that
they KNOW FOR CERTAIN it is taking effect and indeed resulting in
different (lower) I/O request frequency to underlying I/O interface?
This API routine is only used to "push" allocation of the space for storing small dataset raw data together in the file. It has no effect on I/O requests to the underlying system. H5Pset_sieve_buf_size may help you out though...
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://*mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
Mark C. Miller, Lawrence Livermore National Laboratory
================!!LLNL BUSINESS ONLY!!================
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-851
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
I tried sieve buf setting. It has no effect either. But, thats kinda
what I expected as docs indicate that really has to do with chunking for
chunked datasets. Mine are contiguous.
Mark
···
On Wed, 2010-03-03 at 13:04, Quincey Koziol wrote:
Hi Mark,
On Mar 3, 2010, at 12:51 PM, Mark Miller wrote:
> Hi All,
>
> I am seeing some behavior in some tests that I am doing that suggests
> that on the sec2 vfd (or any vfd I've tried) the small data block size
> property (H5Pset_small_data_block_size) is having no effect in changing
> I/O requests to underlying system I/O interface.
>
> Does anyone have experience using this file access property such that
> they KNOW FOR CERTAIN it is taking effect and indeed resulting in
> different (lower) I/O request frequency to underlying I/O interface?
This API routine is only used to "push" allocation of the space for storing small dataset raw data together in the file. It has no effect on I/O requests to the underlying system. H5Pset_sieve_buf_size may help you out though...
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://*mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
Mark C. Miller, Lawrence Livermore National Laboratory
================!!LLNL BUSINESS ONLY!!================
miller86@llnl.gov urgent: miller86@pager.llnl.gov
T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-851