--- On Tue, 9/14/10, Quincey Koziol <koziol@hdfgroup.org> wrote:
From: Quincey Koziol <koziol@hdfgroup.org>
Subject: Re: [Hdf-forum] Store data in hierarchy, does it impact the performance?
To: "HDF Users Discussion List" <hdf-forum@hdfgroup.org>
Date: Tuesday, September 14, 2010, 5:47 PM
Hi Elisa,
On Sep 14, 2010, at 4:43 PM, elisa sibarani wrote:
Hi All,
I really need your help or idea, do I need to store data in hierarchy if I want to use HDF5? When I do a small benchmark, the performance of the file decrease when the data store directly in a dataset (after a 'root' group), rather than in a hierarchical way, Is there any reason behind that result?
Please, really need the reason of this question.
Hmm, I don't have any good reason why this should be so\. Do you have a small benchmark that demonstrates the issue?
Quincey
From: Quincey Koziol <koziol@hdfgroup.org>
Subject: Re: [Hdf-forum] Store data in hierarchy, does it impact the performance?
To: "HDF Users Discussion List" <hdf-forum@hdfgroup.org>
Date: Tuesday, September 14, 2010, 5:47 PM
Hi Elisa,
On Sep 14, 2010, at 4:43 PM, elisa sibarani wrote:
> Hi All,
>
> I really need your help or idea, do I need to store data in hierarchy if I want to use HDF5? When I do a small benchmark, the performance of the file decrease when the data store directly in a dataset (after a 'root' group), rather than in a hierarchical way, Is there any reason behind that result?
>
> Please, really need the reason of this question.
Hmm, I don't have any good reason why this should be so. Do you have a small benchmark that demonstrates the issue?
So probably HDF5 library uses path resolution whenever dataset read process executed?
But in my code, for no hierarchy there is no need to do that process by the library since the dataset directly can be accessed from "root" group, which means that the process must be shorter to open and read dataset. What do you think?
From: Rob Latham <robl@mcs.anl.gov>
Subject: Re: [Hdf-forum] Store data in hierarchy, does it impact the performance?
To: "HDF Users Discussion List" <hdf-forum@hdfgroup.org>
Date: Wednesday, September 15, 2010, 10:20 AM
On Wed, Sep 15, 2010 at 12:23:47AM +0200, Quincey Koziol wrote:
Hi Elisa,
On Sep 15, 2010, at 12:09 AM, elisa sibarani wrote:
> Hi Quincey,
>
> Here I attached the example of HDF5 file with hierarchy, file with no hierarchy inside, and the benchmark result.
Actually, I meant source code for the benchmark\. Can you send that?
the graph is interesting, though: writing either way is essentially
the same. it's the read case that is taking significantly longer.
could there be anything in the way HDF5 resolves groups? In UNIX
land, path resolution happens directory by directory.
--
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA
