Hello,
I have read HDF5 files can be in-memory structure.
That said, is it be possible to the retrieve this HDF5 file as a binary
array and send it by the network (as a float array with MPI for instance) ?
Thanks a lot,
Cyril.
Hi Cyril,
this has been done as part of the German TIKSL project
( http://www.zib.de/visual/projects/TIKSL/ ) with the creation
of the HDF5 Streaming VFD driver, which as of now is available
through
http://hdf5-addons.origo.ethz.ch/
It is however currently not actively worked on, so might require
some code cleanup and revision, though it was recently working with
HDF5 1.8 .
It does not use MPI, but directly operates on TCP sockets. Thereby
it does not conflict with an MPI application and can be used within
some MPI-parallelized code without affecting that one.
Cheers,
Werner
···
On Wed, 20 Jan 2010 11:47:08 +0100, Giraudon Cyril <cyril.giraudon@free.fr> wrote:
Hello,
I have read HDF5 files can be in-memory structure.
That said, is it be possible to the retrieve this HDF5 file as a binary
array and send it by the network (as a float array with MPI for instance) ?Thanks a lot,
Cyril.
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362
Thanks you Wermer,
Must I really use a new driver ?
I thought I could use the hdf5_core driver and then manipulate a C
structure.
What should I do if I would to put a memory HDF5 file into a cache system ?
Regards,
Cyril.
Le 20/01/2010 12:26, Werner Benger a �crit :
···
Hi Cyril,
this has been done as part of the German TIKSL project
( http://www.zib.de/visual/projects/TIKSL/ ) with the creation
of the HDF5 Streaming VFD driver, which as of now is available
throughhttp://hdf5-addons.origo.ethz.ch/
It is however currently not actively worked on, so might require
some code cleanup and revision, though it was recently working with
HDF5 1.8 .It does not use MPI, but directly operates on TCP sockets. Thereby
it does not conflict with an MPI application and can be used within
some MPI-parallelized code without affecting that one.Cheers,
WernerOn Wed, 20 Jan 2010 11:47:08 +0100, Giraudon Cyril > <cyril.giraudon@free.fr> wrote:
Hello,
I have read HDF5 files can be in-memory structure.
That said, is it be possible to the retrieve this HDF5 file as a binary
array and send it by the network (as a float array with MPI for
instance) ?Thanks a lot,
Cyril.
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Hi Cyril,
Thanks you Werner,
Must I really use a new driver ?
I thought I could use the hdf5_core driver and then manipulate a C
structure.
What should I do if I would to put a memory HDF5 file into a cache system ?
You can use the core driver to "store" an HDF5 file in memory, along with the H5Fget_vfd_handle to retrieve a pointer to the memory buffer where the file is stored. However, we are missing the final piece of the puzzle: a way to "open" a file from a buffer in memory with the core driver. So, at the moment, you need to store the buffer in a file on the receiving side and then open it with a file-oriented VFD (sec2, stdio, mpio, etc). However, if you'd like to work on a patch for opening a file from a memory buffer, I'd be happy to work with you on how I think you should proceed, etc.
Quincey
···
On Jan 20, 2010, at 6:21 AM, Giraudon Cyril wrote:
Regards,
Cyril.
Le 20/01/2010 12:26, Werner Benger a écrit :
Hi Cyril,
this has been done as part of the German TIKSL project
( http://www.zib.de/visual/projects/TIKSL/ ) with the creation
of the HDF5 Streaming VFD driver, which as of now is available
throughhttp://hdf5-addons.origo.ethz.ch/
It is however currently not actively worked on, so might require
some code cleanup and revision, though it was recently working with
HDF5 1.8 .It does not use MPI, but directly operates on TCP sockets. Thereby
it does not conflict with an MPI application and can be used within
some MPI-parallelized code without affecting that one.Cheers,
WernerOn Wed, 20 Jan 2010 11:47:08 +0100, Giraudon Cyril >> <cyril.giraudon@free.fr> wrote:
Hello,
I have read HDF5 files can be in-memory structure.
That said, is it be possible to the retrieve this HDF5 file as a binary
array and send it by the network (as a float array with MPI for
instance) ?Thanks a lot,
Cyril.
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Hi Cyril,
there is a memory image driver in the standard HDF5 distribution, from
which this streaming file driver is derived, but the streaming file driver
is currently not part of the HDF5 distribution.
What do you mean by putting a "file into a cache system" ?
Werner
···
On Wed, 20 Jan 2010 13:21:15 +0100, Giraudon Cyril <cyril.giraudon@free.fr> wrote:
Thanks you Wermer,
Must I really use a new driver ?
I thought I could use the hdf5_core driver and then manipulate a C
structure.
What should I do if I would to put a memory HDF5 file into a cache system ?Regards,
Cyril.
Le 20/01/2010 12:26, Werner Benger a écrit :
Hi Cyril,
this has been done as part of the German TIKSL project
( http://www.zib.de/visual/projects/TIKSL/ ) with the creation
of the HDF5 Streaming VFD driver, which as of now is available
throughhttp://hdf5-addons.origo.ethz.ch/
It is however currently not actively worked on, so might require
some code cleanup and revision, though it was recently working with
HDF5 1.8 .It does not use MPI, but directly operates on TCP sockets. Thereby
it does not conflict with an MPI application and can be used within
some MPI-parallelized code without affecting that one.Cheers,
WernerOn Wed, 20 Jan 2010 11:47:08 +0100, Giraudon Cyril >> <cyril.giraudon@free.fr> wrote:
Hello,
I have read HDF5 files can be in-memory structure.
That said, is it be possible to the retrieve this HDF5 file as a binary
array and send it by the network (as a float array with MPI for
instance) ?Thanks a lot,
Cyril.
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362
Hi Wermer,
I work at the moment on a scientific format based on HDF5.
The format permits to describe an electromagnetic simulation and
electromagnetic objects like materials, mesh, result data ...
(if you are interested : http://groups.google.com/group/amelet-hdf)
The idea would be to write data in memory and cache them in a cache
system like http://memcached.org/, to store and retore rapidly or to
share data with other processes.
I imagine a weak coupling system for scientific applications.
Regards,
Cyril.
···
Le 20/01/2010 15:38, Werner Benger a écrit :
Hi Cyril,
there is a memory image driver in the standard HDF5 distribution, from
which this streaming file driver is derived, but the streaming file
driver
is currently not part of the HDF5 distribution.What do you mean by putting a "file into a cache system" ?
Werner
On Wed, 20 Jan 2010 13:21:15 +0100, Giraudon Cyril > <cyril.giraudon@free.fr> wrote:
Thanks you Wermer,
Must I really use a new driver ?
I thought I could use the hdf5_core driver and then manipulate a C
structure.
What should I do if I would to put a memory HDF5 file into a cache
system ?Regards,
Cyril.
Le 20/01/2010 12:26, Werner Benger a écrit :
Hi Cyril,
this has been done as part of the German TIKSL project
( http://www.zib.de/visual/projects/TIKSL/ ) with the creation
of the HDF5 Streaming VFD driver, which as of now is available
throughhttp://hdf5-addons.origo.ethz.ch/
It is however currently not actively worked on, so might require
some code cleanup and revision, though it was recently working with
HDF5 1.8 .It does not use MPI, but directly operates on TCP sockets. Thereby
it does not conflict with an MPI application and can be used within
some MPI-parallelized code without affecting that one.Cheers,
WernerOn Wed, 20 Jan 2010 11:47:08 +0100, Giraudon Cyril >>> <cyril.giraudon@free.fr> wrote:
Hello,
I have read HDF5 files can be in-memory structure.
That said, is it be possible to the retrieve this HDF5 file as a
binary
array and send it by the network (as a float array with MPI for
instance) ?Thanks a lot,
Cyril.
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Hi Cyril,
this sounds like a nice project, similar to what we're doing as well
(file format: http://www.fiberbundle.net has similar intentions).
It seems you would not use memcached itself, as it is only designed for
"small" chunks:
"Memcached is an in-memory key-value store for small chunks of arbitrary data
(strings, objects) from results of database calls, API calls, or page rendering." ?
Rather you'd had something like a server that retrieves complete HDF5 files
via socket and make them available to other clients, like on some NFS or SSHFS
where the file written is closed after each operation, and thereby "published"
for other clients to read the same file?
Werner
···
On Wed, 20 Jan 2010 15:58:33 +0100, Giraudon Cyril <cyril.giraudon@free.fr> wrote:
Hi Wermer,
I work at the moment on a scientific format based on HDF5.
The format permits to describe an electromagnetic simulation and
electromagnetic objects like materials, mesh, result data ...
(if you are interested : http://groups.google.com/group/amelet-hdf)The idea would be to write data in memory and cache them in a cache
system like http://memcached.org/, to store and retore rapidly or to
share data with other processes.I imagine a weak coupling system for scientific applications.
Regards,
Cyril.
Le 20/01/2010 15:38, Werner Benger a écrit :
Hi Cyril,
there is a memory image driver in the standard HDF5 distribution, from
which this streaming file driver is derived, but the streaming file
driver
is currently not part of the HDF5 distribution.What do you mean by putting a "file into a cache system" ?
Werner
On Wed, 20 Jan 2010 13:21:15 +0100, Giraudon Cyril >> <cyril.giraudon@free.fr> wrote:
Thanks you Wermer,
Must I really use a new driver ?
I thought I could use the hdf5_core driver and then manipulate a C
structure.
What should I do if I would to put a memory HDF5 file into a cache
system ?Regards,
Cyril.
Le 20/01/2010 12:26, Werner Benger a écrit :
Hi Cyril,
this has been done as part of the German TIKSL project
( http://www.zib.de/visual/projects/TIKSL/ ) with the creation
of the HDF5 Streaming VFD driver, which as of now is available
throughhttp://hdf5-addons.origo.ethz.ch/
It is however currently not actively worked on, so might require
some code cleanup and revision, though it was recently working with
HDF5 1.8 .It does not use MPI, but directly operates on TCP sockets. Thereby
it does not conflict with an MPI application and can be used within
some MPI-parallelized code without affecting that one.Cheers,
WernerOn Wed, 20 Jan 2010 11:47:08 +0100, Giraudon Cyril >>>> <cyril.giraudon@free.fr> wrote:
Hello,
I have read HDF5 files can be in-memory structure.
That said, is it be possible to the retrieve this HDF5 file as a
binary
array and send it by the network (as a float array with MPI for
instance) ?Thanks a lot,
Cyril.
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362
Wermer,
I don't express my mind correctly.
I 'd like to do strong coupling between applications (different
applications, different processes) without being too intrusive into the
applications. There would be some data exchanges at each step of the
algorithm (time domain computation, FDTD for instance) as with an MPI
solution.
The amount of data is relatively low but I have to perform exchange the
fastest as possible. Amelet-HDF can be seen as a cross language IDL of
kind for electromagnetic computations. I 'd like a simple solution for
C/Fortran languages.
I will consider your proposal, although I did not want to write a
complete server.
Regards,
Cyril.
···
Le 20/01/2010 16:11, Werner Benger a écrit :
Hi Cyril,
this sounds like a nice project, similar to what we're doing as well
(file format: http://www.fiberbundle.net has similar intentions).It seems you would not use memcached itself, as it is only designed for
"small" chunks:"Memcached is an in-memory key-value store for small chunks of
arbitrary data
(strings, objects) from results of database calls, API calls, or page
rendering." ?Rather you'd had something like a server that retrieves complete HDF5
files
via socket and make them available to other clients, like on some NFS
or SSHFS
where the file written is closed after each operation, and thereby
"published"
for other clients to read the same file?Werner
On Wed, 20 Jan 2010 15:58:33 +0100, Giraudon Cyril > <cyril.giraudon@free.fr> wrote:
Hi Wermer,
I work at the moment on a scientific format based on HDF5.
The format permits to describe an electromagnetic simulation and
electromagnetic objects like materials, mesh, result data ...
(if you are interested : http://groups.google.com/group/amelet-hdf)The idea would be to write data in memory and cache them in a cache
system like http://memcached.org/, to store and retore rapidly or to
share data with other processes.I imagine a weak coupling system for scientific applications.
Regards,
Cyril.
Le 20/01/2010 15:38, Werner Benger a écrit :
Hi Cyril,
there is a memory image driver in the standard HDF5 distribution, from
which this streaming file driver is derived, but the streaming file
driver
is currently not part of the HDF5 distribution.What do you mean by putting a "file into a cache system" ?
Werner
On Wed, 20 Jan 2010 13:21:15 +0100, Giraudon Cyril >>> <cyril.giraudon@free.fr> wrote:
Thanks you Wermer,
Must I really use a new driver ?
I thought I could use the hdf5_core driver and then manipulate a C
structure.
What should I do if I would to put a memory HDF5 file into a cache
system ?Regards,
Cyril.
Le 20/01/2010 12:26, Werner Benger a écrit :
Hi Cyril,
this has been done as part of the German TIKSL project
( http://www.zib.de/visual/projects/TIKSL/ ) with the creation
of the HDF5 Streaming VFD driver, which as of now is available
throughhttp://hdf5-addons.origo.ethz.ch/
It is however currently not actively worked on, so might require
some code cleanup and revision, though it was recently working with
HDF5 1.8 .It does not use MPI, but directly operates on TCP sockets. Thereby
it does not conflict with an MPI application and can be used within
some MPI-parallelized code without affecting that one.Cheers,
WernerOn Wed, 20 Jan 2010 11:47:08 +0100, Giraudon Cyril >>>>> <cyril.giraudon@free.fr> wrote:
Hello,
I have read HDF5 files can be in-memory structure.
That said, is it be possible to the retrieve this HDF5 file as a
binary
array and send it by the network (as a float array with MPI for
instance) ?Thanks a lot,
Cyril.
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Hi Cyril,
the aforementioned HDF5 streaming VFD already provides a complete server, all
the socket communication is done internally. Only the TCP socket needs to be opened
by the application, and the socket descriptor is passed to the HDF5 streaming VFD,
like otherwise a filename is passed to it. You can write to a stream or read from it,
and if you include an HTTP header in this stream - which is supported by the most
recent version of the streaming VFD - then it easily becomes its standalone webserver
that produces HDF5 files on demand in memory, just for streaming data, never
a physical file created anywhere. If your application can write to an HDF5 file,
then it can also write to a stream, no change required on the writing code,
only the "file opening" part needs to know about the streaming VFD.
However, I don't want to promise too much, only saying that it has worked
this way some time ago, and it *should* still work.
Cheers,
Werner
···
On Thu, 21 Jan 2010 11:28:25 +0100, Giraudon Cyril <cyril.giraudon@free.fr> wrote:
Wermer,
I don't express my mind correctly.
I 'd like to do strong coupling between applications (different
applications, different processes) without being too intrusive into the
applications. There would be some data exchanges at each step of the
algorithm (time domain computation, FDTD for instance) as with an MPI
solution.The amount of data is relatively low but I have to perform exchange the
fastest as possible. Amelet-HDF can be seen as a cross language IDL of
kind for electromagnetic computations. I 'd like a simple solution for
C/Fortran languages.I will consider your proposal, although I did not want to write a
complete server.Regards,
Cyril.
Le 20/01/2010 16:11, Werner Benger a écrit :
Hi Cyril,
this sounds like a nice project, similar to what we're doing as well
(file format: http://www.fiberbundle.net has similar intentions).It seems you would not use memcached itself, as it is only designed for
"small" chunks:"Memcached is an in-memory key-value store for small chunks of
arbitrary data
(strings, objects) from results of database calls, API calls, or page
rendering." ?Rather you'd had something like a server that retrieves complete HDF5
files
via socket and make them available to other clients, like on some NFS
or SSHFS
where the file written is closed after each operation, and thereby
"published"
for other clients to read the same file?Werner
On Wed, 20 Jan 2010 15:58:33 +0100, Giraudon Cyril >> <cyril.giraudon@free.fr> wrote:
Hi Wermer,
I work at the moment on a scientific format based on HDF5.
The format permits to describe an electromagnetic simulation and
electromagnetic objects like materials, mesh, result data ...
(if you are interested : http://groups.google.com/group/amelet-hdf)The idea would be to write data in memory and cache them in a cache
system like http://memcached.org/, to store and retore rapidly or to
share data with other processes.I imagine a weak coupling system for scientific applications.
Regards,
Cyril.
Le 20/01/2010 15:38, Werner Benger a écrit :
Hi Cyril,
there is a memory image driver in the standard HDF5 distribution, from
which this streaming file driver is derived, but the streaming file
driver
is currently not part of the HDF5 distribution.What do you mean by putting a "file into a cache system" ?
Werner
On Wed, 20 Jan 2010 13:21:15 +0100, Giraudon Cyril >>>> <cyril.giraudon@free.fr> wrote:
Thanks you Wermer,
Must I really use a new driver ?
I thought I could use the hdf5_core driver and then manipulate a C
structure.
What should I do if I would to put a memory HDF5 file into a cache
system ?Regards,
Cyril.
Le 20/01/2010 12:26, Werner Benger a écrit :
Hi Cyril,
this has been done as part of the German TIKSL project
( http://www.zib.de/visual/projects/TIKSL/ ) with the creation
of the HDF5 Streaming VFD driver, which as of now is available
throughhttp://hdf5-addons.origo.ethz.ch/
It is however currently not actively worked on, so might require
some code cleanup and revision, though it was recently working with
HDF5 1.8 .It does not use MPI, but directly operates on TCP sockets. Thereby
it does not conflict with an MPI application and can be used within
some MPI-parallelized code without affecting that one.Cheers,
WernerOn Wed, 20 Jan 2010 11:47:08 +0100, Giraudon Cyril >>>>>> <cyril.giraudon@free.fr> wrote:
Hello,
I have read HDF5 files can be in-memory structure.
That said, is it be possible to the retrieve this HDF5 file as a
binary
array and send it by the network (as a float array with MPI for
instance) ?Thanks a lot,
Cyril.
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
--
___________________________________________________________________________
Dr. Werner Benger Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
211 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809 Fax.: +1 225 578-5362
Hi All,
In HDF5, can I update an existing data item which already exists? I did
not find a way to firstly delete such item and then create a new item.
And H5Dcreate() will fail if such data item already exists. If I use
H5Dopen(), then it won't carry the new information, e.g. new dataspace.
I used H5P_DEFAULT as the data access property list parameter.
Would you please suggest a mechanism on how to do so? Thanks a lot.
Best,
xunlei
Hi Xunlei,
···
On Mar 5, 2010, at 1:24 PM, Xunlei Wu wrote:
Hi All,
In HDF5, can I update an existing data item which already exists? I did
not find a way to firstly delete such item and then create a new item.
And H5Dcreate() will fail if such data item already exists. If I use
H5Dopen(), then it won't carry the new information, e.g. new dataspace.
I used H5P_DEFAULT as the data access property list parameter.
Would you please suggest a mechanism on how to do so? Thanks a lot.
Try H5Ldelete() to remove an existing link to an object. If you want to change the dimensions of a dataset's dataspace, you should be using H5Dset_extent().
Quincey
Hi Quincey,
I didn't get the time test your suggestion until today.
Before I do so, could you please help me to update the mechanism? I used H5Gopen() to detect whether a group already exists, then use H5Gcreate() to create a new one if not.
//-----------
if((group_frame_id = H5Gopen(group_event_id, groupString, H5P_DEFAULT)) < 0){
group_frame_id = H5Gcreate(group_event_id, groupString, H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);
}
else{//such frame already exist, delete it
herr_t status = H5Ldelete(group_event_id, groupString, H5P_DEFAULT);
...
}
//-----------
Can I use H5Ldelete() to remove the group even if I do not have an existing link created by say H5Lcreate_hard()?
Thanks a lot for your time.
Best,
x
···
On 3/5/2010 4:16 PM, Quincey Koziol wrote:
Hi Xunlei,
On Mar 5, 2010, at 1:24 PM, Xunlei Wu wrote:
Hi All,
In HDF5, can I update an existing data item which already exists? I did
not find a way to firstly delete such item and then create a new item.
And H5Dcreate() will fail if such data item already exists. If I use
H5Dopen(), then it won't carry the new information, e.g. new dataspace.
I used H5P_DEFAULT as the data access property list parameter.
Would you please suggest a mechanism on how to do so? Thanks a lot.
Try H5Ldelete() to remove an existing link to an object. If you want to change the dimensions of a dataset's dataspace, you should be using H5Dset_extent().
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Hi Xunlei,
Hi Quincey,
I didn't get the time test your suggestion until today.
Before I do so, could you please help me to update the mechanism? I used H5Gopen() to detect whether a group already exists, then use H5Gcreate() to create a new one if not.
//-----------
if((group_frame_id = H5Gopen(group_event_id, groupString, H5P_DEFAULT)) < 0){
group_frame_id = H5Gcreate(group_event_id, groupString, H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);
}
else{//such frame already exist, delete it
herr_t status = H5Ldelete(group_event_id, groupString, H5P_DEFAULT);
...
}
//-----------
Can I use H5Ldelete() to remove the group even if I do not have an existing link created by say H5Lcreate_hard()?
You should use H5Lexists() to determine if a link to an object exists. H5Ldelete() will fail and return an error if there's no link with the given name.
Quincey
···
On Mar 30, 2010, at 4:06 PM, Dr. X wrote:
Thanks a lot for your time.
Best,
xOn 3/5/2010 4:16 PM, Quincey Koziol wrote:
Hi Xunlei,
On Mar 5, 2010, at 1:24 PM, Xunlei Wu wrote:
Hi All,
In HDF5, can I update an existing data item which already exists? I did
not find a way to firstly delete such item and then create a new item.
And H5Dcreate() will fail if such data item already exists. If I use
H5Dopen(), then it won't carry the new information, e.g. new dataspace.
I used H5P_DEFAULT as the data access property list parameter.
Would you please suggest a mechanism on how to do so? Thanks a lot.
Try H5Ldelete() to remove an existing link to an object. If you want to change the dimensions of a dataset's dataspace, you should be using H5Dset_extent().
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Hi Quincey,
When I use H5Ldelete() to remove a group item, do I need to use H5Ldelete() to remove all datasets under that group as well? I only remove the group item, but seems the file size is keep getting bigger.
Is there a way to iterate through all the children of a group item?
Best,
x
···
On 3/31/2010 9:50 AM, Quincey Koziol wrote:
Hi Xunlei,
On Mar 30, 2010, at 4:06 PM, Dr. X wrote:
Hi Quincey,
I didn't get the time test your suggestion until today.
Before I do so, could you please help me to update the mechanism? I used H5Gopen() to detect whether a group already exists, then use H5Gcreate() to create a new one if not.
//-----------
if((group_frame_id = H5Gopen(group_event_id, groupString, H5P_DEFAULT))< 0){
group_frame_id = H5Gcreate(group_event_id, groupString, H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);
}
else{//such frame already exist, delete it
herr_t status = H5Ldelete(group_event_id, groupString, H5P_DEFAULT);
...
}
//-----------
Can I use H5Ldelete() to remove the group even if I do not have an existing link created by say H5Lcreate_hard()?
You should use H5Lexists() to determine if a link to an object exists. H5Ldelete() will fail and return an error if there's no link with the given name.
Quincey
Thanks a lot for your time.
Best,
xOn 3/5/2010 4:16 PM, Quincey Koziol wrote:
Hi Xunlei,
On Mar 5, 2010, at 1:24 PM, Xunlei Wu wrote:
Hi All,
In HDF5, can I update an existing data item which already exists? I did
not find a way to firstly delete such item and then create a new item.
And H5Dcreate() will fail if such data item already exists. If I use
H5Dopen(), then it won't carry the new information, e.g. new dataspace.
I used H5P_DEFAULT as the data access property list parameter.
Would you please suggest a mechanism on how to do so? Thanks a lot.Try H5Ldelete() to remove an existing link to an object. If you want to change the dimensions of a dataset's dataspace, you should be using H5Dset_extent().
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
Hi Xunlei,
Hi Quincey,
When I use H5Ldelete() to remove a group item, do I need to use H5Ldelete() to remove all datasets under that group as well? I only remove the group item, but seems the file size is keep getting bigger.
Deleting the link to a group should remove all the links to the datasets within it (and therefore the datasets themselves). If the file continues to get bigger, you probably have open IDs to the datasets, which is preventing them from being removed from the file (until the file is closed or the application exits).
Is there a way to iterate through all the children of a group item?
Yes, you can use H5Literate() to iterate over all links in a group. You may also want to look at H5Lvisit() and H5Ovisit() as well.
Quincey
···
On Apr 14, 2010, at 2:11 PM, Dr. X wrote:
Best,
xOn 3/31/2010 9:50 AM, Quincey Koziol wrote:
Hi Xunlei,
On Mar 30, 2010, at 4:06 PM, Dr. X wrote:
Hi Quincey,
I didn't get the time test your suggestion until today.
Before I do so, could you please help me to update the mechanism? I used H5Gopen() to detect whether a group already exists, then use H5Gcreate() to create a new one if not.
//-----------
if((group_frame_id = H5Gopen(group_event_id, groupString, H5P_DEFAULT))< 0){
group_frame_id = H5Gcreate(group_event_id, groupString, H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);
}
else{//such frame already exist, delete it
herr_t status = H5Ldelete(group_event_id, groupString, H5P_DEFAULT);
...
}
//-----------
Can I use H5Ldelete() to remove the group even if I do not have an existing link created by say H5Lcreate_hard()?
You should use H5Lexists() to determine if a link to an object exists. H5Ldelete() will fail and return an error if there's no link with the given name.
Quincey
Thanks a lot for your time.
Best,
xOn 3/5/2010 4:16 PM, Quincey Koziol wrote:
Hi Xunlei,
On Mar 5, 2010, at 1:24 PM, Xunlei Wu wrote:
Hi All,
In HDF5, can I update an existing data item which already exists? I did
not find a way to firstly delete such item and then create a new item.
And H5Dcreate() will fail if such data item already exists. If I use
H5Dopen(), then it won't carry the new information, e.g. new dataspace.
I used H5P_DEFAULT as the data access property list parameter.
Would you please suggest a mechanism on how to do so? Thanks a lot.Try H5Ldelete() to remove an existing link to an object. If you want to change the dimensions of a dataset's dataspace, you should be using H5Dset_extent().
Quincey
_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org