creating a custom VOL driver for HDF5

Hdf-forum archives
1

Hi all,

I am working in a research project at Brandeis University which aims to
provide snapshot support in the HDF5 file system. The idea is to use
Berkeley DB with snapshot support to store the HDF5 metadata. I was
wondering what is the more efficient and modular way to do it, instead of
modifying the core code of HDF5. Is there a way to replace the standard VOL
(or VFL) responsible for writing the data and metadata on disk with a new
VOL which will not write the metadata in regular format, but send it to
Berkeley DB instead?

Thanks a lot in advance,

Dimos

2

Hi Dimos,
  To enabling writing of HDF5 metadata to Berkelely DB, I'd think VOL wold be the way to go. You can find instructions on creating a VOL here: http://hdf-forum.184993.n3.nabble.com/How-can-I-start-a-plugin-for-quot-Virtual-Object-Layer-quot-td4025766.html.

  Note that the VOL is not part of the standard HDF5 release, and as it stands now, is not planned for the 1.10 release.

  Alternatively, could you have a a background process that reads the HDF5 file and copies meta data to the BDB?

Cheers,
John

···

From: Hdf-forum on behalf of Dimos Stamatakis
Reply-To: HDF Users Discussion List
Date: Wednesday, September 9, 2015 at 3:41 PM
To: "hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>"
Subject: [Hdf-forum] creating a custom VOL driver for HDF5

Hi all,

I am working in a research project at Brandeis University which aims to provide snapshot support in the HDF5 file system. The idea is to use Berkeley DB with snapshot support to store the HDF5 metadata. I was wondering what is the more efficient and modular way to do it, instead of modifying the core code of HDF5. Is there a way to replace the standard VOL (or VFL) responsible for writing the data and metadata on disk with a new VOL which will not write the metadata in regular format, but send it to Berkeley DB instead?

Thanks a lot in advance,

Dimos

3

Hi John,

thanks for your response!

I think creating a custom VOL would better match our needs, since we have
to support on-demand snapshot creation.

So, creating a custom VOL means creating a custom VOL plugin, as described
here?
https://fossies.org/linux/misc/hdf5-1.9.229_docs.tar.gz/hdf5-1.9.229_docs/RFCs/HDF5/VOL/2012-05-25-RFC_VOL.docx

Is there a guide for building a custom VOL plugin, something equivalent to
the one for building custom VFL?
https://www.hdfgroup.org/HDF5/doc/TechNotes/VFL.html#SEC7

Thanks,

Dimos

···

On Mon, Sep 14, 2015 at 1:00 PM, <hdf-forum-request@lists.hdfgroup.org> wrote:

Send Hdf-forum mailing list submissions to
        hdf-forum@lists.hdfgroup.org

To subscribe or unsubscribe via the World Wide Web, visit

http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

or, via email, send a message with subject or body 'help' to
        hdf-forum-request@lists.hdfgroup.org

You can reach the person managing the list at
        hdf-forum-owner@lists.hdfgroup.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Hdf-forum digest..."

Today's Topics:

   1. Re: creating a custom VOL driver for HDF5 (John Readey)

----------------------------------------------------------------------

Message: 1
Date: Mon, 14 Sep 2015 14:27:00 +0000
From: John Readey <jreadey@hdfgroup.org>
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
Subject: Re: [Hdf-forum] creating a custom VOL driver for HDF5
Message-ID: <DFFD8FF0-1B54-4250-851B-14C6DD1E8168@hdfgroup.org>
Content-Type: text/plain; charset="utf-8"

Hi Dimos,
  To enabling writing of HDF5 metadata to Berkelely DB, I'd think VOL wold
be the way to go. You can find instructions on creating a VOL here:
http://hdf-forum.184993.n3.nabble.com/How-can-I-start-a-plugin-for-quot-Virtual-Object-Layer-quot-td4025766.html
.

  Note that the VOL is not part of the standard HDF5 release, and as it
stands now, is not planned for the 1.10 release.

  Alternatively, could you have a a background process that reads the HDF5
file and copies meta data to the BDB?

Cheers,
John

From: Hdf-forum on behalf of Dimos Stamatakis
Reply-To: HDF Users Discussion List
Date: Wednesday, September 9, 2015 at 3:41 PM
To: "hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>"
Subject: [Hdf-forum] creating a custom VOL driver for HDF5

Hi all,

I am working in a research project at Brandeis University which aims to
provide snapshot support in the HDF5 file system. The idea is to use
Berkeley DB with snapshot support to store the HDF5 metadata. I was
wondering what is the more efficient and modular way to do it, instead of
modifying the core code of HDF5. Is there a way to replace the standard VOL
(or VFL) responsible for writing the data and metadata on disk with a new
VOL which will not write the metadata in regular format, but send it to
Berkeley DB instead?

Thanks a lot in advance,

Dimos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/attachments/20150914/daa123a0/attachment-0001.html
>

------------------------------

Subject: Digest Footer

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

------------------------------

End of Hdf-forum Digest, Vol 75, Issue 15
*****************************************

4

Hi Dimos,

  I don't believe there is any guide similar to the VFL guide.

  Be aware that the scale of effort to write a VOL plugin is much greater than with the VFL.

  This forum will be a good place to go if you have questions.

Cheers,
John

···

From: Hdf-forum on behalf of Dimos Stamatakis
Reply-To: HDF Users Discussion List
Date: Monday, September 14, 2015 at 10:26 PM
To: "hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>"
Subject: Re: [Hdf-forum] Hdf-forum Digest, Vol 75, Issue 15 - creating a custom VOL driver for HDF5

Hi John,

thanks for your response!

I think creating a custom VOL would better match our needs, since we have to support on-demand snapshot creation.

So, creating a custom VOL means creating a custom VOL plugin, as described here?
https://fossies.org/linux/misc/hdf5-1.9.229_docs.tar.gz/hdf5-1.9.229_docs/RFCs/HDF5/VOL/2012-05-25-RFC_VOL.docx

Is there a guide for building a custom VOL plugin, something equivalent to the one for building custom VFL?
https://www.hdfgroup.org/HDF5/doc/TechNotes/VFL.html#SEC7

Thanks,

Dimos

On Mon, Sep 14, 2015 at 1:00 PM, <hdf-forum-request@lists.hdfgroup.org<mailto:hdf-forum-request@lists.hdfgroup.org>> wrote:
Send Hdf-forum mailing list submissions to
        hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>

To subscribe or unsubscribe via the World Wide Web, visit
        http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

or, via email, send a message with subject or body 'help' to
        hdf-forum-request@lists.hdfgroup.org<mailto:hdf-forum-request@lists.hdfgroup.org>

You can reach the person managing the list at
        hdf-forum-owner@lists.hdfgroup.org<mailto:hdf-forum-owner@lists.hdfgroup.org>

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Hdf-forum digest..."

Today's Topics:

   1. Re: creating a custom VOL driver for HDF5 (John Readey)

----------------------------------------------------------------------

Message: 1
Date: Mon, 14 Sep 2015 14:27:00 +0000
From: John Readey <jreadey@hdfgroup.org<mailto:jreadey@hdfgroup.org>>
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: Re: [Hdf-forum] creating a custom VOL driver for HDF5
Message-ID: <DFFD8FF0-1B54-4250-851B-14C6DD1E8168@hdfgroup.org<mailto:DFFD8FF0-1B54-4250-851B-14C6DD1E8168@hdfgroup.org>>
Content-Type: text/plain; charset="utf-8"

Hi Dimos,
  To enabling writing of HDF5 metadata to Berkelely DB, I'd think VOL wold be the way to go. You can find instructions on creating a VOL here: http://hdf-forum.184993.n3.nabble.com/How-can-I-start-a-plugin-for-quot-Virtual-Object-Layer-quot-td4025766.html.

  Note that the VOL is not part of the standard HDF5 release, and as it stands now, is not planned for the 1.10 release.

  Alternatively, could you have a a background process that reads the HDF5 file and copies meta data to the BDB?

Cheers,
John

From: Hdf-forum on behalf of Dimos Stamatakis
Reply-To: HDF Users Discussion List
Date: Wednesday, September 9, 2015 at 3:41 PM
To: "hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org><mailto:hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>"
Subject: [Hdf-forum] creating a custom VOL driver for HDF5

Hi all,

I am working in a research project at Brandeis University which aims to provide snapshot support in the HDF5 file system. The idea is to use Berkeley DB with snapshot support to store the HDF5 metadata. I was wondering what is the more efficient and modular way to do it, instead of modifying the core code of HDF5. Is there a way to replace the standard VOL (or VFL) responsible for writing the data and metadata on disk with a new VOL which will not write the metadata in regular format, but send it to Berkeley DB instead?

Thanks a lot in advance,

Dimos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/attachments/20150914/daa123a0/attachment-0001.html>

------------------------------

Subject: Digest Footer

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

------------------------------

End of Hdf-forum Digest, Vol 75, Issue 15
*****************************************

5

Here are the more “detailed” docs for creating a VOL (look for the pdf under each link):

code:
http://svn.hdfgroup.uiuc.edu/hdf5/features/vol
RFC/High Level Architecture Design:
http://svn.hdfgroup.uiuc.edu/hdf5doc/trunk/RFCs/HDF5/VOL/RFC/
Developers Guide / Low Level Architecture Design:
http://svn.hdfgroup.uiuc.edu/hdf5doc/trunk/RFCs/HDF5/VOL/developer_guide/
Users Guide / RM:
http://svn.hdfgroup.uiuc.edu/hdf5doc/trunk/RFCs/HDF5/VOL/user_guide/

Thanks,
Mohamad

···

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Dimos Stamatakis
Sent: Tuesday, September 15, 2015 12:26 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] Hdf-forum Digest, Vol 75, Issue 15 - creating a custom VOL driver for HDF5

Hi John,

thanks for your response!

I think creating a custom VOL would better match our needs, since we have to support on-demand snapshot creation.

So, creating a custom VOL means creating a custom VOL plugin, as described here?
https://fossies.org/linux/misc/hdf5-1.9.229_docs.tar.gz/hdf5-1.9.229_docs/RFCs/HDF5/VOL/2012-05-25-RFC_VOL.docx

Is there a guide for building a custom VOL plugin, something equivalent to the one for building custom VFL?
https://www.hdfgroup.org/HDF5/doc/TechNotes/VFL.html#SEC7

Thanks,

Dimos

On Mon, Sep 14, 2015 at 1:00 PM, <hdf-forum-request@lists.hdfgroup.org<mailto:hdf-forum-request@lists.hdfgroup.org>> wrote:
Send Hdf-forum mailing list submissions to
        hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>

To subscribe or unsubscribe via the World Wide Web, visit
        http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

or, via email, send a message with subject or body 'help' to
        hdf-forum-request@lists.hdfgroup.org<mailto:hdf-forum-request@lists.hdfgroup.org>

You can reach the person managing the list at
        hdf-forum-owner@lists.hdfgroup.org<mailto:hdf-forum-owner@lists.hdfgroup.org>

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Hdf-forum digest..."

Today's Topics:

   1. Re: creating a custom VOL driver for HDF5 (John Readey)

----------------------------------------------------------------------

Message: 1
Date: Mon, 14 Sep 2015 14:27:00 +0000
From: John Readey <jreadey@hdfgroup.org<mailto:jreadey@hdfgroup.org>>
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>
Subject: Re: [Hdf-forum] creating a custom VOL driver for HDF5
Message-ID: <DFFD8FF0-1B54-4250-851B-14C6DD1E8168@hdfgroup.org<mailto:DFFD8FF0-1B54-4250-851B-14C6DD1E8168@hdfgroup.org>>
Content-Type: text/plain; charset="utf-8"

Hi Dimos,
  To enabling writing of HDF5 metadata to Berkelely DB, I'd think VOL wold be the way to go. You can find instructions on creating a VOL here: http://hdf-forum.184993.n3.nabble.com/How-can-I-start-a-plugin-for-quot-Virtual-Object-Layer-quot-td4025766.html.

  Note that the VOL is not part of the standard HDF5 release, and as it stands now, is not planned for the 1.10 release.

  Alternatively, could you have a a background process that reads the HDF5 file and copies meta data to the BDB?

Cheers,
John

From: Hdf-forum on behalf of Dimos Stamatakis
Reply-To: HDF Users Discussion List
Date: Wednesday, September 9, 2015 at 3:41 PM
To: "hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org><mailto:hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>>"
Subject: [Hdf-forum] creating a custom VOL driver for HDF5

Hi all,

I am working in a research project at Brandeis University which aims to provide snapshot support in the HDF5 file system. The idea is to use Berkeley DB with snapshot support to store the HDF5 metadata. I was wondering what is the more efficient and modular way to do it, instead of modifying the core code of HDF5. Is there a way to replace the standard VOL (or VFL) responsible for writing the data and metadata on disk with a new VOL which will not write the metadata in regular format, but send it to Berkeley DB instead?

Thanks a lot in advance,

Dimos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/attachments/20150914/daa123a0/attachment-0001.html>

------------------------------

Subject: Digest Footer

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

------------------------------

End of Hdf-forum Digest, Vol 75, Issue 15
*****************************************

6

Thanks a lot for your pointers!

So the code contains both VOL and core HDF5, right?

Also, I cannot find the code for the Metadata Server plugin you describe in
the RFC. Is it included in the distribution?

Thanks,

Dimos

···

On Tue, Sep 15, 2015 at 1:00 PM, <hdf-forum-request@lists.hdfgroup.org> wrote:

Send Hdf-forum mailing list submissions to
        hdf-forum@lists.hdfgroup.org

To subscribe or unsubscribe via the World Wide Web, visit

http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

or, via email, send a message with subject or body 'help' to
        hdf-forum-request@lists.hdfgroup.org

You can reach the person managing the list at
        hdf-forum-owner@lists.hdfgroup.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Hdf-forum digest..."

Today's Topics:

   1. Re: Hdf-forum Digest, Vol 75, Issue 15 - creating a custom
      VOL driver for HDF5 (Mohamad Chaarawi)

----------------------------------------------------------------------

Message: 1
Date: Tue, 15 Sep 2015 13:45:30 +0000
From: Mohamad Chaarawi <chaarawi@hdfgroup.org>
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
Subject: Re: [Hdf-forum] Hdf-forum Digest, Vol 75, Issue 15 - creating
        a custom VOL driver for HDF5
Message-ID:
        <
BY2PR0701MB1957117BD36BDB700B019864BD5C0@BY2PR0701MB1957.namprd07.prod.outlook.com
>

Content-Type: text/plain; charset="utf-8"

Here are the more ?detailed? docs for creating a VOL (look for the pdf
under each link):

code:
http://svn.hdfgroup.uiuc.edu/hdf5/features/vol
RFC/High Level Architecture Design:
http://svn.hdfgroup.uiuc.edu/hdf5doc/trunk/RFCs/HDF5/VOL/RFC/
Developers Guide / Low Level Architecture Design:
http://svn.hdfgroup.uiuc.edu/hdf5doc/trunk/RFCs/HDF5/VOL/developer_guide/
Users Guide / RM:
http://svn.hdfgroup.uiuc.edu/hdf5doc/trunk/RFCs/HDF5/VOL/user_guide/

Thanks,
Mohamad

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf
Of Dimos Stamatakis
Sent: Tuesday, September 15, 2015 12:26 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] Hdf-forum Digest, Vol 75, Issue 15 - creating a
custom VOL driver for HDF5

Hi John,

thanks for your response!

I think creating a custom VOL would better match our needs, since we have
to support on-demand snapshot creation.

So, creating a custom VOL means creating a custom VOL plugin, as described
here?

https://fossies.org/linux/misc/hdf5-1.9.229_docs.tar.gz/hdf5-1.9.229_docs/RFCs/HDF5/VOL/2012-05-25-RFC_VOL.docx

Is there a guide for building a custom VOL plugin, something equivalent to
the one for building custom VFL?
https://www.hdfgroup.org/HDF5/doc/TechNotes/VFL.html#SEC7

Thanks,

Dimos

On Mon, Sep 14, 2015 at 1:00 PM, <hdf-forum-request@lists.hdfgroup.org > <mailto:hdf-forum-request@lists.hdfgroup.org>> wrote:
Send Hdf-forum mailing list submissions to
        hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org>

To subscribe or unsubscribe via the World Wide Web, visit

http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

or, via email, send a message with subject or body 'help' to
        hdf-forum-request@lists.hdfgroup.org<mailto:
hdf-forum-request@lists.hdfgroup.org>

You can reach the person managing the list at
        hdf-forum-owner@lists.hdfgroup.org<mailto:
hdf-forum-owner@lists.hdfgroup.org>

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Hdf-forum digest..."

Today's Topics:

   1. Re: creating a custom VOL driver for HDF5 (John Readey)

----------------------------------------------------------------------

Message: 1
Date: Mon, 14 Sep 2015 14:27:00 +0000
From: John Readey <jreadey@hdfgroup.org<mailto:jreadey@hdfgroup.org>>
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org<mailto:
hdf-forum@lists.hdfgroup.org>>
Subject: Re: [Hdf-forum] creating a custom VOL driver for HDF5
Message-ID: <DFFD8FF0-1B54-4250-851B-14C6DD1E8168@hdfgroup.org<mailto:
DFFD8FF0-1B54-4250-851B-14C6DD1E8168@hdfgroup.org>>
Content-Type: text/plain; charset="utf-8"

Hi Dimos,
  To enabling writing of HDF5 metadata to Berkelely DB, I'd think VOL wold
be the way to go. You can find instructions on creating a VOL here:
http://hdf-forum.184993.n3.nabble.com/How-can-I-start-a-plugin-for-quot-Virtual-Object-Layer-quot-td4025766.html
.

  Note that the VOL is not part of the standard HDF5 release, and as it
stands now, is not planned for the 1.10 release.

  Alternatively, could you have a a background process that reads the HDF5
file and copies meta data to the BDB?

Cheers,
John

From: Hdf-forum on behalf of Dimos Stamatakis
Reply-To: HDF Users Discussion List
Date: Wednesday, September 9, 2015 at 3:41 PM
To: "hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org
><mailto:hdf-forum@lists.hdfgroup.org<mailto:hdf-forum@lists.hdfgroup.org
>>"
Subject: [Hdf-forum] creating a custom VOL driver for HDF5

Hi all,

I am working in a research project at Brandeis University which aims to
provide snapshot support in the HDF5 file system. The idea is to use
Berkeley DB with snapshot support to store the HDF5 metadata. I was
wondering what is the more efficient and modular way to do it, instead of
modifying the core code of HDF5. Is there a way to replace the standard VOL
(or VFL) responsible for writing the data and metadata on disk with a new
VOL which will not write the metadata in regular format, but send it to
Berkeley DB instead?

Thanks a lot in advance,

Dimos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/attachments/20150914/daa123a0/attachment-0001.html
>

------------------------------

Subject: Digest Footer

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org<mailto:Hdf-forum@lists.hdfgroup.org>
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

------------------------------

End of Hdf-forum Digest, Vol 75, Issue 15
*****************************************

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/attachments/20150915/1dccbdb6/attachment-0001.html
>

------------------------------

Subject: Digest Footer

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

------------------------------

End of Hdf-forum Digest, Vol 75, Issue 17
*****************************************

7

Hi Dimos,

···

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Dimos Stamatakis
Sent: Thursday, September 17, 2015 12:54 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] Hdf-forum Digest, Vol 75, Issue 17 - creating a custom VOL driver for HDF5

Thanks a lot for your pointers!

So the code contains both VOL and core HDF5, right?

[msc] yes. The src/H5VLnative.c is the plugin that implements the native HDF5 library.

Also, I cannot find the code for the Metadata Server plugin you describe in the RFC. Is it included in the distribution?

[msc] no it is not included. This is a very outdated plugin anyway and highly unlikely to make it into production. You can check out the h5netvol plugin, which is also a little outdated since it probably hasn’t been updated with the changes we made recently to the VOL architecture and callbacks, but will still serve as another good example.

https://svn.hdfgroup.uiuc.edu/h5netvol/

Thanks,
Mohamad

8

Hi Mohamad,

Our approach will simply treat metadata different than data, so which
plugin you think will help us with this distinction? Or is there a simple
way to see in the code when is metadata accessed, and when is data?

Thanks,

Dimos

···

On Sep 17, 2015 1:00 PM, <hdf-forum-request@lists.hdfgroup.org> wrote:

Send Hdf-forum mailing list submissions to
        hdf-forum@lists.hdfgroup.org

To subscribe or unsubscribe via the World Wide Web, visit

http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

or, via email, send a message with subject or body 'help' to
        hdf-forum-request@lists.hdfgroup.org

You can reach the person managing the list at
        hdf-forum-owner@lists.hdfgroup.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Hdf-forum digest..."

Today's Topics:

   1. Re: Hdf-forum Digest, Vol 75, Issue 17 - creating a custom
      VOL driver for HDF5 (Mohamad Chaarawi)

----------------------------------------------------------------------

Message: 1
Date: Thu, 17 Sep 2015 13:03:51 +0000
From: Mohamad Chaarawi <chaarawi@hdfgroup.org>
To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
Subject: Re: [Hdf-forum] Hdf-forum Digest, Vol 75, Issue 17 - creating
        a custom VOL driver for HDF5
Message-ID:
        <
BY2PR0701MB1957CBFF9D008CACE289B64ABD5A0@BY2PR0701MB1957.namprd07.prod.outlook.com
>

Content-Type: text/plain; charset="utf-8"

Hi Dimos,

From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf
Of Dimos Stamatakis
Sent: Thursday, September 17, 2015 12:54 AM
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] Hdf-forum Digest, Vol 75, Issue 17 - creating a
custom VOL driver for HDF5

Thanks a lot for your pointers!

So the code contains both VOL and core HDF5, right?

[msc] yes. The src/H5VLnative.c is the plugin that implements the native
HDF5 library.

Also, I cannot find the code for the Metadata Server plugin you describe
in the RFC. Is it included in the distribution?

[msc] no it is not included. This is a very outdated plugin anyway and
highly unlikely to make it into production. You can check out the h5netvol
plugin, which is also a little outdated since it probably hasn?t been
updated with the changes we made recently to the VOL architecture and
callbacks, but will still serve as another good example.

https://svn.hdfgroup.uiuc.edu/h5netvol/

Thanks,
Mohamad

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/attachments/20150917/2b300a87/attachment-0001.html
>

------------------------------

Subject: Digest Footer

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

------------------------------

End of Hdf-forum Digest, Vol 75, Issue 20
*****************************************

9

Hello,

I didn't get any replies yet, so I was wondering if I can get information
about my concern.

Is there any VOL plugin which separates data from metadata? If no, is it
straightforward to just look at the default VOL plugin code and find the
data/metadata separation?

Thanks,

Dimos

···

On Fri, Sep 18, 2015 at 1:00 PM, <hdf-forum-request@lists.hdfgroup.org> wrote:

Send Hdf-forum mailing list submissions to
        hdf-forum@lists.hdfgroup.org

To subscribe or unsubscribe via the World Wide Web, visit

http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

or, via email, send a message with subject or body 'help' to
        hdf-forum-request@lists.hdfgroup.org

You can reach the person managing the list at
        hdf-forum-owner@lists.hdfgroup.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Hdf-forum digest..."

Today's Topics:

   1. Re: Hdf-forum Digest, Vol 75, Issue 20 - creating a custom
      VOL driver for HDF5 (Dimos Stamatakis)

----------------------------------------------------------------------

Message: 1
Date: Thu, 17 Sep 2015 13:18:27 -0400
From: Dimos Stamatakis <dimstamat@gmail.com>
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] Hdf-forum Digest, Vol 75, Issue 20 - creating
        a custom VOL driver for HDF5
Message-ID:
        <
CAB8CW0EKN1xh0MzFFeA8rQjN6+HQ4yGXvxT+CAWGo3b7sMQEng@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Mohamad,

Our approach will simply treat metadata different than data, so which
plugin you think will help us with this distinction? Or is there a simple
way to see in the code when is metadata accessed, and when is data?

Thanks,

Dimos
On Sep 17, 2015 1:00 PM, <hdf-forum-request@lists.hdfgroup.org> wrote:

> Send Hdf-forum mailing list submissions to
> hdf-forum@lists.hdfgroup.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>
> or, via email, send a message with subject or body 'help' to
> hdf-forum-request@lists.hdfgroup.org
>
> You can reach the person managing the list at
> hdf-forum-owner@lists.hdfgroup.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Hdf-forum digest..."
>
>
> Today's Topics:
>
> 1. Re: Hdf-forum Digest, Vol 75, Issue 17 - creating a custom
> VOL driver for HDF5 (Mohamad Chaarawi)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 17 Sep 2015 13:03:51 +0000
> From: Mohamad Chaarawi <chaarawi@hdfgroup.org>
> To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
> Subject: Re: [Hdf-forum] Hdf-forum Digest, Vol 75, Issue 17 - creating
> a custom VOL driver for HDF5
> Message-ID:
> <
>
BY2PR0701MB1957CBFF9D008CACE289B64ABD5A0@BY2PR0701MB1957.namprd07.prod.outlook.com
> >
>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Dimos,
>
> From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf
> Of Dimos Stamatakis
> Sent: Thursday, September 17, 2015 12:54 AM
> To: hdf-forum@lists.hdfgroup.org
> Subject: Re: [Hdf-forum] Hdf-forum Digest, Vol 75, Issue 17 - creating a
> custom VOL driver for HDF5
>
> Thanks a lot for your pointers!
>
> So the code contains both VOL and core HDF5, right?
>
> [msc] yes. The src/H5VLnative.c is the plugin that implements the native
> HDF5 library.
>
> Also, I cannot find the code for the Metadata Server plugin you describe
> in the RFC. Is it included in the distribution?
>
> [msc] no it is not included. This is a very outdated plugin anyway and
> highly unlikely to make it into production. You can check out the
h5netvol
> plugin, which is also a little outdated since it probably hasn?t been
> updated with the changes we made recently to the VOL architecture and
> callbacks, but will still serve as another good example.
>
> https://svn.hdfgroup.uiuc.edu/h5netvol/
>
> Thanks,
> Mohamad
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
>
http://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/attachments/20150917/2b300a87/attachment-0001.html
> >
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> Hdf-forum@lists.hdfgroup.org
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>
>
> ------------------------------
>
> End of Hdf-forum Digest, Vol 75, Issue 20
> *****************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/attachments/20150917/fac20f64/attachment-0001.html
>

------------------------------

Subject: Digest Footer

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

------------------------------

End of Hdf-forum Digest, Vol 75, Issue 21
*****************************************

10

Hello,

I didn't get any replies yet, so I was wondering if I can get information
about my concern.

Is there any VOL plugin which separates data from metadata? If no, is it
straightforward to just look at the default VOL plugin code and find the
data/metadata separation?

Thanks,

Dimos

···

On Fri, Sep 18, 2015 at 1:00 PM, <hdf-forum-request@lists.hdfgroup.org> wrote:

Send Hdf-forum mailing list submissions to
        hdf-forum@lists.hdfgroup.org

To subscribe or unsubscribe via the World Wide Web, visit

http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

or, via email, send a message with subject or body 'help' to
        hdf-forum-request@lists.hdfgroup.org

You can reach the person managing the list at
        hdf-forum-owner@lists.hdfgroup.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Hdf-forum digest..."

Today's Topics:

   1. Re: Hdf-forum Digest, Vol 75, Issue 20 - creating a custom
      VOL driver for HDF5 (Dimos Stamatakis)

----------------------------------------------------------------------

Message: 1
Date: Thu, 17 Sep 2015 13:18:27 -0400
From: Dimos Stamatakis <dimstamat@gmail.com>
To: hdf-forum@lists.hdfgroup.org
Subject: Re: [Hdf-forum] Hdf-forum Digest, Vol 75, Issue 20 - creating
        a custom VOL driver for HDF5
Message-ID:
        <
CAB8CW0EKN1xh0MzFFeA8rQjN6+HQ4yGXvxT+CAWGo3b7sMQEng@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Mohamad,

Our approach will simply treat metadata different than data, so which
plugin you think will help us with this distinction? Or is there a simple
way to see in the code when is metadata accessed, and when is data?

Thanks,

Dimos
On Sep 17, 2015 1:00 PM, <hdf-forum-request@lists.hdfgroup.org> wrote:

> Send Hdf-forum mailing list submissions to
> hdf-forum@lists.hdfgroup.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>
> or, via email, send a message with subject or body 'help' to
> hdf-forum-request@lists.hdfgroup.org
>
> You can reach the person managing the list at
> hdf-forum-owner@lists.hdfgroup.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Hdf-forum digest..."
>
>
> Today's Topics:
>
> 1. Re: Hdf-forum Digest, Vol 75, Issue 17 - creating a custom
> VOL driver for HDF5 (Mohamad Chaarawi)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 17 Sep 2015 13:03:51 +0000
> From: Mohamad Chaarawi <chaarawi@hdfgroup.org>
> To: HDF Users Discussion List <hdf-forum@lists.hdfgroup.org>
> Subject: Re: [Hdf-forum] Hdf-forum Digest, Vol 75, Issue 17 - creating
> a custom VOL driver for HDF5
> Message-ID:
> <
>
BY2PR0701MB1957CBFF9D008CACE289B64ABD5A0@BY2PR0701MB1957.namprd07.prod.outlook.com
> >
>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Dimos,
>
> From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf
> Of Dimos Stamatakis
> Sent: Thursday, September 17, 2015 12:54 AM
> To: hdf-forum@lists.hdfgroup.org
> Subject: Re: [Hdf-forum] Hdf-forum Digest, Vol 75, Issue 17 - creating a
> custom VOL driver for HDF5
>
> Thanks a lot for your pointers!
>
> So the code contains both VOL and core HDF5, right?
>
> [msc] yes. The src/H5VLnative.c is the plugin that implements the native
> HDF5 library.
>
> Also, I cannot find the code for the Metadata Server plugin you describe
> in the RFC. Is it included in the distribution?
>
> [msc] no it is not included. This is a very outdated plugin anyway and
> highly unlikely to make it into production. You can check out the
h5netvol
> plugin, which is also a little outdated since it probably hasn?t been
> updated with the changes we made recently to the VOL architecture and
> callbacks, but will still serve as another good example.
>
> https://svn.hdfgroup.uiuc.edu/h5netvol/
>
> Thanks,
> Mohamad
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
>
http://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/attachments/20150917/2b300a87/attachment-0001.html
> >
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> Hdf-forum@lists.hdfgroup.org
> http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
>
>
> ------------------------------
>
> End of Hdf-forum Digest, Vol 75, Issue 20
> *****************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://lists.hdfgroup.org/pipermail/hdf-forum_lists.hdfgroup.org/attachments/20150917/fac20f64/attachment-0001.html
>

------------------------------

Subject: Digest Footer

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org

------------------------------

End of Hdf-forum Digest, Vol 75, Issue 21
*****************************************