How can I start a plugin for "Virtual Object Layer"

Hdf-forum archives
1

Hi all,
     I am planing to conduct some research on I/O optimization like “A Plugin for HDF5 using PLFS for Improved I/O Performance and Semantic Analysis”, and I need to write a plugin for "Virtual Object Layer". I have already downloaded and installed the hdf5-1.8.10 source code. However, I have not found anyplace/guide to start a new plugin. Would you all please give me some guide or help ? Thanks in advance !

Bin Dong

···

--

2

Hi Bin,

The VOL work is not part of any release yet and still sitting in a separate branch. We are aiming to release with 1.10.

You can access the branch from:
http://svn.hdfgroup.uiuc.edu/hdf5/features/vol

I have to warn you that this is not a release branch, but a development branch, so things will change in the future. The VOL interface has been stable for a while now, however with some changes/additions expected to the HDF5 library interface, the VOL interface callbacks will change.

I also attached a sample C file that serves as a guide for adding an external plugin. The H5VLnative.c under src/ would serve as a guide for implementing an internal library specific plugin (which I don't think would be your case).

Thanks,
Mohamad

vol_test.c (17.1 KB)

···

On 1/7/2013 10:43 PM, goon83 wrote:

Hi all,
     I am planing to conduct some research on I/O optimization like "A Plugin for HDF5 using PLFS for Improved I/O Performance and Semantic Analysis", and I need to write a plugin for "Virtual Object Layer". I have already downloaded and installed the hdf5-1.8.10 source code. However, I have not found anyplace/guide to start a new plugin. Would you all please give me some guide or help ? Thanks in advance !

Bin Dong

--
365??? <http://gongyi.163.com/love365&gt;

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

3

Dear Mohamad,
Thanks for your information and sample file. I have downloaded the source code and the sample file, and will explore them in a few days.

Best wishes,

Bin Dong

···

--

At 2013-01-08 23:24:14,"Mohamad Chaarawi" <chaarawi@hdfgroup.org> wrote:

Hi Bin,

The VOL work is not part of any release yet and still sitting in a separate branch. We are aiming to release with 1.10.

You can access the branch from:
http://svn.hdfgroup.uiuc.edu/hdf5/features/vol

I have to warn you that this is not a release branch, but a development branch, so things will change in the future. The VOL interface has been stable for a while now, however with some changes/additions expected to the HDF5 library interface, the VOL interface callbacks will change.

I also attached a sample C file that serves as a guide for adding an external plugin. The H5VLnative.c under src/ would serve as a guide for implementing an internal library specific plugin (which I don't think would be your case).

Thanks,
Mohamad

On 1/7/2013 10:43 PM, goon83 wrote:

Hi all,
     I am planing to conduct some research on I/O optimization like “A Plugin for HDF5 using PLFS for Improved I/O Performance and Semantic Analysis”, and I need to write a plugin for "Virtual Object Layer". I have already downloaded and installed the hdf5-1.8.10 source code. However, I have not found anyplace/guide to start a new plugin. Would you all please give me some guide or help ? Thanks in advance !

Bin Dong

--

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.orghttp://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

4

Hi, Mohamad,
       Thanks for your information. I am starting writing my own plugin right now. I am confused by two functions in the example code you shared with me.

       The first one is in :

···

--------------------
        under_fapl = H5Pcreate (H5P_FILE_ACCESS);
        H5Pset_fapl_native(under_fapl);
        --------------------
        what does the actually means of H5Pset_fapl_native? Why we need these two statements for the log vol plugin.

        The second one is in:
         ---------
         acc_tpl = H5Pcreate (H5P_FILE_ACCESS);
         H5Pset_vol(acc_tpl, vol_id, &under_fapl);
          file_id = H5Fcreate(file_name, H5F_ACC_TRUNC, H5P_DEFAULT, acc_tpl);
          ---------
          Does H5Pset_vol build the relationship between the acc_tpl and vol_id ? If it is YES, does the next function H5Fcreate means that if we create a file with acc_tpl property, the request on this file will handled by the vol with vol_id ?
          BTW, why we need under_fapl in H5Pset_vol, would you mind explain to me?

          Thanks.

Bin

--

At 2013-01-08 23:24:14,"Mohamad Chaarawi" <chaarawi@hdfgroup.org> wrote:

Hi Bin,

The VOL work is not part of any release yet and still sitting in a separate branch. We are aiming to release with 1.10.

You can access the branch from:
http://svn.hdfgroup.uiuc.edu/hdf5/features/vol

I have to warn you that this is not a release branch, but a development branch, so things will change in the future. The VOL interface has been stable for a while now, however with some changes/additions expected to the HDF5 library interface, the VOL interface callbacks will change.

I also attached a sample C file that serves as a guide for adding an external plugin. The H5VLnative.c under src/ would serve as a guide for implementing an internal library specific plugin (which I don't think would be your case).

Thanks,
Mohamad

On 1/7/2013 10:43 PM, goon83 wrote:

Hi all,
     I am planing to conduct some research on I/O optimization like “A Plugin for HDF5 using PLFS for Improved I/O Performance and Semantic Analysis”, and I need to write a plugin for "Virtual Object Layer". I have already downloaded and installed the hdf5-1.8.10 source code. However, I have not found anyplace/guide to start a new plugin. Would you all please give me some guide or help ? Thanks in advance !

Bin Dong

--

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.orghttp://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

5

Hi Bin,

Hi, Mohamad,
       Thanks for your information. I am starting writing my own plugin right now. I am confused by two functions in the example code you shared with me.

       The first one is in :
        --------------------
        under_fapl = H5Pcreate (H5P_FILE_ACCESS);
        H5Pset_fapl_native(under_fapl);
        --------------------
        what does the actually means of H5Pset_fapl_native? Why we need these two statements for the log vol plugin.

Well that log plugin here does a printf() to "log" the operation, then calls the native HDF5 plugin to execute the VOL operation. So we are stacking 2 plugins in this case. First the call goes through the log plugin and the log plugin then uses the under_fapl to send it to the native plugin.

        The second one is in:
         ---------
         acc_tpl = H5Pcreate (H5P_FILE_ACCESS);
         H5Pset_vol(acc_tpl, vol_id, &under_fapl);
          file_id = H5Fcreate(file_name, H5F_ACC_TRUNC, H5P_DEFAULT, acc_tpl);
          ---------
          Does H5Pset_vol build the relationship between the acc_tpl and vol_id ? If it is YES, does the next function H5Fcreate means that if we create a file with acc_tpl property, the request on this file will handled by the vol with vol_id ?

yes, yes.

BTW, why we need under_fapl in H5Pset_vol, would you mind explain to me?

as explained above, the under_fapl here is information for the log plugin to know where to send the other call. If you don't want to send the call to the native plugin in this case, you could pass NULL. Please note that the example I sent you shows how to stack two plugin. If you are just writing your own plugin, you shouldn't worry about using the native plugin here, and pass the info you need (NULL if you don't need any) in the set_vol call.

Thanks,
Mohamad

···

On 3/1/2013 5:17 PM, goon83 wrote:

          Thanks.

Bin

–
365??? <http://gongyi.163.com/love365>

At 2013-01-08 23:24:14,"Mohamad Chaarawi" <chaarawi@hdfgroup.org> wrote:

    Hi Bin,

    The VOL work is not part of any release yet and still sitting in a
    separate branch. We are aiming to release with 1.10.

    You can access the branch from:
    http://svn.hdfgroup.uiuc.edu/hdf5/features/vol

    I have to warn you that this is not a release branch, but a
    development branch, so things will change in the future. The VOL
    interface has been stable for a while now, however with some
    changes/additions expected to the HDF5 library interface, the VOL
    interface callbacks will change.

    I also attached a sample C file that serves as a guide for adding
    an external plugin. The H5VLnative.c under src/ would serve as a
    guide for implementing an internal library specific plugin (which
    I don't think would be your case).

    Thanks,
    Mohamad

    On 1/7/2013 10:43 PM, goon83 wrote:

    Hi all,
         I am planing to conduct some research on I/O optimization
    like "A Plugin for HDF5 using PLFS for Improved I/O Performance
    and Semantic Analysis", and I need to write a plugin for
     "Virtual Object Layer". I have already downloaded and installed
    the hdf5-1.8.10 source code. However, I have not found
    anyplace/guide to start a new plugin. Would you all please
    give me some guide or help ? Thanks in advance !

    Bin Dong

    –
    365??? <http://gongyi.163.com/love365>

    _______________________________________________
    Hdf-forum is for HDF software users discussion.
    Hdf-forum@hdfgroup.org
    http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org