How to get the fapl of a file inside a VOL_file_close callback

Hdf-forum archives
1

Hi Everyone,
I'm trying to create a VOL where all the calls are shipped to a server
process which will process them remotely.
What I want is that only rank 0, of the MPI_COMM associated with my
fapl, forwards the requests to the server.

The problem is that in myvol_file_close callback I cannot retrieve the
VOL info starting from the "void *file" input parameter. Info needed
for determine the rank.
What I'm doing is:
1 - Get the file id from the void *file.
2 - Get the fapl using the function H5Fget_access_plist(file_id).
3 - Finally, get the VOL info using H5Pget_vol_info(fapl).

What is giving me problems is step 2, which seems to trigger the
myvol_file_get callbak. Function that I'm trying to implement and
which is giving me troubles. In fact, looking at the callback of the
native VOL I found a lot of private--not part of the public API--
types and functions which I cannot use because I'm coding my VOL
externally at the library. How could I proceed?

I start thinking that I should implement my VOL e and compile it with
the HDF5 library itself. Which is the suggested way?

Thanks,
Nicola.

Here the code involved:
typedef struct H5VL_myvol_t {
    MPI_Comm comm; /*communicator */
    MPI_Info info; /*file information */
} H5VL_myvol_t;

/* Fuction made public to final user code */
herr_t H5Pset_fapl_myvol(hid_t fapl_id, MPI_Comm comm, MPI_Info info) {
    herr_t ret_value;

    /* Initialize driver specific properties */
    H5VL_myvol_t fa;

    fa.comm = comm;
    fa.info = info;

    if ( (ret_value = H5Pset_vol(fapl_id, id, &fa)) < 0) {
      _ERROR_("Error: Impossible to set myvol at the fapl\n");
    }

return ret_value;
} /* end H5Pset_fapl_myvol() */

static herr_t myvol_file_close(void *file, hid_t dxpl_id, void **req) {
hid_t fapl;
herr_t ret_value = 0;
H5VL_myvol_t *fa = NULL;

/* Retrieve the file_id */
hid_t file_id = (hid_t) file;

/* Retrieve the fapl */
if ( (fapl = H5Fget_access_plist(file_id) ) < 0 ) {
   _ERROR_(MSG_ERR_GET_FAPL);
   return -1;
}

/* Retrieve the VOL info */
if(NULL == ( fa = (H5VL_myvolvol_t *) H5Pget_vol_info(fapl)) ) {
  _ERROR_(MSG_ERR_GET_VOL_INFO);
  return NULL;
}

int mpi_rank, mpi_size;
MPI_Comm_size(fa->comm, &mpi_size);
MPI_Comm_rank(fa->comm, &mpi_rank);

if ( mpi_rank == 0 ) {
...
}
...

return ret_value;
} /* end myvol_file_close() */

···

--
Nicola Cadenelli
Phone (IT) +39 334 6550271
Office (DE) +49 2461 6196 475
Skype: nicolacdnll

2

Hi Nicola,

Can't you duplicate the communicator or add the mpi rank in your file struct?
I am not sure what you are doing exactly, but are you building you plugin on top of the native plugin?
is your file structure just the hid_t created by the native plugin?

You have to implement the GET callback for the file access plist to return the file access properties. If you just call the H5Fget_access_plist in your code, it will go back to the API call and call the get callback on the file_id that you pass. This is the expected behavior of course. If you are forwarding things to a server, then the get fapl should send a request to your server asking for the fapl to get it back.

The native plugin IS supposed to make internal HDF5 calls, because it is the internal HDF5 library plugin :slight_smile:

This part of the code that you pasted seems problematic:

/* Retrieve the file_id */
hid_t file_id = (hid_t) file;

how can you cast a pointer to an hid_t?
if your plugin file struct is just the hid_t, you would need something like:
hid_t file_id = *((hid_t *)file);

Thanks,
Mohamad

···

-----Original Message-----
From: Hdf-forum [mailto:hdf-forum-bounces@lists.hdfgroup.org] On Behalf Of Nicola Cadenelli
Sent: Friday, June 13, 2014 6:09 AM
To: hdf-forum@lists.hdfgroup.org
Subject: [Hdf-forum] How to get the fapl of a file inside a VOL_file_close callback

Hi Everyone,
I'm trying to create a VOL where all the calls are shipped to a server process which will process them remotely.
What I want is that only rank 0, of the MPI_COMM associated with my fapl, forwards the requests to the server.

The problem is that in myvol_file_close callback I cannot retrieve the VOL info starting from the "void *file" input parameter. Info needed for determine the rank.
What I'm doing is:
1 - Get the file id from the void *file.
2 - Get the fapl using the function H5Fget_access_plist(file_id).
3 - Finally, get the VOL info using H5Pget_vol_info(fapl).

What is giving me problems is step 2, which seems to trigger the myvol_file_get callbak. Function that I'm trying to implement and which is giving me troubles. In fact, looking at the callback of the native VOL I found a lot of private--not part of the public API-- types and functions which I cannot use because I'm coding my VOL externally at the library. How could I proceed?

I start thinking that I should implement my VOL e and compile it with the HDF5 library itself. Which is the suggested way?

Thanks,
Nicola.

Here the code involved:
typedef struct H5VL_myvol_t {
    MPI_Comm comm; /*communicator */
    MPI_Info info; /*file information */ } H5VL_myvol_t;

/* Fuction made public to final user code */ herr_t H5Pset_fapl_myvol(hid_t fapl_id, MPI_Comm comm, MPI_Info info) {
    herr_t ret_value;

    /* Initialize driver specific properties */
    H5VL_myvol_t fa;

    fa.comm = comm;
    fa.info = info;

    if ( (ret_value = H5Pset_vol(fapl_id, id, &fa)) < 0) {
      _ERROR_("Error: Impossible to set myvol at the fapl\n");
    }

return ret_value;
} /* end H5Pset_fapl_myvol() */

static herr_t myvol_file_close(void *file, hid_t dxpl_id, void **req) { hid_t fapl; herr_t ret_value = 0; H5VL_myvol_t *fa = NULL;

/* Retrieve the file_id */
hid_t file_id = (hid_t) file;

/* Retrieve the fapl */
if ( (fapl = H5Fget_access_plist(file_id) ) < 0 ) {
   _ERROR_(MSG_ERR_GET_FAPL);
   return -1;
}

/* Retrieve the VOL info */
if(NULL == ( fa = (H5VL_myvolvol_t *) H5Pget_vol_info(fapl)) ) {
  _ERROR_(MSG_ERR_GET_VOL_INFO);
  return NULL;
}

int mpi_rank, mpi_size;
MPI_Comm_size(fa->comm, &mpi_size);
MPI_Comm_rank(fa->comm, &mpi_rank);

if ( mpi_rank == 0 ) {
...
}
...

return ret_value;
} /* end myvol_file_close() */

--
Nicola Cadenelli
Phone (IT) +39 334 6550271
Office (DE) +49 2461 6196 475
Skype: nicolacdnll

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum@lists.hdfgroup.org
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
Twitter: https://twitter.com/hdf5

3

First of all thanks for the reply.

Can't you duplicate the communicator or add the mpi rank in your file struct?
is your file structure just the hid_t created by the native plugin?

I didn't define a file struct. I thought that I had to stick with the H5F_t.
The main problem is that I got lost trying to understand how to do my
VOL looking at the native and having as only documentation this RFC
[1].

I am not sure what you are doing exactly, but are you building you plugin on top of the native plugin?

Yes for now the idea is that one.

You have to implement the GET callback for the file access plist to return the file access properties. If you just call the H5Fget_access_plist in your code, it will go back to the API call and call the get callback on the file_id that you pass. This is the expected behavior of course. If you are forwarding things to a server, then the get fapl should send a request to your server asking for the fapl to get it back.

Just to be sure, If I implement a function H5VL_myvol_get_fapl as you
said, I should also adapt the set_fapl function so that will send to
the server the information. I'm right or misunderstanding it?

The native plugin IS supposed to make internal HDF5 calls, because it is the internal HDF5 library plugin :slight_smile:

I understood that, but I would you suggest to implement a VOL as part
of the internal library?

how can you cast a pointer to an hid_t?
if your plugin file struct is just the hid_t, you would need something like:
hid_t file_id = *((hid_t *)file);

You are totally right; this happen when I stop thinking.
In fact, I neither know what is given to me behind that void pointer.
If I look at the native VOL I expect a pointer to a H5F_t, is that
always the case?

Thanks again,
Nicola.
[1] https://fossies.org/linux/misc/hdf5-1.9.183_docs.tar.gz/hdf5-1.9.183_docs/RFCs/HDF5/VOL/2012-05-25-RFC_VOL.docx

···

On 16 June 2014 16:32, Mohamad Chaarawi <chaarawi@hdfgroup.org> wrote:

4

I then resolved my problem but I forgot to report it in the remote
case it could be useful to someone.

The problem was that I wasn't understanding what is given behind the
void pointer in the close callback. Pointer that (correct me if I'm
wrong) is the same returned by the file_open/file_create callbacks. So
one can create a file structure containing all the needed information
which will be accessible on the close callback as well.

···

On 17 June 2014 09:39, Nicola Cadenelli <nicolacdnll@gmail.com> wrote:

First of all thanks for the reply.

On 16 June 2014 16:32, Mohamad Chaarawi <chaarawi@hdfgroup.org> wrote:

Can't you duplicate the communicator or add the mpi rank in your file struct?
is your file structure just the hid_t created by the native plugin?

I didn't define a file struct. I thought that I had to stick with the H5F_t.
The main problem is that I got lost trying to understand how to do my
VOL looking at the native and having as only documentation this RFC
[1].

I am not sure what you are doing exactly, but are you building you plugin on top of the native plugin?

Yes for now the idea is that one.

You have to implement the GET callback for the file access plist to return the file access properties. If you just call the H5Fget_access_plist in your code, it will go back to the API call and call the get callback on the file_id that you pass. This is the expected behavior of course. If you are forwarding things to a server, then the get fapl should send a request to your server asking for the fapl to get it back.

Just to be sure, If I implement a function H5VL_myvol_get_fapl as you
said, I should also adapt the set_fapl function so that will send to
the server the information. I'm right or misunderstanding it?

The native plugin IS supposed to make internal HDF5 calls, because it is the internal HDF5 library plugin :slight_smile:

I understood that, but I would you suggest to implement a VOL as part
of the internal library?

how can you cast a pointer to an hid_t?
if your plugin file struct is just the hid_t, you would need something like:
hid_t file_id = *((hid_t *)file);

You are totally right; this happen when I stop thinking.
In fact, I neither know what is given to me behind that void pointer.
If I look at the native VOL I expect a pointer to a H5F_t, is that
always the case?

Thanks again,
Nicola.
[1] https://fossies.org/linux/misc/hdf5-1.9.183_docs.tar.gz/hdf5-1.9.183_docs/RFCs/HDF5/VOL/2012-05-25-RFC_VOL.docx

--
Nicola Cadenelli
Phone (IT) +39 334 6550271
Office (DE) +49 2461 6196 475
Skype: nicolacdnll