# How do I impose that only certain processors write the H5p file? **URL:** \ **Category:** Hdf-forum archives\ **Created:** [March 9, 2015, 12:55pm UTC](https://forum.hdfgroup.org/t/how-do-i-impose-that-only-certain-processors-write-the-h5p-file/3427 "2015-03-09T12:55:48Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Maria\_Castela](https://forum.hdfgroup.org/letter_avatar_proxy/v4/letter/m/e56c9b/32.png) [@Maria\_Castela](https://forum.hdfgroup.org/u/Maria_Castela)\ **Post date:** [March 9, 2015, 12:55pm UTC](https://forum.hdfgroup.org/t/how-do-i-impose-that-only-certain-processors-write-the-h5p-file/3427/1 "2015-03-09T12:55:48Z")
Dear all, When I use CALL h5pcreate\_f(H5P\_FILE\_ACCESS\_F, plist\_id, error) CALL h5pset\_fapl\_mpio\_f(plist\_id, comm, info, error) I assume that all processors are called to write the solution. However, I just want the processors which have coords(3) equals a certain value to write the solution… (constant z plane) Figure below shows an example of what I want and what the program is doing. How do I impose this condition? I have already tried « IF (coord(3) == 1) THEN CALL h5pcreate\_f(H5P\_FILE\_ACCESS\_F, plist\_id, error) etc… ENDIF « However It doesn’t like it. Thank you in advance, Cheers, Maria Maria Luis CASTELA Doctorante Ecole Centrale Paris Laboratoire EM2C Grande Voie des Vignes 92295 Chatenay-Malabry, France Tel. : +33 (0)1 41 13 10 34 Courriel : maria.castela@ecp.fr ![](https://hdf-discourse-1.s3.amazonaws.com/original/1X/c5e5ce2ed3539129bd5705447f459ee3f3b24abc.png) ---
**Author:** ![angelv](https://forum.hdfgroup.org/user_avatar/forum.hdfgroup.org/angelv/32/1095_2.png) [@angelv](https://forum.hdfgroup.org/u/angelv)\ **Post date:** [March 10, 2015, 1:38pm UTC](https://forum.hdfgroup.org/t/how-do-i-impose-that-only-certain-processors-write-the-h5p-file/3427/2 "2015-03-10T13:38:50Z")
Hi Maria, Maria Castela \ writes: > Dear all, > When I use > CALL h5pcreate\_f(H5P\_FILE\_ACCESS\_F, plist\_id, error) > CALL h5pset\_fapl\_mpio\_f(plist\_id, comm, info, error) > I assume that all processors are called to write the solution. > However, I just want the processors which have coords(3) equals a certain value to write the solution… (constant z plane) > Figure below shows an example of what I want and what the program is doing. > How do I impose this condition? I have already tried > « IF (coord(3) == 1) THEN > CALL h5pcreate\_f(H5P\_FILE\_ACCESS\_F, plist\_id, error) > etc… > ENDIF « > However It doesn’t like it. We do need this in our code as well, and we have two different ways of doing it: 1) if all processors are going to take part in the I/O operation, then    when you define the amount of data each processor is contributing,    this will be 0 for all processors except those with coord(3) .EQ. 1    then all processors do exactly the same collective operations, except    that the amount of data they read/write will be different. 2) you create a MPI communicator that groups those processors where    coord(3) .EQ. 1. Then, only those processors will contribute to the    I/O operation, which is similar to what your post seems to imply you    want to do, but then you have to make sure that the communicator in    your calls is not the global one (MPI\_COMM\_WORLD), but rather the one    you created specifially for coord(3) .EQ. 1 If this doesn't lead you very far, I can give you further details. Cheers, > **···** > > -- > Ángel de Vicente > [http://www.iac.es/galeria/angelv/](http://www.iac.es/galeria/angelv/) > --------------------------------------------------------------------------------------------- > ADVERTENCIA: Sobre la privacidad y cumplimiento de la Ley de Protección de Datos, acceda a [Legal notice | Instituto de Astrofísica de Canarias • IAC](http://www.iac.es/disclaimer.php) > WARNING: For more information on privacy and fulfilment of the Law concerning the Protection of Data, consult [Legal notice | Instituto de Astrofísica de Canarias • IAC](http://www.iac.es/disclaimer.php?lang=en) ---
**Author:** ![Maria\_Castela](https://forum.hdfgroup.org/letter_avatar_proxy/v4/letter/m/e56c9b/32.png) [@Maria\_Castela](https://forum.hdfgroup.org/u/Maria_Castela)\ **Post date:** [March 10, 2015, 9:09pm UTC](https://forum.hdfgroup.org/t/how-do-i-impose-that-only-certain-processors-write-the-h5p-file/3427/3 "2015-03-10T21:09:19Z")
Yes! Indeed, the second is the efficient way to store my 2D-slice. So, following option 2: 1- I’ve grouped the processors with coord(3).EQ.1 from original\_group:   MPI\_COMM\_GROUP(MPI\_COMM\_WORLD, original\_group)   MPI\_GROUP\_INCL(original\_group, nb\_process\_2D\_SLICE, processes\_2D\_SLICE, 2D\_group,code)    2- I’ve created a MPI communicator for this group:   MPI\_COMM\_CREAT(MPI\_COMM\_WORLD, my\_group\_2D, MPI\_COMM\_2D\_SLICE, code) —Problem— When I do:   CALL MPI\_COMM\_RANK(MPI\_COMM\_2D\_SLICE, 2D\_ranks, code) I’ve a segmentation fault… Did you have this problem? —After solving this problem — 3 - You say that instead of MPI\_COMM\_WORLD I shall use MPI\_COMM\_2D\_SLICE here?:          comm = MPI\_COMM\_2D\_SLICE     h5pset\_fapl\_mpio\_f(plist\_id, comm, info) Thank you a lot! It was very helpful Cheers, Maria > **···** > > On 10 Mar 2015, at 14:38, Angel de Vicente \ wrote: > > > Hi Maria, > > > > Maria Castela \ writes: > > > > > Dear all, > > > When I use > > > CALL h5pcreate\_f(H5P\_FILE\_ACCESS\_F, plist\_id, error) > > > CALL h5pset\_fapl\_mpio\_f(plist\_id, comm, info, error) > > > I assume that all processors are called to write the solution. > > > However, I just want the processors which have coords(3) equals a certain value to write the solution… (constant z plane) > > > Figure below shows an example of what I want and what the program is doing. > > > How do I impose this condition? I have already tried > > > « IF (coord(3) == 1) THEN > > > CALL h5pcreate\_f(H5P\_FILE\_ACCESS\_F, plist\_id, error) > > > etc… > > > ENDIF « > > > However It doesn’t like it. > > > > We do need this in our code as well, and we have two different ways of > > doing it: > > > > 1) if all processors are going to take part in the I/O operation, then > >   when you define the amount of data each processor is contributing, > >   this will be 0 for all processors except those with coord(3) .EQ. 1 > >   then all processors do exactly the same collective operations, except > >   that the amount of data they read/write will be different. > > > > 2) you create a MPI communicator that groups those processors where > >   coord(3) .EQ. 1. Then, only those processors will contribute to the > >   I/O operation, which is similar to what your post seems to imply you > >   want to do, but then you have to make sure that the communicator in > >   your calls is not the global one (MPI\_COMM\_WORLD), but rather the one > >   you created specifially for coord(3) .EQ. 1 > > > > If this doesn't lead you very far, I can give you further details. > > > > Cheers, > > -- > > Ángel de Vicente > > [Website no longer available](http://www.iac.es/galeria/angelv/) > > --------------------------------------------------------------------------------------------- > > ADVERTENCIA: Sobre la privacidad y cumplimiento de la Ley de Protección de Datos, acceda a [Legal notice | Instituto de Astrofísica de Canarias • IAC](http://www.iac.es/disclaimer.php) > > WARNING: For more information on privacy and fulfilment of the Law concerning the Protection of Data, consult [Legal notice | Instituto de Astrofísica de Canarias • IAC](http://www.iac.es/disclaimer.php?lang=en) > > > > \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ > > Hdf-forum is for HDF software users discussion. > > Hdf-forum@lists.hdfgroup.org > > [http://lists.hdfgroup.org/mailman/listinfo/hdf-forum\_lists.hdfgroup.org](http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org) > > Twitter: [x.com](https://twitter.com/hdf5) ---
**Author:** ![angelv](https://forum.hdfgroup.org/user_avatar/forum.hdfgroup.org/angelv/32/1095_2.png) [@angelv](https://forum.hdfgroup.org/u/angelv)\ **Post date:** [March 18, 2015, 11:16am UTC](https://forum.hdfgroup.org/t/how-do-i-impose-that-only-certain-processors-write-the-h5p-file/3427/4 "2015-03-18T11:16:34Z")
Hi, sorry for the delay, I was sucked into a hole last week... Maria Luis Castela \ writes: > Yes! Indeed, the second is the efficient way to store my 2D-slice. well, the way for storing (in RAM or in file) for both options is exactly the same. The only thing that changes is how you call the I/O writing routines, but if we stick to option 2: > So, following option 2: > > 1- I’ve grouped the processors with coord(3).EQ.1 from original\_group: > >   MPI\_COMM\_GROUP(MPI\_COMM\_WORLD, original\_group) >   MPI\_GROUP\_INCL(original\_group, nb\_process\_2D\_SLICE, >   processes\_2D\_SLICE, 2D\_group,code) >    > 2- I’ve created a MPI communicator for this group: > >   MPI\_COMM\_CREAT(MPI\_COMM\_WORLD, my\_group\_2D, MPI\_COMM\_2D\_SLICE, code) OK. I do it with MPI\_CART\_SUB, but I guess it doesn't matter, as far as you have a communicator that holds those processors only with coord(3).EQ.1 > —Problem— > > When I do: >   CALL MPI\_COMM\_RANK(MPI\_COMM\_2D\_SLICE, 2D\_ranks, code) > I’ve a segmentation fault… Did you have this problem? I never need to call MPI\_COMM\_RANK with the 'slice' communicator. > 3 - You say that instead of MPI\_COMM\_WORLD I shall use MPI\_COMM\_2D\_SLICE here?: >      >     comm = MPI\_COMM\_2D\_SLICE >     h5pset\_fapl\_mpio\_f(plist\_id, comm, info) the relevant part of my code to make sure that only those at the bottom of my domain write to file looks like this:             IF (my\_coordinates(3) .EQ. 0) THEN                CALL h5pcreate\_f(H5P\_FILE\_ACCESS\_F, fapl\_id, error)                CALL h5pset\_fapl\_mpio\_f(fapl\_id, pmlzcomm, MPI\_INFO\_NULL,                error)                CALL h5fopen\_f(H5filename, H5F\_ACC\_RDWR\_F, file\_id,                error, access\_prp = fapl\_id)                CALL MPI\_BARRIER(pmlzcomm,error) !!! I only need this                in one of the clusters we use, don't now why...                [...]                CALL h5dwrite\_f(dset\_id, H5T\_NATIVE\_DOUBLE,                pml\_v2%z\_low(slx:elx,sly:ely,:,:), dims\_pml, error, &                     file\_space\_id = fspace\_id, mem\_space\_id = mspace\_id,                xfer\_prp = dxpl\_id)                CALL h5sclose\_f(fspace\_id,error)                CALL h5sclose\_f(mspace\_id,error)                CALL h5dclose\_f(dset\_id, error)                CALL h5pclose\_f(fapl\_id, error)                CALL h5pclose\_f(dxpl\_id, error)                CALL h5fclose\_f(file\_id, error)             END IF Cheers, > **···** > > -- > Ángel de Vicente > [http://www.iac.es/galeria/angelv/](http://www.iac.es/galeria/angelv/) > --------------------------------------------------------------------------------------------- > ADVERTENCIA: Sobre la privacidad y cumplimiento de la Ley de Protección de Datos, acceda a [Legal notice | Instituto de Astrofísica de Canarias • IAC](http://www.iac.es/disclaimer.php) > WARNING: For more information on privacy and fulfilment of the Law concerning the Protection of Data, consult [Legal notice | Instituto de Astrofísica de Canarias • IAC](http://www.iac.es/disclaimer.php?lang=en) ---
**Author:** ![Maria\_Castela](https://forum.hdfgroup.org/letter_avatar_proxy/v4/letter/m/e56c9b/32.png) [@Maria\_Castela](https://forum.hdfgroup.org/u/Maria_Castela)\ **Post date:** [April 10, 2015, 10:49am UTC](https://forum.hdfgroup.org/t/how-do-i-impose-that-only-certain-processors-write-the-h5p-file/3427/5 "2015-04-10T10:49:14Z")
Angel, Thank you for your help, I now have my horizontal and vertical slices 😃 Cheers! Maria > **···** > > Le 18 mars 2015 à 12:16, Angel de Vicente \ a écrit : > > > Hi, > > > > sorry for the delay, I was sucked into a hole last week... > > > > Maria Luis Castela \ writes: > > > > > Yes! Indeed, the second is the efficient way to store my 2D-slice. > > > > well, the way for storing (in RAM or in file) for both options is > > exactly the same. The only thing that changes is how you call the I/O > > writing routines, but if we stick to option 2: > > > > > So, following option 2: > > > > > > 1- I’ve grouped the processors with coord(3).EQ.1 from original\_group: > > > > > >   MPI\_COMM\_GROUP(MPI\_COMM\_WORLD, original\_group) > > >   MPI\_GROUP\_INCL(original\_group, nb\_process\_2D\_SLICE, > > >   processes\_2D\_SLICE, 2D\_group,code) > > >    > > > 2- I’ve created a MPI communicator for this group: > > > > > >   MPI\_COMM\_CREAT(MPI\_COMM\_WORLD, my\_group\_2D, MPI\_COMM\_2D\_SLICE, code) > > > > OK. I do it with MPI\_CART\_SUB, but I guess it doesn't matter, as far as > > you have a communicator that holds those processors only with coord(3).EQ.1 > > > > > —Problem— > > > > > > When I do: > > >   CALL MPI\_COMM\_RANK(MPI\_COMM\_2D\_SLICE, 2D\_ranks, code) > > > I’ve a segmentation fault… Did you have this problem? > > > > I never need to call MPI\_COMM\_RANK with the 'slice' communicator. > > > > > 3 - You say that instead of MPI\_COMM\_WORLD I shall use MPI\_COMM\_2D\_SLICE here?: > > >      > > >     comm = MPI\_COMM\_2D\_SLICE > > >     h5pset\_fapl\_mpio\_f(plist\_id, comm, info) > > > > the relevant part of my code to make sure that only those at the bottom > > of my domain write to file looks like this: > > > >            IF (my\_coordinates(3) .EQ. 0) THEN > >               CALL h5pcreate\_f(H5P\_FILE\_ACCESS\_F, fapl\_id, error) > >               CALL h5pset\_fapl\_mpio\_f(fapl\_id, pmlzcomm, MPI\_INFO\_NULL, > >               error) > >               CALL h5fopen\_f(H5filename, H5F\_ACC\_RDWR\_F, file\_id, > >               error, access\_prp = fapl\_id) > > > >               CALL MPI\_BARRIER(pmlzcomm,error) !!! I only need this > >               in one of the clusters we use, don't now why... > > > >               [...] > > > >               CALL h5dwrite\_f(dset\_id, H5T\_NATIVE\_DOUBLE, > >               pml\_v2%z\_low(slx:elx,sly:ely,:,:), dims\_pml, error, & > >                    file\_space\_id = fspace\_id, mem\_space\_id = mspace\_id, > >               xfer\_prp = dxpl\_id) > > > >               CALL h5sclose\_f(fspace\_id,error) > >               CALL h5sclose\_f(mspace\_id,error) > >               CALL h5dclose\_f(dset\_id, error) > >               CALL h5pclose\_f(fapl\_id, error) > >               CALL h5pclose\_f(dxpl\_id, error) > >               CALL h5fclose\_f(file\_id, error) > >            END IF > > > > Cheers, > > -- > > Ángel de Vicente > > [Website no longer available](http://www.iac.es/galeria/angelv/) > > --------------------------------------------------------------------------------------------- > > ADVERTENCIA: Sobre la privacidad y cumplimiento de la Ley de Protección de Datos, acceda a [Legal notice | Instituto de Astrofísica de Canarias • IAC](http://www.iac.es/disclaimer.php) > > WARNING: For more information on privacy and fulfilment of the Law concerning the Protection of Data, consult [Legal notice | Instituto de Astrofísica de Canarias • IAC](http://www.iac.es/disclaimer.php?lang=en) > > > > \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ > > Hdf-forum is for HDF software users discussion. > > Hdf-forum@lists.hdfgroup.org > > [http://lists.hdfgroup.org/mailman/listinfo/hdf-forum\_lists.hdfgroup.org](http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org) > > Twitter: [x.com](https://twitter.com/hdf5)