[petsc-users] Parallel sort in Petsc

[Junchao Zhang]

Antoine,
  See the attached example. It does sort on rank 0 but puts the permutation
results in parallel.  Run it with mpirun -n 3 ./ex2 for a demo.
  Thanks.
--Junchao Zhang


On Thu, Jul 9, 2020 at 1:24 PM Antoine Côté <Antoine.Cote3 at usherbrooke.ca>
wrote:

> The Vec containing the scalars represents stresses at nodes points of a
> DMDA. My program sorts stresses, and makes calculations according to their
> rank. This needs to be done while keeping track of the original positions
> of stresses in Vec. Using permutations, I can access the node of my choice
> and get its associated rank, without changing the original Vec.
>
> Regards,
>
> Antoine
>
>
> ------------------------------
> *De :* Junchao Zhang <junchao.zhang at gmail.com>
> *Envoyé :* 9 juillet 2020 12:42
> *À :* Antoine Côté <Antoine.Cote3 at USherbrooke.ca>
> *Objet :* Re: [petsc-users] Parallel sort in Petsc
>
> It is easier without permutations. So let me ask first do you need the
> permutations or just a (parallel) sorted vector?
>
> --Junchao Zhang
>
>
> On Thu, Jul 9, 2020 at 11:21 AM Antoine Côté <Antoine.Cote3 at usherbrooke.ca>
> wrote:
>
> That is worth trying!
>
> Say I gather to process 0 as proposed here :
> https://www.mcs.anl.gov/petsc/documentation/faq.html#mpi-vec-to-mpi-vec
> <https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mcs.anl.gov%2Fpetsc%2Fdocumentation%2Ffaq.html%23mpi-vec-to-mpi-vec&data=02%7C01%7CAntoine.Cote3%40usherbrooke.ca%7C5b19e2103f434ddd79a308d824270a5c%7C3a5a8744593545f99423b32c3a5de082%7C0%7C0%7C637299097387394607&sdata=4qRikFqiT7IavQKuVjS6rs9L9KsPPIHC23Ekfov1Jvg%3D&reserved=0>
> Let Vec A be the vector containing the scalars. If I compute the
> permutations I'm looking for, how do I :
>  - tell the code to run PetscSortRealWithPermutation() *only *on process
> 0 ?
>  - distribute back the permutations on all process (that is, send only the
> permutations concerning the local portion of Vec A stored by a given
> process) ?
>  - make sure other processes wait until they receive the permutations
> prior proceeding with the rest of the code ?
>
> Thank you very much!
>
> Antoine
>
> ------------------------------
> *De :* Junchao Zhang <junchao.zhang at gmail.com>
> *Envoyé :* 7 juillet 2020 10:52
> *À :* Antoine Côté <Antoine.Cote3 at USherbrooke.ca>
> *Cc :* petsc-users at mcs.anl.gov <petsc-users at mcs.anl.gov>
> *Objet :* Re: [petsc-users] Parallel sort in Petsc
>
> Gather all values to rank 0 and do a sequential sort there, then profile
> to see whether it is a performance bottleneck in your code.
>
> --Junchao Zhang
>
>
> On Tue, Jul 7, 2020 at 8:14 AM Antoine Côté <Antoine.Cote3 at usherbrooke.ca>
> wrote:
>
> Hi,
>
> I need to compute the permutations to sort scalar values of a Vec object
> in decreasing order. I've been developing on a single process for now,
> using VecGetArrayRead() to extract the values
> and PetscSortRealWithPermutation() to get the permutations.
>
> I would like to run the code on multiple processes. From these links, I
> can see it's an issue :
> https://lists.mcs.anl.gov/pipermail/petsc-users/2013-November/019628.html
> <https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mcs.anl.gov%2Fpipermail%2Fpetsc-users%2F2013-November%2F019628.html&data=02%7C01%7CAntoine.Cote3%40usherbrooke.ca%7C5b19e2103f434ddd79a308d824270a5c%7C3a5a8744593545f99423b32c3a5de082%7C0%7C0%7C637299097387404606&sdata=I1XaEkfaCXpIpHOHH3ErtQyoxMLnRwqFysh1v%2BybfSY%3D&reserved=0>
> https://lists.mcs.anl.gov/pipermail/petsc-users/2009-June/004621.html
> <https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mcs.anl.gov%2Fpipermail%2Fpetsc-users%2F2009-June%2F004621.html&data=02%7C01%7CAntoine.Cote3%40usherbrooke.ca%7C5b19e2103f434ddd79a308d824270a5c%7C3a5a8744593545f99423b32c3a5de082%7C0%7C0%7C637299097387404606&sdata=A9i3R3enHTQjWXLY2yiDNSesporutJPUobU1%2F3Nvk9s%3D&reserved=0>
>
> One work around I've been considering : get the max scalar value using
> VecMax() ; compute a scaling factor that makes this max value equals the
> largest integer value tolerated by PetscInt ; scale all values of Vec and
> convert them to integers ; use PetscParallelSortInt() ; inverse the scaling
> on (now sorted) integers values and convert them back to scalars.
>
> This is only a patch really, the main issues being : (1) I would get a
> sorted Vec, not the permutations and (2) there would be a lost of
> information when converting scalars to int values.
>
> Isn't there a way to parallel sort real values using Petsc?
>
> Thank you very much!
>
> Antoine Côté
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200709/d2b2b6b2/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ex2.c
Type: application/octet-stream
Size: 3365 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200709/d2b2b6b2/attachment-0001.obj>