passing data between different communicators using PETSc

Matthew Knepley knepley at
Mon Mar 9 11:41:20 CDT 2009

You can use a VecScatter, and PETSc will try to use the best MPI


On Mon, Mar 9, 2009 at 1:24 AM, Khan, Irfan <irfan.khan at> wrote:

> Hi
> I have divided my processes into fluid compute ranks (n) and solid compute
> ranks (m), where n > m. For most of the part, the fluid compute ranks
> communicate among themselves and so do the solid compute nodes. However
> twice during each timestep the some data is transfered between the fluid and
> solid compute nodes.
> For instance, the fluid compute nodes generate fluid force on the solid
> body that needs to be transmitted every time step and the solid compute node
> calculate the resulting displacement from the forces and transmit back the
> displacement information to the fluid nodes. The force and displacement
> vectors for total nodes are distributed randomly on the fluid and solid
> nodes. I have attached a pdf that basically shows that the distribution of
> vectors is random
> I am currently trying to use MPI_Allgatherv to accumulate all the data from
> send ranks to receive ranks, but I am sure this will be very costly for
> large data distributed over many ranks. Is there an efficient way to do this
> using PETSc?
> Please let me know if you need more information or better explanation.
> Thanks
> Irfan

What most experimenters take for granted before they begin their experiments
is infinitely more interesting than any results to which their experiments
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the petsc-users mailing list