[petsc-users] Simple questions about vector scattering

[Yaoyu Hu]

Hi everyone,

I am trying to write a simple CFD program with PETSc and I have
encountered some simple problems with vector scattering.

I use finite volume method. When calculating the flux of a scalar
variable on a control volume face, the values of the control volumes
straddling the face should be obtained. The scalar values are stored
in a parallel Vec. I thought that the calculation of the flux values
of different faces can be performed, somehow, in parallel. By 'in
parallel', I mean let different processors calculate the fluxes of
different faces. Because the scalar values are stored in a parallel
Vec, I have to do vector scattering to retrieve values that not stored
on the local processor. And different processes may require different
portions of the parallel Vec. And here comes the problem. It happened
that one processor encountered a boundary face, and no vector
scattering was needed. And this processor omitted the scattering
operation. Then the whole program just halted because, I thought, all
other processors were waiting for the processor to trigger a vector
scattering.

I actually have three questions here:

(1)     Is it OK for different processors perform vector scattering to
retrieve different portions of a parallel Vec in a single scattering
operation? (By test I think it is OK…)

(2)     Should the processor, which is dealing with the boundary face
flux, perform a sort of ‘dummy scattering’ operation? How it is done
in the usual way?

(3)     Am I miss using PETSc to do stuff ‘in parallel’? Should I
perform the face flux calculation and other tasks alike in serial, and
then only use PETSc to solve the discretized equations in parallel?


Thanks in advance.

HU Yaoyu
School of Mechanical Engineering,
Shanghai Jiao Tong University, Shanghai, China.