[petsc-users] Fwd: Building the same petsc matrix with different numprocs gives different results!

[Jed Brown]

Analabha Roy <hariseldon99 at gmail.com> writes:

> There is one thing
>
> In the code, the evaluation of each element of AVG_BIBJ requires a
> read-only matrix U_parallel that I input from another program, and a
> writeable sequential vector BDB_AA that is different for each element.
>
> I sequentiate U_parallel to U_seq by using MatCopy here in lines
> 242+<https://code.google.com/p/daneelrepo/source/browse/eth_question/eth.c#242>

  /*
     - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
     Create a sequential matrix and copy U_parallel to it
     so that each processor has complete local copy of U
     - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   */
  ierr = MatCreateSeqAIJ (PETSC_COMM_SELF, dim, dim, 0, PETSC_NULL, &U_seq);
  CHKERRQ (ierr);
  ierr = MatSetOption (U_seq, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE);
  CHKERRQ (ierr);
  ierr = MatCopy (U_parallel, U_seq, DIFFERENT_NONZERO_PATTERN);

This is wrong and should raise an error (definitely in a debugging
build).  Are you intending to create a redundant matrix containing all
of U_parallel replicated onto each process?  If so, you may as well
assemble the whole thing redundantly, but if you'd rather distribute the
computation of entries, you can use MatGetRedundantMatrix().  If you
want something else, what is it?

  CHKERRQ (ierr);
  ierr = MatAssemblyBegin (U_seq, MAT_FINAL_ASSEMBLY);
  CHKERRQ (ierr);
  ierr = MatAssemblyEnd (U_seq, MAT_FINAL_ASSEMBLY);
  CHKERRQ (ierr);


> and each process is supposed to update its copy of BDB_AA at every loop
> iteration here in line
> 347+<https://code.google.com/p/daneelrepo/source/browse/eth_question/eth.c#347>
>
>
> Is this right? Or are sequential vectors/matrices handled by the root
> process only? 

PETSC_COMM_SELF means the current process, so every process does that stuff.

> I know how to scatter a parallel vector to all processes using PetSc
> scatter contexts but don't see any way to do so to a matrix other than
> MatCopy. How do I ensure that each process has its own private
> writeable copy of a sequential vector?

Is the last sentence supposed to say "vector"?  If so, see
VecScatterCreateToAll.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20130925/f40f75a3/attachment.pgp>