[petsc-dev] [pflotran-dev: 976] Re: [pflotran-users: 998] Distorted structured grid

[Jed Brown]

Paolo Orsini <paolo.orsini at gmail.com> writes:

> Hi Jed,
>
> I tried the MatConvert updated version, (from MPIAdj type to MPIAij type)
> but it seems still slow as before.

Did you update to 'next' after my changes on Friday afternoon?  I tested
performance with a 3D Laplacian:

diff --git i/src/ksp/ksp/examples/tutorials/ex45.c w/src/ksp/ksp/examples/tutorials/ex45.c
index 4a67263..36f61d7 100644
--- i/src/ksp/ksp/examples/tutorials/ex45.c
+++ w/src/ksp/ksp/examples/tutorials/ex45.c
@@ -149,5 +149,13 @@ PetscErrorCode ComputeMatrix(KSP ksp,Mat jac,Mat B,MatStructure *stflg,void *ctx
   ierr   = MatAssemblyBegin(B,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);
   ierr   = MatAssemblyEnd(B,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);
   *stflg = SAME_NONZERO_PATTERN;
+
+  {
+    Mat Badj,Baij;
+    ierr = MatConvert(B,MATMPIADJ,MAT_INITIAL_MATRIX,&Badj);CHKERRQ(ierr);
+    ierr = MatConvert(Badj,MATMPIAIJ,MAT_INITIAL_MATRIX,&Baij);CHKERRQ(ierr);
+    ierr = MatDestroy(&Badj);CHKERRQ(ierr);
+    ierr = MatDestroy(&Baij);CHKERRQ(ierr);
+  }
   PetscFunctionReturn(0);
 }


$ mpiexec -n 4 ./ex45 -da_grid_x 100 -da_grid_y 100 -da_grid_z 100 -ksp_max_it 1 -log_summary | great MatConvert
MatConvert             2 1.0 3.6934e-01 1.0 0.00e+00 0.0 1.6e+01 1.0e+04 1.9e+01 41  0 25 14 24  41  0 25 14 25     0

> Anyway, don't worry to much. I abandoned the idea of the conversion.
> I am forming the adjacency matrix directly in MPI AIJ format, so I don't
> have to do the conversion.
> I also realised that the Dual Graph is formed in parallel (in the current
> PFLOTRAN implementation), so I can't use the MatMatTransposeMult (which
> works only for sequential matrix).
>
> I will use instead MatCreatTranspose 

Use MatTranspose().  MatCreateTranspose doesn't move any data around, it
just makes a matrix that behaves like the transpose.

> and MAtMatMult, then filtering the resulting matrix row by row and
> from the DualGraph.  It will require more memory, but it should
> work. Do you see any problem with this?

Forming the AIJ matrix up-front is less memory than creating a MatAdj.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20131118/592d782a/attachment.sig>