[petsc-users] Transpose MATMPIBAIJ matrix

[Jed Brown]

On Fri, Jan 20, 2012 at 05:30, <S.H.Jongsma at utwente.nl> wrote:

> ** **
>
> Hello fellow PETSc users,****
>
> ** **
>
> While using PETSc I came across some conspicuous behavior regarding the
> use of MatTranspose.  When I use this function with a MATSEQBAIJ matrix it
> performs considerably better than when I use it with a MATMPIBAIJ matrix. I
> was wondering if anyone of you noticed the same behavior and if this is the
> case, what did you do about it to circumvent the ‘problem’. In case no one
> came across this problem before, any suggestions that can help solving this
> performance issue are welcome. ****
>
> ** **
>
> To make my problem more clear, I will give some details on my
> implementation and the test I performed to come to my conclusion. The PETSc
> version I use is 3.1-p8.
>

Just a friendly reminder to upgrade to petsc-3.2.


> I transpose the matrix in place using the following command:****
>
> ** **
>
> MatTranspose(myMatrix, MAT_REUSE_MATRIX, &myMatrix);
>

The code is not doing correct preallocation here. Can you try AIJ? It still
uses a heuristic, but is more likely to be sufficient.

Or, still with MPIBAIJ if you have a square matrix with symmetric nonzero
pattern, this out-of-place transpose should perform well.

MatDuplicate(myMatrix,MAT_DO_NOT_COPY_VALUES,&myMatrixTranspose);
MatTranspose(myMatrix,MAT_REUSE_MATRIX,&myMatrixTranspose);


Why do you need a transpose? Sparse transpose is a very expensive operation
in parallel (even with correct preallocation) and it should always be
avoidable (KSPSolveTranspose(), etc).


> ****
>
> ** **
>
> The matrix I used to test the performance is square and consists of 24576
> times 24576 blocks of size 5, which means that the matrix has 122880 rows
> (and columns, of course). The number of non-zero blocks in the matrix is:
> 291830. Timing the performance gives the following results:****
>
> ** **
>
> MATSEQBAIJ:    0.320 s****
>
> MATMPIBAIJ: 1474     s (running on 1 processor) ****
>
> MATMPIBAIJ:  376     s (running on 2 processors)****
>
> ** **
>
> As I said, the difference in performance is quite considerable, so any
> suggestion that can help me solving this issue are greatly appreciated. **
> **
>
> ** **
>
> Thank you in advance,****
>
> ** **
>
> Kind regards,****
>
> ** **
>
> ** **
>
> Sietse Jongsma****
>
> ** **
>
> ** **
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120120/d701d885/attachment.htm>