[petsc-users] MatCreateBAIJ, SNES, Preallocation...

[William Coirier]

Mark:

Should the size of d_nnz and o_nnz be the number of rows accessed on this processor or the number of block rows?
________________________________________
From: Smith, Barry F. [bsmith at mcs.anl.gov]
Sent: Friday, May 17, 2019 9:15 AM
To: Mark Adams
Cc: William Coirier; petsc-users at mcs.anl.gov; Michael Robinson; Andrew Holm
Subject: Re: [petsc-users] MatCreateBAIJ, SNES, Preallocation...

  I don't understand. For BAIJ and SBAIJ it is the number of block nonzeros in the two parts of the matrix. It is not the number of nonzeros per row.
For Mat[MPI]AIJSetPreallocation() it is the number of nonzeros (even if a block size has been set) because for AIJ matrices the block size does not affect the storage of the matrix and it is merely informative.

   For MatXAIJSetPreallocation() however (which can be used with AIJ, BAIJ, and SBAIJ) the nonzero blocks are for all three matrix formats. MatXAIJSetPreallocation is a convenience function that allows the user to avoid needing to allocate three sets of arrays and fill them up separately for each of the AIJ, BAIJ, and SBAIJ formats.

   Is everything now clear, if not please feel free to ask more specific questions to clarify.

  Barry

> On May 17, 2019, at 6:16 AM, Mark Adams <mfadams at lbl.gov> wrote:
>
>
>
> On Thu, May 16, 2019 at 6:28 PM William Coirier via petsc-users <petsc-users at mcs.anl.gov> wrote:
> Ok, got it. My misinterpretation was how to fill the d_nnz and o_nnz arrays.
>
> Thank you for your help!
>
> Might I make a suggestion related to the documentation? Perhaps I have not fully read the page on the MatMPIBAIJSetPreallocation so you can simply disregard and I'm ok with that! The documentation has for the d_nnz:
>
> d_nnz   - array containing the number of block nonzeros in the various block rows of the in diagonal portion of the local (possibly different for each block row) or NULL. If you plan to factor the matrix you must leave room for the diagonal entry and set it even if it is zero.
>
> Am I correct in that this array should be of size numRows, where numRows is found from  calling MatGetOwnershipRange(J,&iLow,&iHigh) so numRows=iHigh-iLow.
>
> yes, this interface does not change if you set the block size or not. It is at the equation level.
>
>
> I think my error was allocating this only to be numRows/bs since I thought it's a block size thing.
>
>
> This documentation looks wrong to me, and at least confusing. "number of block nonzeros" reads wrong to me. We now have:
>
> d_nz- number of block nonzeros per block row in diagonal portion of local submatrix (same for all local rows)
> d_nnz- array containing the number of block nonzeros in the various block rows of the in diagonal portion of the local (possibly different for each block row) or NULL. If you plan to factor the matrix you must leave room for the diagonal entry and set it even if it is zero.
> o_nz- number of block nonzeros per block row in the off-diagonal portion of local submatrix (same for all local rows).
> o_nnz- array containing the number of nonzeros in the various block rows of the off-diagonal portion of the local submatrix (possibly different for each block row) or NULL.
>
> I can suggest:
>
> d_nz- number of nonzeros per row in diagonal portion of local submatrix (same for all local rows)
> d_nnz- array containing the number of nonzeros in each row of the diagonal portion of the local matrix (the same for each row within a block) or NULL. You must have a diagonal entry and set it even if it is zero if you plan to factor the matrix.
> o_nz- number of nonzeros per row in the off-diagonal portion of local submatrix (same for all local rows).
> o_nnz- array containing the number of nonzeros in each row of the off-diagonal portion of the local submatrix (the same for each row within a block) or NULL.
>
> I can change this if this is acceptable.