[petsc-users] MatMPIBAIJSetPreallocation() diagonal block:

[Mark Adams]

On Mon, Jan 17, 2022 at 6:40 PM Ferrand, Jesus A. <FERRANJ2 at my.erau.edu>
wrote:

> Greetings!
>
> I have a question about preallocating memory for a parallel BAIJ matrix.
> From the documentation of  MatMPIBAIJSetPreallocation(), the preallocation
> is divided between the so-called "diagonal" and "off-diagonal" sub
> matrices. In the example from the documentation, the following illustration
> is given for the portion of a matrix that owns rows 3,4, and 5 of some
> 12-column matrix:
>
>           0 1 2 3 4 5 6 7 8 9 10 11
>          --------------------------
>    row 3 |o o o d d d o o o o  o  o
>    row 4 |o o o d d d o o o o  o  o
>    row 5 |o o o d d d o o o o  o  o
>          --------------------------
>
> I have a few questions: about MPIBAIJ matrices, I am somewhat confused by
> the description:
>
>    1. If the rank owned, say 5 rows instead of 3 on the same matrix,
>    would the "d" block become a 5 by 5? as in:
>
>
When you create a matrix you specify how many local rows -- and columns (n)
-- that you want.

int MatCreateMPIBAIJ(MPI_Comm comm,int bs,int m,int n,int M,int N,int
d_nz,int *d_nnz,int o_nz,int *o_nnz,Mat *A)




>
>    1.
>
>              0 1 2 3 4 5 6 7 8 9 10 11
>             --------------------------
>       row 3 |o o o d d d d d o o  o  o
>       row 4 |o o o d d d d d o o  o  o
>       row 5 |o o o d d d d d o o  o  o
>       row 6 |o o o d d d d d o o  o  o
>       row 7 |o o o d d d d d o o  o  o
>             --------------------------
>
>
>
>    1. Is there a way to not have to deal with "d" blocks and "o" blocks
>    separately? I know how many nonzero blocks are in each row of my matrix but
>    I can't easily determine which of those zeros are in the "d" block or "o"
>    block of my ranks.
>
>
No. It is best for performance to give an upper bound for each.

>
>    1. The block size "bs", that is the size of each individual "d" or "o"
>    as shown in the diagrams right?
>
> d and o have the same "size" but the API is such that

>
>    1. TRUE/FALSE: The parameters d_nz, o_nz, d_nnz,  and o_nnz, those
>    specify the number of "d" and "o" blocks that are nonzero (where each "d"
>    or "o" is its own dense submatrix), not the actual number of nonzeros in
>    the rows, right? Say the block size is 3 and I set d_nz to 2, then for the
>    diagram from the documentation this translates to 54 actual nonzeros in the
>    diagonal block, right?
>
>
FALSE. This is a little confusing but the interface is the same for blocked
or unblocked versions of the same matrix. Thus you can set from the command
line -mat_type aij or baij and they both work. Keep that in mind to figure
out the API.
This first example would work with a bs==3 or bs==1, which is an unblocked
matrix. Your bs==5 example will not work with a blocked BAIJ matrix.

Now performance sensitive functions like MatSetValues need blocked versions
(https://petsc.org/main/docs/manualpages/Mat/MatSetValuesBlocked.html).
MatSetValues will work for a blocked matrix, but it is slower than
exploiting the blocks with the blocked version. To be flexible, as I
described with command line options, you would need to query the matrix to
figure out which type it is and have two versions of the
assembly (MatSetValues) code.
In practice, people often hardwire the code to a BAIJ matrix to avoid this
complexity if they don't care about this flexibility.

>
>    1.
>
>
>
> Sincerely:
>
> *J.A. Ferrand*
>
> Embry-Riddle Aeronautical University - Daytona Beach FL
>
> M.Sc. Aerospace Engineering | May 2022
>
> B.Sc. Aerospace Engineering
>
> B.Sc. Computational Mathematics
>
>
>
> Sigma Gamma Tau
>
> Tau Beta Pi
>
> Honors Program
>
>
>
> *Phone:* (386)-843-1829
>
> *Email(s):* ferranj2 at my.erau.edu
>
>     jesus.ferrand at gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20220117/c3a3f66d/attachment-0001.html>