[petsc-users] Enhancing MatScale computing time

[Dave May]

On Thu 22. Oct 2020 at 21:23, Antoine Côté <Antoine.Cote3 at usherbrooke.ca>
wrote:

> Hi,
>
> I'm working with a 3D DMDA, with 3 dof per "node", used to create a sparse
> matrix Mat K. The Mat is modified repeatedly by the program, using the
> commands (in that order) :
>
> MatZeroEntries(K)
> In a for loop : MatSetValuesLocal(K, 24, irow, 24, icol, vals, ADD_VALUES)
> MatAssemblyBegin(K, MAT_FINAL_ASSEMBLY)
> MatAssemblyEnd(K, MAT_FINAL_ASSEMBLY)
> MatDiagonalScale(K, vec1, vec1)
> MatDiagonalSet(K, vec2, ADD_VALUES)
>

Why not just assemble the entire operator you seek locally in vals?
You would then avoid the calls to MatDiagonalScale and MatDiagonalSet by
instead calling VecGetArrayRead on vec1 and vec2 and using the local parts
of these vectors you need with vals. You probably need to scatter vec1,
vec2 first before VecGetArrayRead.

Thanks,
Dave




> Computing time seems high and I would like to improve it. Running tests
> with "-log_view" tells me that MatScale() is the bottle neck (50% of total
> computing time) . From manual pages, I've tried a few tweaks :
>
>    - DMSetMatType(da, MATMPIBAIJ) : "For problems with multiple degrees
>    of freedom per node, ... BAIJ can significantly enhance performance",
>    Chapter 14.2.4
>    - Used MatMissingDiagonal() to confirm there is no missing diagonal
>    entries : "If the matrix Y is missing some diagonal entries this routine
>    can be very slow", MatDiagonalSet() manual
>    - Tried MatSetOption()
>       - MAT_NEW_NONZERO_LOCATIONS == PETSC_FALSE : to increase assembly
>       efficiency
>       - MAT_NEW_NONZERO_LOCATION_ERR == PETSC_TRUE : "When true, assembly
>       processes have one less global reduction"
>       - MAT_NEW_NONZERO_ALLOCATION_ERR == PETSC_TRUE : "When true,
>       assembly processes have one less global reduction"
>       - MAT_USE_HASH_TABLE == PETSC_TRUE : "Improve the searches during
>       matrix assembly"
>
> According to "-log_view", assembly is fast (0% of total time), and the
> use of a DMDA makes me believe preallocation isn't the cause of performance
> issue.
>
> I would like to know how could I improve MatScale(). What are the best
> practices (during allocation, when defining Vecs and Mats, the DMDA, etc.)?
> Instead of MatDiagonalScale(), should I use another command to obtain the
> same result faster?
>
> Thank you very much!
>
> Antoine Côté
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20201023/799ff90b/attachment.html>