[petsc-users] Enhancing MatScale computing time

[Matthew Knepley]

On Thu, Oct 22, 2020 at 3:23 PM Antoine Côté <Antoine.Cote3 at usherbrooke.ca>
wrote:

> Hi,
>
> I'm working with a 3D DMDA, with 3 dof per "node", used to create a sparse
> matrix Mat K. The Mat is modified repeatedly by the program, using the
> commands (in that order) :
>
> MatZeroEntries(K)
> In a for loop : MatSetValuesLocal(K, 24, irow, 24, icol, vals, ADD_VALUES)
> MatAssemblyBegin(K, MAT_FINAL_ASSEMBLY)
> MatAssemblyEnd(K, MAT_FINAL_ASSEMBLY)
> MatDiagonalScale(K, vec1, vec1)
> MatDiagonalSet(K, vec2, ADD_VALUES)
>
> Computing time seems high and I would like to improve it. Running tests
> with "-log_view" tells me that MatScale() is the bottle neck (50% of total
> computing time) . From manual pages, I've tried a few tweaks :
>
>    - DMSetMatType(da, MATMPIBAIJ) : "For problems with multiple degrees
>    of freedom per node, ... BAIJ can significantly enhance performance",
>    Chapter 14.2.4
>    - Used MatMissingDiagonal() to confirm there is no missing diagonal
>    entries : "If the matrix Y is missing some diagonal entries this routine
>    can be very slow", MatDiagonalSet() manual
>    - Tried MatSetOption()
>       - MAT_NEW_NONZERO_LOCATIONS == PETSC_FALSE : to increase assembly
>       efficiency
>       - MAT_NEW_NONZERO_LOCATION_ERR == PETSC_TRUE : "When true, assembly
>       processes have one less global reduction"
>       - MAT_NEW_NONZERO_ALLOCATION_ERR == PETSC_TRUE : "When true,
>       assembly processes have one less global reduction"
>       - MAT_USE_HASH_TABLE == PETSC_TRUE : "Improve the searches during
>       matrix assembly"
>
> According to "-log_view", assembly is fast (0% of total time), and the
> use of a DMDA makes me believe preallocation isn't the cause of performance
> issue.
>
> I would like to know how could I improve MatScale(). What are the best
> practices (during allocation, when defining Vecs and Mats, the DMDA, etc.)?
> Instead of MatDiagonalScale(), should I use another command to obtain the
> same result faster?
>

Something is definitely strange. Can you please send the output of

  -log_view -info :mat

  Thanks,

     Matt


> Thank you very much!
>
> Antoine Côté
>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20201022/9baf75a7/attachment.html>