[petsc-users] MatColoring

[Barry Smith]

  It is unlikely that the true number of colors is much less than 230 so I don't think you can save by reducing the number of colors more. 

  Are you using IS_COLORING_LOCAL it eliminates much parallel communication so you would definitely benefit by using this, depending on how your FormFunction is written you may need to refactor your code slightly. 

  There is also a hidden parameter which is a tradeoff of memory usage vs speed. If memory is not an issue for you then you could change this parameter. It is computed in MatFDColoringCreate_MPIXAIJ() called bcols You can just hardware bcols to iscoloring->n This may help a bit.

   The only way to drastically improve the time after the above is to reorganize the PETSc code and your code to evaluate multiple function evaluations "at the same time" but this is a major project (it has to be done for GPUs).

  MATCOLORINGMIS doesn't exist, it is a typo in the docs.

  Barry


> On May 11, 2022, at 2:25 PM, Jorti, Zakariae <zjorti at lanl.gov> wrote:
> 
> Hello,
> 
> I have used the -mat_coloring_view and -mat_fd_coloring_view flags as you suggested and it turns out that the coloring used was MATCOLORINGSL. 
> I have also tried other methods and obtained the following numbers of function evaluations per Jacobian building:   
> - MATCOLORINGJP : 593
> - MATCOLORINGID : 327
> - MATCOLORINGSL : 321
> - MATCOLORINGGREEDY : 316
> - MATCOLORINGLF : 230
> 
> Is there any parameter/trick we can try to further cut down the number of function evaluations? And also how could we use MATCOLORINGMIS as this latter does not appear in the MatColoringType options?
> Many thanks.
> 
> Zakariae
> From: Tang, Qi <tangqi at msu.edu <mailto:tangqi at msu.edu>>
> Sent: Tuesday, May 10, 2022 10:51:07 AM
> To: Barry Smith; petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>
> Cc: Jorti, Zakariae; Tang, Xianzhu
> Subject: [EXTERNAL] Re: [petsc-users] MatColoring
>  
> We are using SNES + TS + dmstag. The current bottleneck is the number of residual evaluation (more than 300 per Jacobian building using the default coloring from dmstag). We talked to Patrick and we are not sure how to improve further.
> 
> So it looks like we should play with mat_coloring_type and see if others give us better performance.
> 
> If there is anything else we can play with, please also let us know. We also lag Jacobian and only build once every three Newton iterations, which works well. Thanks,
> 
> Qi
> 
> 
>> On May 10, 2022, at 10:35 AM, Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>> wrote:
>> 
>> 
>>   This depends to some degree on how you are accessing applying the Jacobian construction process. 
>> 
>>    If you are using SNES or TS then the SNES object handles most of the work of organizing the PETSc objects needed to compute the Jacobian and you can control the choices via the options database. 
>> 
>>    With SNES/TS using a DM you can skip calling SNESSetJacobian() and simply use the option -snes_fd_color to have SNES compute the Jacobian for you. By default, the coloring is obtained from the DM (which is generally the best available coloring), you can have it use a coloring computed directly from the matrix structure with the options -snes_fd_color_use_mat -mat_coloring_type  jp power sl lf ld or greedy (see MatColoringType, MatColoringSetFromOptions) Use  -mat_fd_coloring_view to get information on the computation of the Jacobian from the coloring. Use  -mat_coloring_view to get information on the coloring used. (see MatFDColoringSetFromOptions)
>> 
>>   If you are using SNES/TS but not using a DM you need to compute the nonzero structure of the matrix yourself into J and call SNESSetJacobian(snes,J,J,SNESComputeJacobianDefaultColor,matfdcoloring); you need to create the matfdcoloring object with MatFDColoringCreate() using an iscoloring you obtained with MatColoringCreate() etc.  The same command line arguments as above allow you to control the coloring algorithm used and to view them etc.
>> 
>>   Barry
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>>> On May 10, 2022, at 11:40 AM, Jorti, Zakariae via petsc-users <petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>> wrote:
>>> 
>>> Hi, 
>>> 
>>> I am solving a non-linear problem and using a finite difference approximation with coloring to compute the Jacobian matrix.
>>> 
>>> There are several coloring algorithms available in PETSc as indicated here:
>>> https://petsc.org/release/docs/manualpages/Mat/MatColoring.html <https://urldefense.com/v3/__https://petsc.org/release/docs/manualpages/Mat/MatColoring.html__;!!HXCxUKc!1czn7o25s7aYCp0mPqVxWzr-S-Fc0qBskSUMuqiRAf5CiUQoJAYOwuyAak2yJhKKkULePn_T7-fyd0U$>
>>> 
>>> And I was wondering how to switch from one to another in the Jacobian setup routine and also how to check which coloring algorithm I am currently using.
>>> 
>>> Thank you.
>>> 
>>> Zakariae Jorti

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20220511/8efe80ac/attachment-0001.html>