[petsc-users] reusing matrix created with MatCreateMPIAIJWithSplitArrays

[Hassan Raiesi]

Thanks Barry,

That works, however, the code seems to run a bit faster when I destroy and re-create the matrix at each time step as suggested by Dave!, 

I added this right after updating the values of the diagonal and off-diagonal parts,

ierr = PetscObjectStateIncrease((PetscObject)(mat)); 

to avoid calls to  MatAssemblyBegin/MatAssemblyEnd()  and it does the trick!

-H

-----Original Message-----
From: Barry Smith [mailto:bsmith at mcs.anl.gov] 
Sent: Thursday, June 30, 2016 6:17 PM
To: Hassan Raiesi <Hassan.Raiesi at aero.bombardier.com>
Cc: petsc-users at mcs.anl.gov
Subject: Re: [petsc-users] reusing matrix created with MatCreateMPIAIJWithSplitArrays


> On Jun 30, 2016, at 2:40 PM, Hassan Raiesi <Hassan.Raiesi at aero.bombardier.com> wrote:
> 
> Hello,
>  
> We are using PETSC in our CFD code, and  noticed that using “MatCreateMPIAIJWithSplitArrays” is almost 60% faster for large problem size (i.e DOF > 725M, using GAMG each time-step only takes 5sec, compared to 8.3 sec when assembling the matrix one row at a time using matsetvaluesblocked()  as recommended). 
>  
> The problem is that the memory usage goes up after each call to MatCreateMPIAIJWithSplitArrays  to update the matrix values. As MatCreateMPIAIJWithSplitArrays is not supposed to copy the values, do we need to call it each time to update the values? We tried to just update the values of the diagonal and off-diagonal part of the arrays passed to “MatCreateMPIAIJWithSplitArrays”, (the sparsity structure is fixed) but it looks like that the values are not updated, what is the proper way to update the values of the matrix created by MatCreateMPIAIJWithSplitArrays?

   Since you have direct access to the two numeric arrays passed to MatCreateMPIAIJWithSplitArrays() you can simply change the values in those locations

   AND THEN immediately CALL MatAssemblyBegin/MatAssemblyEnd() on the matrix; this will increase the the PETSc object state value for the matrix so the matrix routines (and preconditioner) will know you changed the matrix values. If you don't call the MatAssemblyBegin/MatAssemblyEnd() the preconditioner will think the matrix has not been changed so just use its old values as you observed.

   Barry

Of course if you change any nonzero locations in the matrix you need to destroy the matrix and call MatCreateMPIAIJWithSplitArrays() again. 

>  
>  
> Thank you
>  
> Hassan Raiesi,
> Advanced Aerodynamics Department
> Bombardier Aerospace
>  
> hassan.raiesi at aero.bombardier.com
>  
> 2351 boul. Alfred-Nobel (BAN1)
> Ville Saint-Laurent, Québec, H4S 2A9
>  
>  
>  
> Tél.
>   514-855-5001    # 62204
>  
>  
>  
> <image001.png>
>  
>  
> CONFIDENTIALITY NOTICE - This communication may contain privileged or confidential information.
> If you are not the intended recipient or received this communication 
> by error, please notify the sender and delete the message without copying, forwarding and/or disclosing it.