[petsc-users] Lengthy time to assemble the matrix

[Dave May]

You can easily check whether your matrix preallocation is correct by
running your application with the
following additional command line args
  -info | grep malloc

If the preallocation is correct you'll see something like this.
  [0] MatAssemblyEnd_SeqAIJ(): Number of mallocs during MatSetValues() is 0
If the malloc count is not zero, your preallocation algorithm is the problem.

Dave



On 19 November 2010 01:19, Barry Smith <bsmith at mcs.anl.gov> wrote:
>
>  This is due to not setting the proper preallocation.  If
>
> (1) you are using DAGetMatrix() to get your matrix then you have not provided the DA the correct stencil information that corresponds to matrix you are setting. For example you used a stencil width of 1 in the DA but are setting values two widths array into the matrix
>
> (2) you are calling MatCreateMPIAIJ() or MatCreateMPIBAIJ() or using MatCreate() and then setting the type. See the manual pages for MatCreateMPIAIJ(), MatMPIAIJSetPreallocation() etc and look in the users manual for the chapter on optimization.
>
>  With proper allocation the time will come down dramatically.
>
>   Barry
>
> On Nov 18, 2010, at 6:14 PM, Li, Zhisong (lizs) wrote:
>
>>  Hi, Petsc Team,
>>
>> I noticed that the Petsc program often cost a lot of time to assemble a matrix, particularly when DOF is large. For example, I write a 3D structured CFD code with a linear system of 40 x 22 x 20 in dimension and DOF = 6. In a sequential computation, it takes more than 20 minutes to finish assembling the matrix before ksp operates. This same problem should take only a few seconds for a commercial CFD software to initialize.
>>
>> The processor is 2.67G Hz in speed with sufficient RAM (6GB). The largest matrix stencil is 19 points. I typically use the function: MatSetValuesStencil(A, 1, &abc, 19, def, val, INSERT_VALUES). I wonder the assembling can be faster if abc is an array of multiple matrix row coordinates. But in the case of DOF >1, this seems difficult to implement.
>>
>> Is this normal for Petsc or do we have any means to improve this?
>>
>>
>> Thank you.
>>
>>
>> Zhisong Li
>
>