[petsc-users] parallelize matrix assembly process

[김성익]

Hello Mark,

Following your comments,
I did run with '-info' and the outputs are as below
[image: image.png]
Global matrix seem to have preallocated well enough
And, ass I said earlier in the former email, If I run this code with mpi ,
It will be 70,000secs..
In this case, What is the problem?

And my loops already iterates over elements.
element_vec is just 1~125,000 array. for getting proper element index in
each process.
That example code is just simple schematic of my code.

Thanks,
Hyung Kim

2022년 12월 12일 (월) 오후 10:24, Mark Adams <mfadams at lbl.gov>님이 작성:

> Hi Hyung,
>
> First, verify that you are preallocating correctly.
> Run with '-info' and grep on "alloc" in the large output that you get.
> You will see lines like "number of mallocs in assembly: 0". You want 0.
> Do this with one processor and the 8.
>
> I don't understand your loop. You are iterating over vertices. You want to
> iterate over elements.
>
> Mark
>
>
>
> On Mon, Dec 12, 2022 at 6:16 AM 김성익 <ksi2443 at gmail.com> wrote:
>
>> Hello,
>>
>>
>> I need some keyword or some examples for parallelizing matrix assemble
>> process.
>>
>> My current state is as below.
>> - Finite element analysis code for Structural mechanics.
>> - problem size : 3D solid hexa element (number of elements : 125,000),
>> number of degree of freedom : 397,953
>> - Matrix type : seqaij, matrix set preallocation by using
>> MatSeqAIJSetPreallocation
>> - Matrix assemble time by using 1 core : 120 sec
>>    for (int i=0; i<125000; i++) {
>>     ~~ element matrix calculation}
>>    matassemblybegin
>>    matassemblyend
>> - Matrix assemble time by using 8 core : 70,234sec
>>   int start, end;
>>   VecGetOwnershipRange( element_vec, &start, &end);
>>   for (int i=start; i<end; i++){
>>    ~~ element matrix calculation
>>    matassemblybegin
>>    matassemblyend
>>
>>
>> As you see the state, the parallel case spent a lot of time than
>> sequential case..
>> How can I speed up in this case?
>> Can I get some keyword or examples for parallelizing assembly of matrix
>> in finite element analysis ?
>>
>> Thanks,
>> Hyung Kim
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20221212/f61116ea/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 44388 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20221212/f61116ea/attachment-0001.png>