Verry time consuming first matrix assembly

[devteam]

Hi all, Hi Mat,

I have to apologize. I was using the info displayed by Mat_GetInfo function.
If instead I use -log_info or -info then I see that my matrix is not well 
preallocated.

I think that I have to debug my stuff and stop bothering you
Sorry for that,
regards,
Etienne Perchat

----- Original Message ----- 
From: "Matthew Knepley" <knepley at gmail.com>
To: <petsc-users at mcs.anl.gov>
Sent: Wednesday, May 23, 2007 3:22 PM
Subject: Re: Verry time consuming first matrix assembly


> Please send the results of -info -log_summary.
>
>  Thanks,
>
>    Matt
>
> On 5/23/07, devteam <devteam at transvalor.com> wrote:
>> Hello,
>>
>>
>>
>> I have a FEM code that perform with several meshes. We handle
>> interactions/contacts between bodies/meshes by assembling coupling terms
>> between the « contact » nodes of the meshes.
>>
>>
>>
>> I have a very large bandwidth : The numbering of the whole problem is 
>> done
>> mesh by mesh (my problem is of size 4*N where N is the total number of 
>> node
>> and N = N1 + N2 + .. + Nq with Nq the number of nodes of mesh q. Nodes of
>> mesh q are numbered from N1+N2 + .. + N(q-1) + 1 to N1+N2+..+Nq)
>>
>>
>>
>> Typically N # 100000 to 1000000.
>>
>>
>>
>>
>>
>> The matrix is a MPIBAIJ one and the d_nnz and o_nnz info are specified 
>> when
>> created.
>>
>> It is filled using MatSetValuesBlockedLocal in mode ADD_VALUES.
>>
>>
>>
>> At each increment of my time step scheme, the connections between mesh 
>> nodes
>> may change and I have to rebuild the matrix.
>>
>> It appears that the CPU required for the first matrix assembly is very 
>> large
>> (three to four times the CPU for 1 system solve) and depend on the number 
>> of
>> meshes : if I have only one mesh of an equivalent size the assembly CPU
>> remain almost zero.
>>
>>
>>
>> So I wonder what is causing the assembly to last so much ? I was thinking
>> that the system solve would have been longer because of my large 
>> bandwidth
>> but I don't understand why it is the matrix assembly that last so much.
>>
>>
>>
>> I have investigated using Mat_info but all seems to be correct : the 
>> number
>> of malloc during MatSetValue is always zero and I have a ratio non zero 
>> used
>> / non zero allocated between 1% and 10% (same ratio than when I have only
>> one mesh).
>>
>> I have tested using a simple SOR preconditionner instead of ILU, 
>> wondering
>> if it was the precond assembly that last long because of the bandwidth, 
>> but
>> it does not change anything !
>>
>>
>>
>> Thanks a lot for any remarks or any tip.
>>
>>
>>
>> Best regards,
>>
>> Etienne Perchat
>>
>>
>>
>
>
> -- 
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which
> their experiments lead.
> -- Norbert Wiener
>
>
>
>