assembly

[Barry Smith]

    The matstash has a concept of preallocation also. During the first  
setvalues
it is allocating more and more memory for the stash. In the second  
setvalues
the stash is large enough so does not require any addition allocation.

    You can use the option -matstash_initial_size <size> to allocate  
enough space
initially so that the first setvalues is also fast. It does not look  
like there is a way
coded to get the <size> that you should use. It should be set to the  
maximum nonzeros
any process has that belongs to other processes. The stash handling  
code is
in src/mat/utils/matstash.c, perhaps you can figure out how to  
printout with PetscInfo()
the sizes needed?


    Barry



On Feb 2, 2008, at 12:30 PM, Thomas Geenen wrote:

> On Saturday 02 February 2008 18:33, Hong Zhang wrote:
>> On Sat, 2 Feb 2008, Thomas Geenen wrote:
>>> Dear Petsc users,
>>>
>>> I would like to understand what is slowing down the assembly phase  
>>> of my
>>> matrix. I create a matrix with MatCreateMPIAIJ i make a rough  
>>> guess of
>>> the number of off diagonal entries and then use a conservative  
>>> value to
>>> make sure I do not need extra mallocs. (the number of diagonal  
>>> entries is
>>> exact)
>>> next i call MatSetValues and MatAssemblyBegin, MatAssemblyEnd.
>>> The first time i call MatSetValues and MatAssemblyBegin,
>>> MatAssemblyEnd it takes about 170 seconds
>>> the second time 0.3 seconds.
>>> I run it on 6 cpu's and I do fill quit a number of row-entries on  
>>> the
>>> "wrong" cpu. However thats also the case the second run. I checked
>>> that there are no additional mallocs
>>> MatGetInfo info.mallocs=0 both after MatSetValues and after
>>> MatAssemblyBegin, MatAssemblyEnd.
>>
>> Run your code with the option '-log_summary' and check which function
>> call dominates the execution time.
>
> the time is spend in MatStashScatterGetMesg_Private
>
>>
>>> I run it on 6 cpu's and I do fill quit a number of row-entries on  
>>> the
>>> "wrong" cpu.
>>
>> Likely, the communication that sending the entries to the
>> corrected cpu consume the time. Can you fill the entries in the
>> correct cpu?
>
> the second time the entries are filled on the wrong CPU as well.
> i am curious about the difference in time between run 1 and 2.
>
>>
>> Hong
>>
>>> cheers
>>> Thomas
>