[mumps-dev] Re: Question on using MUMPS in PETSC

[Hong Zhang]

Randy,

I'll check it.
Did you use
/src/ksp/ksp/examples/tutorials/ex2.c ?

Can you give me the petsc version, num of processors, and the runtime 
options used.

Thanks,

Hong

On Fri, 1 Aug 2008, Randall Mackie wrote:

> Barry,
>
> No, this is the same program I've used quite successfully using iterative
> methods within PETSc for years. Each processors portion of the matrix
> is constructed on the individual processors.
>
> In fact, I downloaded and recompiled PETSc to use SUPERLU, and using the 
> exact
> same program, but changing the matrix type from aijmumps to superlu_dist,
> and it worked just fine.
>
> So, I'm not sure why MUMPS is not working.
>
> Randy
>
>
> Barry Smith wrote:
>>
>>   Are you sure you are not constructing the original matrix with all its 
>> rows and columns
>> on the first process?
>>
>>   Barry
>> On Aug 1, 2008, at 5:49 PM, Randall Mackie wrote:
>> 
>>> In fact, during the Analysis step, the max amount of memory 300 Mbytes is
>>> used by one process. However, during the Factorization stage, that same 
>>> process
>>> then starts to increase in memory, with all the other processes staying 
>>> the same.
>>> 
>>> I've re-run this several times using different numbers of processors, and 
>>> I
>>> keep getting the same behavior.
>>> 
>>> 
>>> 
>>> Randy
>>> 
>>> 
>>> Jean-Yves L Excellent wrote:
>>>> Hi,
>>>> Clearly in MUMPS processor 0 uses more memory during
>>>> the analysis step because the analysis is sequential.
>>>> So until we provide a parallel analysis, processor 0
>>>> is gathering the graph of the matrix from all other
>>>> processors to perform the analysis. But that memory
>>>> is freed at the end of the analysis so it should
>>>> not affect the factorization.
>>>> Thanks for letting us know if you have more information.
>>>> Regards,
>>>> Jean-Yves
>>>> Hong Zhang wrote:
>>>>> 
>>>>> Randy,
>>>>> The petsc interface does not create much of extra
>>>>> memories.
>>>>> The analysis phase of MUMPS solver is sequential - which might causes 
>>>>> one process blow up with memory.
>>>>> I'm forwarding this email to the mumps developer
>>>>> for their input.
>>>>> 
>>>>> Jean-Yves,
>>>>> What do you think about the reported problem
>>>>> (see attached below)?
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Hong
>>>>> 
>>>>> On Thu, 31 Jul 2008, Randall Mackie wrote:
>>>>> 
>>>>>> Barry,
>>>>>> 
>>>>>> I don't think it's the matrix - I saw the same behavior when I ran your
>>>>>> ex2.c program and set m=n=5000.
>>>>>> 
>>>>>> Randy
>>>>>> 
>>>>>> 
>>>>>> Barry Smith wrote:
>>>>>>>
>>>>>>>   If m and n are the number of rows and columns of the sparse matrix 
>>>>>>> (i.e. it is
>>>>>>> tiny problem) then please
>>>>>>> send us matrix so we can experiment with it to petsc-maint at mcs.anl.log
>>>>>>>
>>>>>>>  You can send us the matrix by simply running with -ksp_view_binary 
>>>>>>> and
>>>>>>> sending us the file binaryoutput.
>>>>>>>
>>>>>>>   Barry
>>>>>>> 
>>>>>>> On Jul 31, 2008, at 5:56 PM, Randall Mackie wrote:
>>>>>>> 
>>>>>>>> When m = n = small (like 50), it works fine. When I set m=n=5000, I 
>>>>>>>> see
>>>>>>>> the same thing, where one process on the localhost is taking >4 G of 
>>>>>>>> RAM,
>>>>>>>> while all other processes are taking 137 M.
>>>>>>>> 
>>>>>>>> Is this the standard behavior for MUMPS? It seems strange to me.
>>>>>>>> 
>>>>>>>> Randy
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Matthew Knepley wrote:
>>>>>>>>> Does it work on KSP ex2?
>>>>>>>>> Matt
>>>>>>>>> On Thu, Jul 31, 2008 at 4:35 PM, Randall Mackie 
>>>>>>>>> <rlmackie862 at gmail.com> wrote:
>>>>>>>>>> I've compiled PETSc with MUMPS support, and I'm trying to run a 
>>>>>>>>>> small test
>>>>>>>>>> problem, but I'm having some problems. It seems to begin just fine, 
>>>>>>>>>> but
>>>>>>>>>> what I notice is that on one process (out of 64), the memory just 
>>>>>>>>>> keeps
>>>>>>>>>> going up and up and up until it crashes, while on the other 
>>>>>>>>>> processes,
>>>>>>>>>> the memory usage is reasonable. I'm wondering if anyone might have 
>>>>>>>>>> any idea
>>>>>>>>>> why? By the way, my command file is like this:
>>>>>>>>>> 
>>>>>>>>>> -ksp_type preonly
>>>>>>>>>> -pc_type lu
>>>>>>>>>> -mat_type aijmumps
>>>>>>>>>> -mat_mumps_cntl_4 3
>>>>>>>>>> -mat_mumps_cntl_9 1
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Randy
>>>>>>>>>> 
>>>>>>>>>> ps. This happens after the analysis stage and in the factorization 
>>>>>>>>>> stage.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>> 
>> 
>
>