Multiple communicators

[Barry Smith]

On Nov 24, 2009, at 7:13 PM, irfan.khan at gatech.edu wrote:

> I use seperete communicators for fluid and solid ranks in a fluid- 
> structure interaction code. I use PETSc tools (KSP) to solve for the  
> solid phase (FEA) which is carried out under a different  
> communicator (FEA_Comm).
>
> Unless I am doing something wrong, I have found that the command  
> line options: -ksp_type; -mat_partitioning_type, don't work if there  
> are 2 communicators.

    Are you sure that they don't work? Or is it just printing the  
message?

WARNING! There are options you set that were not used!
WARNING! could be spelling mistake, etc!
Option left: name:-ksp_type value: cg
Option left: name:-mat_partitioning_type value: parmetis

    Any option that is not accessed on PROCESS ZERO will be listed  
here as unused, even if it may be used on some process.  I suppose we  
could/should fix this by adding some complicated communication that  
determines just the options that are never used. I never liked this  
warning, only added it after pressure from people who couldn't type.

> But for single communicator they work. Please see the attached files  
> containing output of -log_summary for 2 different codes (scroll down  
> to the end). One with single communicator and the other with two  
> communicators. Both the output have been generated with the same  
> PETSc compilation.
>
> Futher -log_summary results are not able to detect all the events  
> that have been registered if they are not in the WORLD communicator.

    Is your concern here that it does not print the name of the event,  
it leaves the name blank? If so this is likely the type of problem as  
the print of options; we don't have a mechanism to get them to process  
0 to print.

    In summary, yes these are some weaknesses of PETSc. If you switch  
the two communicators you create and use the one that has process 0 of  
MPI_COMM_WORLD for the PETSc stuff then it will look nicer and hide  
these two problems.

    Barry

>
>
> Thank you
> Irfan
> ----- Original Message -----
> From: "Matthew Knepley" <knepley at gmail.com>
> To: "PETSc users list" <petsc-users at mcs.anl.gov>
> Sent: Tuesday, November 24, 2009 8:03:00 PM GMT -05:00 US/Canada  
> Eastern
> Subject: Re: Multiple communicators
>
> On Tue, Nov 24, 2009 at 6:59 PM, <irfan.khan at gatech.edu> wrote:
> Hello
>
> Does the procedure for providing command line options and to obtain  
> profiling data through -log_summary change if there are multiple MPI  
> communicators in a petsc code?
>
> There is always WORLD, so it should be fine unless I misunderstand  
> your question.
>
>   Matt
>
> If so, can somebody point me to information, references on how to  
> provide command line options if there are multiple communicators?
>
> Thank you
> Irfan
>
> -- 
> PhD Candidate
> G.W. Woodruff School of Mechanical Engineering
> Georgia Institute of Technology
> Atlanta, GA (30332)
>
>
>
> -- 
> What most experimenters take for granted before they begin their  
> experiments is infinitely more interesting than any results to which  
> their experiments lead.
> -- Norbert Wiener
>
>
> -- 
> PhD Candidate
> G.W. Woodruff School of Mechanical Engineering
> Georgia Institute of Technology
> Atlanta, GA (30332)
> <1communicator.out><2communicators.out>