[petsc-users] Some general questions

[Matthew Knepley]

On Fri, May 12, 2017 at 2:25 AM, Dave May <dave.mayhem23 at gmail.com> wrote:

>
>
> On 12 May 2017 at 07:50, Matt Baker <mbaker112 at outlook.de> wrote:
>
>> Hello,
>>
>>
>> I have a few questions on how to improve performance of my program. I'm
>> solving Poisson's equation on a (large) 3D FD grid with Dirichlet boundary
>> conditions and multiple right hand sides. I set up the matrix and
>> everything's working fine so far, but I'm sure the solving process could go
>> faster. I know multigrid is generally the best preconditioner in such a
>> case and algebraic multigrid currently works best.
>>
>
> If you use a DMDA for your FD problem, consider using PCMG with Galerkin.
> It will set up a geometric multigrid hierarchy. Depending on the specifics
> of your Poisson problem (constant coefficient versus highly hetegoneous),
> geometric MG is likely superior (faster time to solution) than AMG.
>
>
>>
>> So generally speaking:
>>
>>
>> Should I make the effort of symmetrizising the system matrix? I know how
>> to do it, but it would probably take some time. CG does currently work, but
>> is not competitive against other methods, so I guess the matrix might not
>> be "symmetric enough"?
>>
>
> In its basic form, CG is only guaranteed to converge with with an SPD
> operator.
> If you want to use CG, definitely do the work and make the operator
> symmetric.
>

For Poisson, CG is never, ever ever, ever ever faster than Full Multigrid
(FMG). Don't use it. All the people publishing that are idiots :)
but of course try it out for yourself with

  -pc_mg_type full

It should converge to discretization error in 1 iterate if the smoother is
strong enough (you might need to use 2 iterates on the downsmooth).

>
>> For the various multigrid preconditioners: I always read that the problem
>> should be solved exactly on the coarsest grid, but wouldn't an iterative
>> solver do the same job if its provided accuracy is high enough, since the
>> coarse discretization and the subsequent interpolation process introduce
>> errors themselves?
>>
>
> Yes iterative can work well. If your Poisson problem has a constant
> coefficient, rtol 1.0e-1 is likely a sufficient tolerance to use for an the
> coarse grid solve (e.g. overall convergence of solve won't be affected). If
> the Poisson problem has a highly variable coefficient (jumps of O(1e3) or
> more), or it has very large gradients say 1e3 variation over a few cells,
> then you will have to perform a more accurate iterative coarse level solve
> (say rtol 1e-4 to 1e-6). Note that the numbers for rtol I quote are purely
> empirical.
>

Always compare to direct. I don't think anything beats direct on problems
the size of your coarse problem. If iterative is winning, likely
your coarse problem is too big. However, again you can try it yourself
easily with options.

   Matt

>
>> I submit my program to a batch system, but PETSc was compiled on the
>> login node with different hardware. Is this affecting performance? What
>> parts of the configuration process should I perform on a compute node then?
>>
> If the login and compute nodes are fundamentally different, you should
> configure petsc with the option
>  --with-batch
> and following the instructions.
>
> Thanks,
>   Dave
>
>>
>> Thanks.
>>
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170512/83b98fc0/attachment-0001.html>