[petsc-users] saving results

Tue Feb 19 07:49:38 CST 2019

On Tue, Feb 19, 2019 at 8:21 AM Sal Am <tempohoper at gmail.com> wrote:

> Matthew maybe indeed I was not clear and wrong in the wording. What I
> meant is the matrix is undefined, but since it is a finite element method
> (using second order elements). The pattern is like this:
>

Okay, the paper says that the formulation of Maxwell in frequency space is
"well conditioned", and in the paper they solve using
BiCG/Jacobi:

  -ksp_type bicg -pc_type jacobi

Since you are taking thousands of iterates:

  a) They are mistaken and the formulation is not well-conditioned

  b) You have a bug in the discretization

  c) You are outside the parameter range/boundary conditions/forcing they
were talking about for conditioning

Just to get something solving so you can check the discretization, I would

  a) Use SuperLU or MUMPS, -pc_type lu

  b) Then to get bigger use ASM/LU, -pc_type asm -pc_sub_type lu

Then finally

  c) For very big problems, I suspect that PCPATCHcould be made to work
with the right choices, but this is a research problem

  Thanks,

     Matt

> [image: image.png]
> We are indeed solving Maxwell's equations. I have added the paper.
>
> On Tue, Feb 19, 2019 at 12:15 PM Matthew Knepley <knepley at gmail.com>
> wrote:
>
>> On Tue, Feb 19, 2019 at 4:12 AM Sal Am via petsc-users <
>> petsc-users at mcs.anl.gov> wrote:
>>
>>> It is a finite-element problem of an RF antenna dielectric interaction
>>> where all the non-zero elements are on the diagonal of the sparse matrix
>>> (if that is relevant).
>>>
>>
>> I am unsure whether this is what you mean. If you matrix is diagonal, you
>> can invert it exactly in a single step so you
>> do not need a Krylov solver. What equations are you solving? Maxwell?
>> Electrostatic? What finite element are you using?
>>   Thanks,
>>
>>     Matt
>>
>>
>>> More about the matrix: 25Mx25M, 2B non-zeros. I checked the KSPMonitor,
>>> it seems that I have to write my own routine?
>>>
>>>> Running for a Krylov method for tens of thousands of iterations is very
>>>> rarely recommended.
>>>>
>>> GAMG and BCGS is the only ones that have actually worked for me so far,
>>> I increased the max_it because it was not enough with the default one. With
>>> the default tolerance I got ||r(i)||/||b|| in the order of 10^-3, but I
>>> need more accuracy so I increased the tol to 10^-7 and that is when it
>>> crashed after ~51000 iterations.
>>>
>>> @Barry
>>>
>>>> I'm guessing you mange your own time stepping (and nonlinear solver if
>>>> there is one)?
>>>>
>>>
>>> It is the default from the solver (attached the short code).
>>>
>>>> As Jed says it doesn't make sense to save "partial" solutions within
>>>> the linear solver.  Just save it every several time-steps.
>>>>
>>>
>>> Yes maybe I worded it wrong. I did mean to save checkpoints basically
>>> such that the next time it is running it can pick up from where it crashed.
>>>
>>>    Also the code should not be "crashing" at seemingly long times (after
>>>> hours) with a Segmentation fault. Send us the full error message and we'll
>>>> see if there is some way we can help you fix this problem.
>>>>
>>> I have added this before, but seemingly they conclusion was that it is
>>> memory error...although I am not sure how, but I have added the entire
>>> error.
>>>
>>> On Mon, Feb 18, 2019 at 8:21 PM Smith, Barry F. <bsmith at mcs.anl.gov>
>>> wrote:
>>>
>>>>
>>>>    I'm guessing you mange your own time stepping (and nonlinear solver
>>>> if there is one)?
>>>>
>>>>    You can save the solution with a call to VecView() and then reload
>>>> the solution with a VecLoad() but you need to manage any other restart data
>>>> that you may need, like the value of the current time etc.
>>>>
>>>>    As Jed says it doesn't make sense to save "partial" solutions within
>>>> the linear solver.  Just save it every several time-steps.
>>>>
>>>>    Barry
>>>>
>>>>    Also the code should not be "crashing" at seemingly long times
>>>> (after hours) with a Segmentation fault. Send us the full error message and
>>>> we'll see if there is some way we can help you fix this problem.
>>>>
>>>>
>>>> > On Feb 18, 2019, at 9:47 AM, Sal Am via petsc-users <
>>>> petsc-users at mcs.anl.gov> wrote:
>>>> >
>>>> > Is there a function/command line option to save the solution as it is
>>>> solving (and read in the file from where it crashed and keep iterating from
>>>> there perhaps)?
>>>> > Had a seg fault and all the results until that point seems to have
>>>> been lost.
>>>> >
>>>> >
>>>> >
>>>> >
>>>>
>>>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://www.cse.buffalo.edu/~knepley/
>> <http://www.cse.buffalo.edu/~knepley/>
>>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190219/5e3b6c50/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 14363 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190219/5e3b6c50/attachment-0001.png>