[petsc-users] Problem with solving Poisson eqn for some cases
Matthew Knepley
knepley at gmail.com
Mon Mar 26 07:10:37 CDT 2018
On Mon, Mar 26, 2018 at 1:15 AM, TAY wee-beng <zonexo at gmail.com> wrote:
>
> On 24/3/2018 6:00 AM, Matthew Knepley wrote:
>
> On Fri, Mar 23, 2018 at 5:23 AM, TAY wee-beng <zonexo at gmail.com> wrote:
>
>>
>> On 21/3/2018 7:47 PM, Matthew Knepley wrote:
>>
>> On Wed, Mar 21, 2018 at 4:59 AM, TAY wee-beng <zonexo at gmail.com> wrote:
>>
>>>
>>> On 19/3/2018 6:32 PM, Matthew Knepley wrote:
>>>
>>> On Mon, Mar 19, 2018 at 5:19 AM, TAY wee-beng <zonexo at gmail.com> wrote:
>>>
>>>>
>>>> On 17/3/2018 1:15 AM, Matthew Knepley wrote:
>>>>
>>>> On Fri, Mar 16, 2018 at 12:54 PM, TAY wee-beng <zonexo at gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>> On 15/3/2018 6:21 PM, Matthew Knepley wrote:
>>>>>
>>>>> On Thu, Mar 15, 2018 at 3:51 PM, TAY wee-beng <zonexo at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm running a CFD code which solves the momentum and Poisson eqns.
>>>>>>
>>>>>> Due to poor scaling with HYPRE at higher cpu no., I decided to try
>>>>>> using PETSc with boomeramg and gamg.
>>>>>>
>>>>>> I tested for some small cases and it work well. However, for the
>>>>>> large problem which has poor scaling, it gives an error when I change my
>>>>>> Poisson solver from pure HYPRE to PETSc with boomeramg and gamg.
>>>>>>
>>>>>> The error is :
>>>>>>
>>>>>> Caught signal number 11 SEGV: Segmentation Violation, probably memory
>>>>>> access out of range
>>>>>>
>>>>>> I tried using:
>>>>>>
>>>>>> -poisson_ksp_type richardson -poisson_pc_type hypre
>>>>>> -poisson_pc_type_hypre boomeramg
>>>>>>
>>>>>> -poisson_ksp_type gmres -poisson_pc_type hypre -poisson_pc_type_hypre
>>>>>> boomeramg
>>>>>>
>>>>>> -poisson_pc_type gamg -poisson_pc_gamg_agg_nsmooths 1
>>>>>>
>>>>>> but they all gave similar error.
>>>>>>
>>>>>> So why is this so? How should I troubleshoot? I am now running a
>>>>>> debug ver of PETSc to check the error msg.
>>>>>
>>>>>
>>>>> 1) For anything like this, we would like to see a stack trace from the
>>>>> debugger or valgrind output.
>>>>>
>>>>> 2) We do have several Poisson examples. Does it fail for you on those?
>>>>>
>>>>> Hi,
>>>>>
>>>>> Can you recommend me some suitable egs? Esp in Fortran?
>>>>>
>>>>
>>>> Here is 2D Poisson
>>>>
>>>> https://bitbucket.org/petsc/petsc/src/4b6141395f14f0c7d141
>>>> 5a2ff0158eec75a27d63/src/snes/examples/tutorials/ex5f.F90?at
>>>> =master&fileviewer=file-view-default
>>>>
>>>>
>>>>>
>>>>> Hi,
>>>
>>> I tried to use the different options like ML, MG BoomerAMG and while
>>> they worked for the small problem, it failed for the large problem. I
>>> checked and compared the 2D Poisson example with my 3D Poisson subroutine.
>>> I found that my subroutine is very similar to my momentum subroutine, which
>>> is based on KSPSolve:
>>>
>>> 0. DMDACreate, DMDACreate3d etc
>>> 1. Assemble matrix
>>> 2. KSPSetOperators, KSPSetType etc
>>> 3. KSPSolve
>>>
>>> However, the 2D Poisson example uses SNESSetDM, SNESSolve etc. So does
>>> it matter if I use SNESSolve or KSPSolve?
>>>
>>
>> Not if you are using Algebraic Multigrid. "It failed for the large
>> problem" is not descriptive enough. Please send the output of
>>
>> -ksp_view -ksp_monitor_true_residual
>>
>>
>>> I also didn't set any nullspace. Is this required for a Poisson eqn
>>> solve?
>>>
>>
>> If you only have the constant null space, no. Most AMG packages do that
>> one by default.
>>
>> Matt
>>
>> Hi,
>>
>> I tried to use
>>
>> 1. -ksp_view -ksp_monitor_true_residual
>> 2. -poisson_ksp_view -poisson_ksp_monitor_true_residual
>>
>> However, this did not give any output. May I know why?
>>
>
> You need to call
>
> KSPSetFromOptions()
>
> Hi Matt,
>
> But I thought I already called:
>
> call KSPSetOperators(ksp,A_mat,A_mat,ierr)
>
> call KSPGetPC(ksp,pc,ierr)
>
> call KSPSetOptionsPrefix(ksp,"poisson_",ierr)
>
> call KSPSetType(ksp,ksptype,ierr)
>
> ksptype=KSPRICHARDSON
>
> call KSPSetType(ksp,ksptype,ierr)
>
> call PCSetType(pc,'hypre',ierr)
>
> call KSPSetFromOptions(ksp,ierr)
>
> ...
>
> Is my sequence of calling wrong?
>
If its not viewing, something is wrong. Here is the easiest way to make
things work.
Take an example, and make it do what you want, like view. Then change it
until it
is your own thing.
The code snippet is right, so there must be something around it that is
messing things up.
Thanks,
Matt
> Thanks.
>
>
> after you create it. When you launch valgrind you need
>
> valgrind --trace-children=yes <exec>
>
> Matt
>
>
>> I also tried valgrind:
>>
>> ==32157== Memcheck, a memory error detector
>> ==32157== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
>> ==32157== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright
>> info
>> ==32157== Command: /app/intel/xe2018/compilers_an
>> d_libraries_2018.0.128/linux/mpi/intel64/bin/mpirun ./a.out
>> -poisson_ksp_type richardson -poisson_pc_type hypre -poisson_pc_type_hypre
>> boomeramg
>> ==32157==
>> ==32159==
>> ==32159== HEAP SUMMARY:
>> ==32159== in use at exit: 49,981 bytes in 1,137 blocks
>> ==32159== total heap usage: 1,542 allocs, 405 frees, 72,030 bytes
>> allocated
>> ==32159==
>> ==32159== LEAK SUMMARY:
>> ==32159== definitely lost: 0 bytes in 0 blocks
>> ==32159== indirectly lost: 0 bytes in 0 blocks
>> ==32159== possibly lost: 0 bytes in 0 blocks
>> ==32159== still reachable: 49,981 bytes in 1,137 blocks
>> ==32159== suppressed: 0 bytes in 0 blocks
>> ==32159== Reachable blocks (those to which a pointer was found) are not
>> shown.
>> ==32159== To see them, rerun with: --leak-check=full --show-reachable=yes
>> ==32159==
>> ==32159== For counts of detected and suppressed errors, rerun with: -v
>> ==32159== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 8 from 6)
>> [362]PETSC ERROR: ------------------------------
>> ------------------------------------------
>> [362]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
>> probably memory access out of range
>> [362]PETSC ERROR: Try option -start_in_debugger or
>> -on_error_attach_debugger
>> [362]PETSC ERROR: or see http://www.mcs.anl.gov/petsc/d
>> ocumentation/faq.html#valgrind
>> [362]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac
>> OS X to find memory corruption errors
>> [362]PETSC ERROR: configure using --with-debugging=yes, recompile, link,
>> and run
>> [362]PETSC ERROR: to get more information on the crash.
>> [362]PETSC ERROR: --------------------- Error Message
>> --------------------------------------------------------------
>> [362]PETSC ERROR: Signal received
>> [362]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
>> for trouble shooting.
>> [362]PETSC ERROR: Petsc Release Version 3.8.3, Dec, 09, 2017
>> [362]PETSC ERROR: ./a.out on a petsc-3.8.3_intel_rel named std1055 by
>> tsltaywb Thu Mar 22 17:43:54 2018
>> [362]PETSC ERROR: Configure options --with-mpi-dir=/app/intel/xe20
>> 18/compilers_and_libraries_2018.0.128/linux/mpi/intel64
>> --with-blaslapack-dir=/app/intel/xe2018/compilers_and_librar
>> ies_2018.0.128/linux/mkl/lib/intel64 --download-hypre=/home/users/n
>> us/tsltaywb/source/git.hypre.tar.gz --with-debugging=0
>> --prefix=/home/users/nus/tsltaywb/lib/petsc-3.8.3_intel_rel
>> --with-shared-libraries=0 --known-mpi-shared-libraries=0
>> --with-fortran-interfaces=1 --CFLAGS="-xHost -g -O2" --CXXFLAGS="-xHost -g
>> -O2" --FFLAGS="-xHost -g -O2"
>> [362]PETSC ERROR: #1 User provided function() line 0 in unknown file
>> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 362
>> ==32409==
>> ==32409== HEAP SUMMARY:
>> ==32409== in use at exit: 52,168 bytes in 1,223 blocks
>> ==32409== total heap usage: 3,055 allocs, 1,832 frees, 122,188 bytes
>> allocated
>> ==32409==
>> ==32409== 10 bytes in 1 blocks are definitely lost in loss record 89 of
>> 319
>> ==32409== at 0x4C28A2E: malloc (vg_replace_malloc.c:270)
>> ==32409== by 0x466742: xmalloc (in /bin/bash)
>> ==32409== by 0x42EC78: ??? (in /bin/bash)
>> ==32409== by 0x430462: execute_command_internal (in /bin/bash)
>> ==32409== by 0x43320D: ??? (in /bin/bash)
>> ==32409== by 0x433629: ??? (in /bin/bash)
>> ==32409== by 0x4303BC: execute_command_internal (in /bin/bash)
>> ==32409== by 0x43110D: execute_command (in /bin/bash)
>> ==32409== by 0x41D6D5: reader_loop (in /bin/bash)
>> ==32409== by 0x41CEBB: main (in /bin/bash)
>> ==32409==
>> ==32409== LEAK SUMMARY:
>> ==32409== definitely lost: 10 bytes in 1 blocks
>> ==32409== indirectly lost: 0 bytes in 0 blocks
>> ==32409== possibly lost: 0 bytes in 0 blocks
>> ==32409== still reachable: 52,158 bytes in 1,222 blocks
>> ==32409== suppressed: 0 bytes in 0 blocks
>> ==32409== Reachable blocks (those to which a pointer was found) are not
>> shown.
>> ==32409== To see them, rerun with: --leak-check=full --show-reachable=yes
>> ==32409== Reachable blocks (those to which a pointer was found) are not
>> shown.
>> ==32409== To see them, rerun with: --leak-check=full --show-reachable=yes
>> ==32409==
>> ==32409== For counts of detected and suppressed errors, rerun with: -v
>> ==32409== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 8 from 6)
>> ==32157==
>> ==32157== HEAP SUMMARY:
>> ==32157== in use at exit: 55,861 bytes in 1,202 blocks
>> ==32157== total heap usage: 3,249 allocs, 2,047 frees, 125,857 bytes
>> allocated
>> ==32157==
>> ==32157== LEAK SUMMARY:
>> ==32157== definitely lost: 0 bytes in 0 blocks
>> ==32157== indirectly lost: 0 bytes in 0 blocks
>> ==32157== possibly lost: 0 bytes in 0 blocks
>> ==32157== still reachable: 55,861 bytes in 1,202 blocks
>> ==32157== suppressed: 0 bytes in 0 blocks
>> ==32157== Reachable blocks (those to which a pointer was found) are not
>> shown.
>> ==32157== To see them, rerun with: --leak-check=full --show-reachable=yes
>> ==32157==
>> ==32157== For counts of detected and suppressed errors, rerun with: -v
>> ==32157== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 8 from 6)
>>
>> Is this the output I'm supposed to get?
>>
>> Thanks!
>>
>>
>>
>>
>>> Thanks
>>>
>>> 3) You can also try ML, which is the same type of MG as GAMG.
>>>>> (--download-ml).
>>>>>
>>>>> I have recompiled PETSc with ML. Is there an example command line
>>>> options which I can use for ML?
>>>>
>>>
>>> -pc_type ml
>>>
>>>
>>>> Another question is generally speaking, is geometric multigrid (GMG)
>>>> faster than algebraic?
>>>>
>>>
>>> No, only for the setup time.
>>>
>>>
>>>> I tested on a small problem and the time taken varies from 1.15min
>>>> (HYPRE, geometric) to 3.25 (GAMG). BoomerAMG is 1.45min.
>>>>
>>>
>>> I am not sure what you are running when you say Hypre geometric.
>>>
>>>
>>>> Besides HYPRE, is there any other GMG I can use?
>>>>
>>>
>>> As I said above, it does not converge any faster and the solve is not
>>> faster, its all setup time, so small problems will look faster.
>>> You should be doing 10-20 iterates. If you are doing more, MG is not
>>> working.
>>>
>>> If you truly have a structured grid, then use DMDA in PETSc and you can
>>> use GMG with
>>>
>>> -pc_type mg -pc_mg_nlevels <n>
>>>
>>> Matt
>>>
>>>> My cluster can't connect to the internet. Where can I 1st download it?
>>>>>
>>>>> Similarly, how can I find out the location of the ext software by
>>>>> myself?
>>>>>
>>>>
>>>> The locations are all in the configure Python modules:
>>>>
>>>> https://bitbucket.org/petsc/petsc/src/4b6141395f14f0c7d141
>>>> 5a2ff0158eec75a27d63/config/BuildSystem/config/packages/ml.p
>>>> y?at=master&fileviewer=file-view-default
>>>>
>>>> Thanks,
>>>>
>>>> Matt
>>>>
>>>>
>>>>> Thanks,
>>>>>
>>>>> Matt
>>>>>
>>>>>
>>>>>>
>>>>>> --
>>>>>> Thank you very much.
>>>>>>
>>>>>> Yours sincerely,
>>>>>>
>>>>>> ================================================
>>>>>> TAY Wee-Beng (Zheng Weiming) 郑伟明
>>>>>> Personal research webpage: http://tayweebeng.wixsite.com/website
>>>>>> Youtube research showcase: https://www.youtube.com/channe
>>>>>> l/UC72ZHtvQNMpNs2uRTSToiLA
>>>>>> linkedin: www.linkedin.com/in/tay-weebeng
>>>>>> ================================================
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> What most experimenters take for granted before they begin their
>>>>> experiments is infinitely more interesting than any results to which their
>>>>> experiments lead.
>>>>> -- Norbert Wiener
>>>>>
>>>>> https://www.cse.buffalo.edu/~knepley/
>>>>> <http://www.caam.rice.edu/%7Emk51/>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> What most experimenters take for granted before they begin their
>>>> experiments is infinitely more interesting than any results to which their
>>>> experiments lead.
>>>> -- Norbert Wiener
>>>>
>>>> https://www.cse.buffalo.edu/~knepley/
>>>> <http://www.caam.rice.edu/%7Emk51/>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> -- Norbert Wiener
>>>
>>> https://www.cse.buffalo.edu/~knepley/
>>> <http://www.caam.rice.edu/%7Emk51/>
>>>
>>>
>>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/%7Emk51/>
>>
>>
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/%7Emk51/>
>
>
>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/~mk51/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20180326/c488a6be/attachment-0001.html>
More information about the petsc-users
mailing list