[petsc-users] h-FGMRES
Gaetan Kenway
gaetank at gmail.com
Mon May 20 11:12:35 CDT 2013
Thanks for having a look at it.
I look forward to giving it a try when it implemented. And like bjacobi,
support for mpibaij would be very beneficial to me.
Thank you
Gaetan
On Mon, May 20, 2013 at 12:08 PM, Hong Zhang <hzhang at mcs.anl.gov> wrote:
> Gaetan:
>
>>
>> On a related note, I tried using the ASM version of the same approach;
>> that is -pc_type asm -pc_asm_blocks 4 with the remainder of the options
>> the same. This gives a message that the number of blocks is less than the
>> number of processors (sorry I don't have the exact message anymore). I get
>> this error with both mpiaij and mpibaij types.
>>
>
> Thanks for reporting it.
> The '_Multiproc' version of asm is not implemented yet, although it is
> logically same as bjacobi_Multiproc.
> I added to our 'to-do' list
> https://bitbucket.org/petsc/petsc/issue/42/implement-pcsetup_asm_multiproc
>
>>
>> Has this approach been implemented/do you think there would be any
>> benefit from the approach?
>>
> It would be beneficial for applications that give better convergence and
> performance with asm than bjacobi.
> It is necessary to provide such support in petsc library.
> We'll let you know when it is implemented.
>
> Hong
>
>>
>> On Mon, May 20, 2013 at 10:34 AM, Hong Zhang <hzhang at mcs.anl.gov> wrote:
>>
>>> Gaetan :
>>>
>>>>
>>>> It runs if the mattype is mpiaij instead of mpibaij. I gather this is
>>>> not implemented for the blocked matrix types?
>>>
>>> It is not tested for mpibaij format yet. I'll check it.
>>> The paper uses mpiaij format.
>>>
>>> Hong
>>>
>>>>
>>>> Gaetan
>>>>
>>>> On Mon, May 20, 2013 at 9:26 AM, Gaetan Kenway <gaetank at gmail.com>wrote:
>>>>
>>>>> Hi again
>>>>>
>>>>> I installed petsc3.4.0 and I am still getting the following error when
>>>>> running with the following options (on 64 procs)
>>>>>
>>>>> # Matrix Options
>>>>> -matload_block_size 5 -mat_type mpibaij
>>>>>
>>>>> # KSP solver options
>>>>> -ksp_type fgmres -ksp_max_it 1000 -ksp_gmres_restart 200 -ksp_monitor
>>>>> -ksp_view -ksp_pc_side right -ksp_rtol 1e-6
>>>>>
>>>>> # Nested GMRES Options
>>>>> -pc_type bjacobi -pc_bjacobi_blocks 4 -sub_ksp_type gmres
>>>>> -sub_ksp_max_it 5 -sub_pc_type bjacobi -sub_sub_pc_type ilu
>>>>> -sub_sub_pc_factor_mat_ordering_type rcm -sub_sub_pc_factor_levels 1
>>>>>
>>>>> Any thoughts?
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Gaetan
>>>>>
>>>>> [44]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>> [44]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
>>>>> probably memory access out of range
>>>>> [44]PETSC ERROR: Try option -start_in_debugger or
>>>>> -on_error_attach_debugger
>>>>> [44]PETSC ERROR: or see
>>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[44]PETSCERROR: or try
>>>>> http://valgrind.org on GNU/linux and Apple Mac OS X to find memory
>>>>> corruption errors
>>>>> [44]PETSC ERROR: likely location of problem given in stack below
>>>>> [44]PETSC ERROR: --------------------- Stack Frames
>>>>> ------------------------------------
>>>>> [44]PETSC ERROR: Note: The EXACT line numbers in the stack are not
>>>>> available,
>>>>> [44]PETSC ERROR: INSTEAD the line number of the start of the
>>>>> function
>>>>> [44]PETSC ERROR: is given.
>>>>> [44]PETSC ERROR: [44] PCSetUp_BJacobi_Multiproc line 1197
>>>>> /home/j/jmartins/kenway/packages/petsc-3.4.0/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>> [44]PETSC ERROR: [44] PCSetUp_BJacobi line 24
>>>>> /home/j/jmartins/kenway/packages/petsc-3.4.0/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>> [44]PETSC ERROR: [44] PCSetUp line 868
>>>>> /home/j/jmartins/kenway/packages/petsc-3.4.0/src/ksp/pc/interface/precon.c
>>>>> [44]PETSC ERROR: [44] KSPSetUp line 192
>>>>> /home/j/jmartins/kenway/packages/petsc-3.4.0/src/ksp/ksp/interface/itfunc.c
>>>>> [44]PETSC ERROR: [44] KSPSolve line 356
>>>>> /home/j/jmartins/kenway/packages/petsc-3.4.0/src/ksp/ksp/interface/itfunc.c
>>>>> [43]PETSC ERROR:
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>>
>>>>> On Sun, May 19, 2013 at 11:15 PM, Barry Smith <bsmith at mcs.anl.gov>wrote:
>>>>>
>>>>>>
>>>>>> You should be using PETSc version 3.4 which was recently released
>>>>>> and is what the paper is based on.
>>>>>>
>>>>>> Barry
>>>>>>
>>>>>> On May 19, 2013, at 10:11 PM, Gaetan Kenway <gaetank at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> > Hi Everyone
>>>>>> >
>>>>>> > I am trying to replicate the type of preconditioner described in
>>>>>> "Hierarchical and Nested Krylov Methods for Extreme-Scale Computing".
>>>>>> >
>>>>>> > I have used the following options: (I'm using fortran so the
>>>>>> following is my petsc_options file)
>>>>>> >
>>>>>> > # Matrix Options
>>>>>> > -matload_block_size 5
>>>>>> > -mat_type mpibaij
>>>>>> >
>>>>>> > # KSP solver options
>>>>>> > -ksp_type gmres
>>>>>> > -ksp_max_it 1000
>>>>>> > -ksp_gmres_restart 200
>>>>>> > -ksp_monitor
>>>>>> > -ksp_view
>>>>>> > -ksp_pc_side right
>>>>>> > -ksp_rtol 1e-6
>>>>>> >
>>>>>> > # Nested GMRES Options
>>>>>> > -pc_type bjacobi
>>>>>> > -pc_bjacobi_blocks 4
>>>>>> > -sub_ksp_type gmres
>>>>>> > -sub_ksp_max_it 5
>>>>>> > -sub_pc_type bjacobi
>>>>>> > -sub_sub_pc_type ilu
>>>>>> > -sub_sub_pc_factor_mat_ordering_type rcm
>>>>>> > -sub_sub_pc_factor_levels 1
>>>>>> >
>>>>>> > The test is run on 64 processors and the total number of block
>>>>>> jacobi blocks is 4 (less than nproc). The error I get is:
>>>>>> >
>>>>>> > [6]PETSC ERROR: Note: The EXACT line numbers in the stack are not
>>>>>> available,
>>>>>> > [6]PETSC ERROR: INSTEAD the line number of the start of the
>>>>>> function
>>>>>> > [6]PETSC ERROR: is given.
>>>>>> > [6]PETSC ERROR: [6] PCSetUp_BJacobi_Multiproc line 1269
>>>>>> /home/j/jmartins/kenway/packages/petsc-3.3-p5/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>> > [6]PETSC ERROR: [6] PCSetUp_BJacobi line 24
>>>>>> /home/j/jmartins/kenway/packages/petsc-3.3-p5/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>> > [6]PETSC ERROR: [6] PCSetUp line 810
>>>>>> /home/j/jmartins/kenway/packages/petsc-3.3-p5/src/ksp/pc/interface/precon.c
>>>>>> > [6]PETSC ERROR: [6] KSPSetUp line 182
>>>>>> /home/j/jmartins/kenway/packages/petsc-3.3-p5/src/ksp/ksp/interface/itfunc.c
>>>>>> > [6]PETSC ERROR: [6] KSPSolve line 351
>>>>>> /home/j/jmartins/kenway/packages/petsc-3.3-p5/src/ksp/ksp/interface/itfunc.c
>>>>>> > [6]PETSC ERROR: --------------------- Error Message
>>>>>> ------------------------------------
>>>>>> > [6]PETSC ERROR: Signal received!
>>>>>> > [6]PETSC ERROR:
>>>>>> ------------------------------------------------------------------------
>>>>>> > [6]PETSC ERROR: Petsc Release Version 3.3.0, Patch 5, Sat Dec 1
>>>>>> 15:10:41 CST 2012
>>>>>> > [6]PETSC ERROR: See docs/changes/index.html for recent updates.
>>>>>> > [6]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>>>>> > [6]PETSC ERROR:
>>>>>> ------------------------------------------------------------------------
>>>>>> > [6]PETSC ERROR:
>>>>>> ------------------------------------------------------------------------
>>>>>> > [6]PETSC ERROR: ./main on a intel-rea named gpc-f109n001 by kenway
>>>>>> Sun May 19 23:01:52 2013
>>>>>> > [6]PETSC ERROR: Libraries linked from
>>>>>> /home/j/jmartins/kenway/packages/petsc-3.3-p5/intel-real-debug/lib
>>>>>> > [6]PETSC ERROR: Configure run at Sun Jan 20 15:52:20 2013
>>>>>> > [6]PETSC ERROR: Configure options --with-shared-libraries
>>>>>> --download-superlu_dist=yes --download-parmetis=yes --download-metis=yes
>>>>>> --with-fortran-interfaces=1 --with-debugging=yes --with-scalar-type=real
>>>>>> -with-petsc-arch=intel-real-debug --with-blas-lapack-dir= --with-pic
>>>>>> > [6]PETSC ERROR:
>>>>>> ------------------------------------------------------------------------
>>>>>> >
>>>>>> > If the number of blocks is greater than or equal to the number of
>>>>>> processors it runs fine. I'm using version 3.3-p5.
>>>>>> >
>>>>>> > The options as listed in the paper are:
>>>>>> > -flow_ksp_type fgmres -flow_ksp_pc_side right -flow_pc_type bjacobi
>>>>>> -flow_pc_bjacobi_blocks ngp
>>>>>> > -flow_sub_ksp_type gmres -flow_sub_ksp_max_it 6 -flow_sub_pc_type
>>>>>> bjacobi
>>>>>> > -flow_sub_sub_pc_type ilu
>>>>>> >
>>>>>> > Any suggestions would be greatly appreciated.
>>>>>> >
>>>>>> > Thank you,
>>>>>> >
>>>>>> > Gaetan Kenway
>>>>>> >
>>>>>> >
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20130520/c79e44af/attachment-0001.html>
More information about the petsc-users
mailing list