[petsc-users] h-FGMRES

Mon May 20 09:34:29 CDT 2013

Gaetan :

>
> It runs if the mattype is mpiaij instead of mpibaij. I gather this is not
> implemented for the blocked matrix types?

It is not tested for mpibaij format yet. I'll check it.
The paper uses mpiaij format.

Hong

>
> Gaetan
>
> On Mon, May 20, 2013 at 9:26 AM, Gaetan Kenway <gaetank at gmail.com> wrote:
>
>> Hi again
>>
>> I installed petsc3.4.0 and I am still getting the following error when
>> running with the following options (on 64 procs)
>>
>> # Matrix Options
>> -matload_block_size 5 -mat_type mpibaij
>>
>> # KSP solver options
>> -ksp_type fgmres -ksp_max_it 1000 -ksp_gmres_restart 200 -ksp_monitor
>> -ksp_view -ksp_pc_side right -ksp_rtol 1e-6
>>
>> # Nested GMRES Options
>>  -pc_type bjacobi -pc_bjacobi_blocks 4 -sub_ksp_type gmres
>> -sub_ksp_max_it 5 -sub_pc_type bjacobi -sub_sub_pc_type ilu
>> -sub_sub_pc_factor_mat_ordering_type rcm -sub_sub_pc_factor_levels 1
>>
>> Any thoughts?
>>
>> Thank you,
>>
>> Gaetan
>>
>> [44]PETSC ERROR:
>> ------------------------------------------------------------------------
>> [44]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
>> probably memory access out of range
>> [44]PETSC ERROR: Try option -start_in_debugger or
>> -on_error_attach_debugger
>> [44]PETSC ERROR: or see
>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[44]PETSCERROR: or try
>> http://valgrind.org on GNU/linux and Apple Mac OS X to find memory
>> corruption errors
>> [44]PETSC ERROR: likely location of problem given in stack below
>> [44]PETSC ERROR: ---------------------  Stack Frames
>> ------------------------------------
>> [44]PETSC ERROR: Note: The EXACT line numbers in the stack are not
>> available,
>> [44]PETSC ERROR:       INSTEAD the line number of the start of the
>> function
>> [44]PETSC ERROR:       is given.
>> [44]PETSC ERROR: [44] PCSetUp_BJacobi_Multiproc line 1197
>> /home/j/jmartins/kenway/packages/petsc-3.4.0/src/ksp/pc/impls/bjacobi/bjacobi.c
>> [44]PETSC ERROR: [44] PCSetUp_BJacobi line 24
>> /home/j/jmartins/kenway/packages/petsc-3.4.0/src/ksp/pc/impls/bjacobi/bjacobi.c
>> [44]PETSC ERROR: [44] PCSetUp line 868
>> /home/j/jmartins/kenway/packages/petsc-3.4.0/src/ksp/pc/interface/precon.c
>> [44]PETSC ERROR: [44] KSPSetUp line 192
>> /home/j/jmartins/kenway/packages/petsc-3.4.0/src/ksp/ksp/interface/itfunc.c
>> [44]PETSC ERROR: [44] KSPSolve line 356
>> /home/j/jmartins/kenway/packages/petsc-3.4.0/src/ksp/ksp/interface/itfunc.c
>> [43]PETSC ERROR:
>> ------------------------------------------------------------------------
>>
>>
>> On Sun, May 19, 2013 at 11:15 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>
>>>
>>>    You should be using PETSc version 3.4 which was recently released and
>>> is what the paper is based on.
>>>
>>>     Barry
>>>
>>> On May 19, 2013, at 10:11 PM, Gaetan Kenway <gaetank at gmail.com> wrote:
>>>
>>> > Hi Everyone
>>> >
>>> > I am trying to replicate the type of preconditioner described in
>>> "Hierarchical and Nested Krylov Methods for Extreme-Scale Computing".
>>> >
>>> > I have used the following options: (I'm using fortran so the following
>>> is my petsc_options file)
>>> >
>>> > # Matrix Options
>>> > -matload_block_size 5
>>> > -mat_type mpibaij
>>> >
>>> > # KSP solver options
>>> > -ksp_type gmres
>>> > -ksp_max_it 1000
>>> > -ksp_gmres_restart 200
>>> > -ksp_monitor
>>> > -ksp_view
>>> > -ksp_pc_side right
>>> > -ksp_rtol 1e-6
>>> >
>>> > # Nested GMRES Options
>>> > -pc_type bjacobi
>>> > -pc_bjacobi_blocks 4
>>> > -sub_ksp_type gmres
>>> > -sub_ksp_max_it 5
>>> > -sub_pc_type bjacobi
>>> > -sub_sub_pc_type ilu
>>> > -sub_sub_pc_factor_mat_ordering_type rcm
>>> > -sub_sub_pc_factor_levels 1
>>> >
>>> > The test is run on 64 processors and the total  number of block jacobi
>>> blocks is 4 (less than nproc). The error I get is:
>>> >
>>> > [6]PETSC ERROR: Note: The EXACT line numbers in the stack are not
>>> available,
>>> > [6]PETSC ERROR:       INSTEAD the line number of the start of the
>>> function
>>> > [6]PETSC ERROR:       is given.
>>> > [6]PETSC ERROR: [6] PCSetUp_BJacobi_Multiproc line 1269
>>> /home/j/jmartins/kenway/packages/petsc-3.3-p5/src/ksp/pc/impls/bjacobi/bjacobi.c
>>> > [6]PETSC ERROR: [6] PCSetUp_BJacobi line 24
>>> /home/j/jmartins/kenway/packages/petsc-3.3-p5/src/ksp/pc/impls/bjacobi/bjacobi.c
>>> > [6]PETSC ERROR: [6] PCSetUp line 810
>>> /home/j/jmartins/kenway/packages/petsc-3.3-p5/src/ksp/pc/interface/precon.c
>>> > [6]PETSC ERROR: [6] KSPSetUp line 182
>>> /home/j/jmartins/kenway/packages/petsc-3.3-p5/src/ksp/ksp/interface/itfunc.c
>>> > [6]PETSC ERROR: [6] KSPSolve line 351
>>> /home/j/jmartins/kenway/packages/petsc-3.3-p5/src/ksp/ksp/interface/itfunc.c
>>> > [6]PETSC ERROR: --------------------- Error Message
>>> ------------------------------------
>>> > [6]PETSC ERROR: Signal received!
>>> > [6]PETSC ERROR:
>>> ------------------------------------------------------------------------
>>> > [6]PETSC ERROR: Petsc Release Version 3.3.0, Patch 5, Sat Dec  1
>>> 15:10:41 CST 2012
>>> > [6]PETSC ERROR: See docs/changes/index.html for recent updates.
>>> > [6]PETSC ERROR: See docs/faq.html for hints about trouble shooting.
>>> > [6]PETSC ERROR:
>>> ------------------------------------------------------------------------
>>> > [6]PETSC ERROR:
>>> ------------------------------------------------------------------------
>>> > [6]PETSC ERROR: ./main on a intel-rea named gpc-f109n001 by kenway Sun
>>> May 19 23:01:52 2013
>>> > [6]PETSC ERROR: Libraries linked from
>>> /home/j/jmartins/kenway/packages/petsc-3.3-p5/intel-real-debug/lib
>>> > [6]PETSC ERROR: Configure run at Sun Jan 20 15:52:20 2013
>>> > [6]PETSC ERROR: Configure options --with-shared-libraries
>>> --download-superlu_dist=yes --download-parmetis=yes --download-metis=yes
>>> --with-fortran-interfaces=1 --with-debugging=yes --with-scalar-type=real
>>> -with-petsc-arch=intel-real-debug --with-blas-lapack-dir= --with-pic
>>> > [6]PETSC ERROR:
>>> ------------------------------------------------------------------------
>>> >
>>> > If the number of blocks is greater than or equal to the number of
>>> processors it runs fine.  I'm using version 3.3-p5.
>>> >
>>> > The options as listed in the paper are:
>>> > -flow_ksp_type fgmres -flow_ksp_pc_side right -flow_pc_type bjacobi
>>> -flow_pc_bjacobi_blocks ngp
>>> > -flow_sub_ksp_type gmres -flow_sub_ksp_max_it 6 -flow_sub_pc_type
>>> bjacobi
>>> > -flow_sub_sub_pc_type ilu
>>> >
>>> > Any suggestions would be greatly appreciated.
>>> >
>>> > Thank you,
>>> >
>>> > Gaetan Kenway
>>> >
>>> >
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20130520/4a88a676/attachment-0001.html>