[petsc-users] pc_type asm

[(Rebecca) Xuefei YUAN]

Dear all,

If I add the option with -pc_type asm, I will get some memory errors  
from valgrind for np>1.

I think it is for setup of the pc, but I am not sure how to fix this problem.

Here is the valgrind error:
rebecca at YuanWork:~/linux/code/twoway/twoway_new/workingspace$ mpiexec  
-np 2 valgrind ./twqt2ff.exe -options_file option_all
==15113== Memcheck, a memory error detector
==15113== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==15113== Using Valgrind-3.5.0 and LibVEX; rerun with -h for copyright info
==15113== Command: ./twqt2ff.exe -options_file option_all
==15113==
==15114== Memcheck, a memory error detector
==15114== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==15114== Using Valgrind-3.5.0 and LibVEX; rerun with -h for copyright info
==15114== Command: ./twqt2ff.exe -options_file option_all
==15114==
**************************************************
number of processors = 2
viscosity = 5.0000000000000003e-02
resistivity = 5.0000000000000001e-03
skin depth = 0.0000000000000000e+00
hyper resistivity = 3.2768000000000001e-04
hyper viscosity = 2.6214399999999999e-01
problem size: 51 by 51
dx = 2.5098039215686274e-01
dy = 1.2800000000000000e-01
dt = 1.0000000000000001e-01
adaptive time step size (1:yes;0:no) = 0
**************************************************
   0 SNES Function norm 2.189505265331e-02
==15114== Invalid read of size 4
==15114==    at 0x88A09C4: MPIDI_CH3I_Progress_handle_sock_event  
(ch3_progress.c:738)
==15114==    by 0x88A0F17: MPIDI_CH3I_Progress (ch3_progress.c:212)
==15114==    by 0x88802B4: PMPI_Waitany (waitany.c:203)
==15114==    by 0x85A15A4: MatGetSubMatrices_MPIAIJ_Local (mpiov.c:1195)
==15114==    by 0x859BDC2: MatGetSubMatrices_MPIAIJ (mpiov.c:781)
==15114==    by 0x820BD51: MatGetSubMatrices (matrix.c:5691)
==15114==    by 0x8285317: PCSetUp_ASM (asm.c:299)
==15114==    by 0x8250FD3: PCSetUp (precon.c:796)
==15114==    by 0x82F0134: KSPSetUp (itfunc.c:272)
==15114==    by 0x82F125C: KSPSolve (itfunc.c:390)
==15114==    by 0x838A9DA: SNES_KSPSolve (snes.c:2961)
==15114==    by 0x87C2C0C: SNESSolve_LS (ls.c:191)
==15114==  Address 0x46411d0 is 368 bytes inside a [MPICH2 handle:  
objptr=0x4641060 handle=0xec00000c INDIRECT/REQUEST] of size 372  
client-defined
==15114==    at 0x889528A: MPIU_Handle_obj_alloc_unsafe (handlemem.c:217)
==15114==    by 0x88E1920: MPIDI_CH3U_Recvq_FDU_or_AEP (ch3u_recvq.c:342)
==15114==    by 0x88AAFA7: MPID_Irecv (mpid_irecv.c:46)
==15114==    by 0x886D080: MPIC_Sendrecv (helper_fns.c:153)
==15114==    by 0x88625E0: MPIR_Barrier (barrier.c:75)
==15114==    by 0x88628D7: MPIR_Barrier_or_coll_fn (barrier.c:244)
==15114==    by 0x8862998: PMPI_Barrier (barrier.c:421)
==15114==    by 0x81174EA: PetscCommDuplicate (tagm.c:190)
==15114==    by 0x811AE1E: PetscHeaderCreate_Private (inherit.c:43)
==15114==    by 0x8304017: KSPCreate (itcreate.c:476)
==15114==    by 0x8273121: PCMGSetLevels (mg.c:173)
==15114==    by 0x839A21E: DMMGSetUpLevel (damg.c:379)
==15114==
==15113== Invalid read of size 4
==15113==    at 0x88A09C4: MPIDI_CH3I_Progress_handle_sock_event  
(ch3_progress.c:738)
==15113==    by 0x88A0F17: MPIDI_CH3I_Progress (ch3_progress.c:212)
==15113==    by 0x88802B4: PMPI_Waitany (waitany.c:203)
==15113==    by 0x85A15A4: MatGetSubMatrices_MPIAIJ_Local (mpiov.c:1195)
==15113==    by 0x859BDC2: MatGetSubMatrices_MPIAIJ (mpiov.c:781)
==15113==    by 0x820BD51: MatGetSubMatrices (matrix.c:5691)
==15113==    by 0x8285317: PCSetUp_ASM (asm.c:299)
==15113==    by 0x8250FD3: PCSetUp (precon.c:796)
==15113==    by 0x82F0134: KSPSetUp (itfunc.c:272)
==15113==    by 0x82F125C: KSPSolve (itfunc.c:390)
==15113==    by 0x838A9DA: SNES_KSPSolve (snes.c:2961)
==15113==    by 0x87C2C0C: SNESSolve_LS (ls.c:191)
==15113==  Address 0x472d4d8 is 368 bytes inside a [MPICH2 handle:  
objptr=0x472d368 handle=0xec00000c INDIRECT/REQUEST] of size 372  
client-defined
==15113==    at 0x889528A: MPIU_Handle_obj_alloc_unsafe (handlemem.c:217)
==15113==    by 0x88E1920: MPIDI_CH3U_Recvq_FDU_or_AEP (ch3u_recvq.c:342)
==15113==    by 0x88AAFA7: MPID_Irecv (mpid_irecv.c:46)
==15113==    by 0x886D080: MPIC_Sendrecv (helper_fns.c:153)
==15113==    by 0x88625E0: MPIR_Barrier (barrier.c:75)
==15113==    by 0x88628D7: MPIR_Barrier_or_coll_fn (barrier.c:244)
==15113==    by 0x8862998: PMPI_Barrier (barrier.c:421)
==15113==    by 0x81174EA: PetscCommDuplicate (tagm.c:190)
==15113==    by 0x811AE1E: PetscHeaderCreate_Private (inherit.c:43)
==15113==    by 0x8304017: KSPCreate (itcreate.c:476)
==15113==    by 0x8273121: PCMGSetLevels (mg.c:173)
==15113==    by 0x839A21E: DMMGSetUpLevel (damg.c:379)
==15113==
Linear solve converged due to CONVERGED_RTOL iterations 1
   1 SNES Function norm 6.250178106647e-03
Linear solve converged due to CONVERGED_RTOL iterations 3
   2 SNES Function norm 8.139700042246e-04
Linear solve converged due to CONVERGED_RTOL iterations 5
   3 SNES Function norm 1.228451086501e-05
Linear solve converged due to CONVERGED_RTOL iterations 10
   4 SNES Function norm 1.375513293330e-08
Linear solve converged due to CONVERGED_RTOL iterations 16
   5 SNES Function norm 3.634919075810e-13
Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE
**************************************************
time step = 1
current time step size= 1.0000000000000001e-01
time = 1.0000000000000001e-01
number of nonlinear iterations = 5
number of linear iterations = 35
function norm = 3.6349190758097565e-13
**************************************************
total number of time steps = 1
total number of nonlinear iterations = 5
total number of linear iterations = 35






-- 
(Rebecca) Xuefei YUAN
Department of Applied Physics and Applied Mathematics
Columbia University
Tel:917-399-8032
www.columbia.edu/~xy2102