[petsc-dev] CUDA guys, stuff still broken in master/next

Barry Smith bsmith at mcs.anl.gov
Sat Feb 21 11:34:32 CST 2015 

> On Feb 21, 2015, at 11:23 AM, Karl Rupp <rupp at iue.tuwien.ac.at> wrote:
> 
> Hi Dominic,
> 
> yes, it is due to the changes for block-Jacobi, but I haven't found the exact cause. It is definitely related to the following code lines in bjacobi.c, around lines 820:
> 
>  /*
>    The reason we need to generate these vectors is to serve
>    as the right-hand side and solution vector for the solve on the
>    block. We do not need to allocate space for the vectors since
>    that is provided via VecPlaceArray() just before the call to
>    KSPSolve() on the block.
>  */
> MatGetSize(pmat,&m,&m);
> VecCreateSeqWithArray(PETSC_COMM_SELF,1,m,NULL,&bjac->x);
> VecCreateSeqWithArray(PETSC_COMM_SELF,1,m,NULL,&bjac->y);
> #ifdef PETSC_HAVE_CUSP
>  VecSetType(bjac->x,VECCUSP);
>  VecSetType(bjac->y,VECCUSP);
> #endif
> 
> I don't think that it is a good idea to unconditionally set the vector type to VECCUSP here (and at a few other locations in bjacobi.c). Particularly as this causes a test to leak memory which does not even use CUSP vectors. We may not have the 'correct' vector information available at this point, so we can only guess which type is correct.

  Would a call to MatGetVecs() here tell you the type of the vectors you need? Yeah you create and then immediately destroy a vector but that is ok.

> 
> One way to circumvent this is to set the correct (GPU) vector type only when applying the preconditioner, where the vector type is known. This would, however, require some tweaks to VecSetType in order to avoid repeated memory allocations and the like. Any other ideas?
> 
> Best regards,
> Karli
> 
> 
> On 02/19/2015 11:13 PM, Dominic Meiser wrote:
>> On 02/19/2015 02:31 PM, Karl Rupp wrote:
>>> Hi,
>>> 
>>> > Have you had a chance to look into this?
>>> 
>>> sorry, no, I was busy with a project proposal. I'll look into it
>>> tomorrow.
>> I've been really swamped, too. I'm afraid these failures might be
>> related to the changes we made for asm and bjacobi with GPU matrices.
>> Feel free to ping me via google chat or skype if you'd like me to have a
>> look at something or to go over some code together.
>> 
>> Cheers,
>> Dominic
>> 
>>> 
>>> Best regards,
>>> Karli
>>> 
>>> 
>>> >
>>>> On 02/19/2015 12:46 PM, Barry Smith wrote:
>>>>> http://ftp.mcs.anl.gov/pub/petsc/nightlylogs/archive/2015/02/18/examples_master_arch-cuda-double_bb-proxy.log
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> =========================================
>>>>> 1,2c1,150
>>>>> < Linear stokes_ solve converged due to CONVERGED_RTOL iterations 16
>>>>> < 9.0909e-02   1.9273e-03   2.9842e-04   4.8732e-03
>>>>> ---
>>>>>> Linear stokes_ solve converged due to CONVERGED_ATOL iterations 0
>>>>>> 9.0909e-02   7.9577e-02   1.7911e-02   2.9249e-01
>>>>>> [0]Total space allocated 64 bytes
>>>>>> [ 0]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [0]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [0]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [0]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [0]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [0]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [0]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [0]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [0]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [0]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [0]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [0]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [0]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [0]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [0]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [0]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [0]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [0]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>>> [ 0]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [0]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [0]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [0]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [0]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [0]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [0]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [0]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [0]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [0]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [0]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [0]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [0]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [0]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [0]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [0]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [0]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [0]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>>> [1]Total space allocated 64 bytes
>>>>>> [ 1]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [1]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [1]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [1]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [1]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [1]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [1]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [1]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [1]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [1]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [1]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [1]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [1]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [1]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [1]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [1]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [1]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [1]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>>> [ 1]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [1]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [1]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [1]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [1]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [1]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [1]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [1]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [1]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [1]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [1]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [1]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [1]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [1]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [1]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [1]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [1]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [1]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>>> [2]Total space allocated 64 bytes
>>>>>> [ 2]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [2]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [2]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [2]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [2]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [2]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [2]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [2]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [2]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [2]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [2]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [2]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [2]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [2]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [2]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [2]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [2]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [2]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>>> [ 2]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [2]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [2]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [2]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [2]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [2]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [2]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [2]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [2]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [2]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [2]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [2]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [2]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [2]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [2]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [2]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [2]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [2]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>>> [3]Total space allocated 64 bytes
>>>>>> [ 3]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [3]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [3]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [3]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [3]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [3]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [3]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [3]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [3]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [3]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [3]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [3]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [3]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [3]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [3]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [3]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [3]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [3]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>>> [ 3]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [3]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [3]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [3]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [3]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [3]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [3]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [3]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [3]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [3]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [3]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [3]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [3]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [3]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [3]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [3]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [3]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [3]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials
>>>>> Possible problem with ex43_bjacobi, diffs above
>>>>> =========================================
>>>>> 1,2c1,150
>>>>> < Linear stokes_ solve converged due to CONVERGED_RTOL iterations 16
>>>>> < 9.0909e-02   1.9273e-03   2.9842e-04   4.8732e-03
>>>>> ---
>>>>>> Linear stokes_ solve converged due to CONVERGED_ATOL iterations 0
>>>>>> 9.0909e-02   7.9577e-02   1.7911e-02   2.9249e-01
>>>>>> [0]Total space allocated 64 bytes
>>>>>> [ 0]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [0]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [0]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [0]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [0]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [0]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [0]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [0]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [0]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [0]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [0]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [0]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [0]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [0]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [0]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [0]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [0]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [0]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>>> [ 0]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [0]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [0]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [0]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [0]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [0]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [0]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [0]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [0]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [0]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [0]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [0]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [0]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [0]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [0]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [0]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [0]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [0]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>>> [1]Total space allocated 64 bytes
>>>>>> [ 1]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [1]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [1]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [1]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [1]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [1]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [1]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [1]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [1]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [1]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [1]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [1]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [1]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [1]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [1]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [1]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [1]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [1]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>>> [ 1]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [1]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [1]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [1]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [1]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [1]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [1]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [1]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [1]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [1]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [1]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [1]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [1]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [1]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [1]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [1]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [1]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [1]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>>> [2]Total space allocated 64 bytes
>>>>>> [ 2]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [2]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [2]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [2]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [2]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [2]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [2]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [2]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [2]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [2]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [2]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [2]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [2]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [2]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [2]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [2]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [2]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [2]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>>> [ 2]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [2]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [2]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [2]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [2]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [2]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [2]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [2]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [2]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [2]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [2]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [2]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [2]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [2]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [2]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [2]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [2]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [2]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>>> [3]Total space allocated 64 bytes
>>>>>> [ 3]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [3]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [3]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [3]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [3]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [3]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [3]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [3]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [3]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [3]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [3]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [3]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [3]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [3]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [3]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [3]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [3]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [3]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>>> [ 3]32 bytes VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [3]  VecCreate_Seq_Private() line 1250 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/bvec2.c
>>>>>>       [3]  VecCreate_SeqCUSP() line 1942 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/seq/seqcusp/veccusp.cu
>>>>>>       [3]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [3]  VecCreate_CUSP() line 242 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/impls/mpi/mpicusp/mpicusp.cu
>>>>>>       [3]  VecSetType() line 38 in
>>>>>> /home/balay/petsc.clone/src/vec/vec/interface/vecreg.c
>>>>>>       [3]  PCSetUp_BJacobi_Singleblock() line 783 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [3]  PCSetUp_BJacobi() line 24 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [3]  PCSetUp() line 882 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [3]  KSPSetUp() line 245 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [3]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [3]  PCApply_BJacobi_Multiproc() line 1178 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/impls/bjacobi/bjacobi.c
>>>>>>       [3]  PCApply() line 430 in
>>>>>> /home/balay/petsc.clone/src/ksp/pc/interface/precon.c
>>>>>>       [3]  KSP_PCApply() line 233 in
>>>>>> /home/balay/petsc.clone/include/petsc-private/kspimpl.h
>>>>>>       [3]  KSPInitialResidual() line 44 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itres.c
>>>>>>       [3]  KSPSolve_GMRES() line 224 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/impls/gmres/gmres.c
>>>>>>       [3]  KSPSolve() line 492 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/interface/itfunc.c
>>>>>>       [3]  solve_stokes_2d_coupled() line 1143 in
>>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials/ex43.c
>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials
>>>>> Possible problem with ex43_bjacobi_baij, diffs above
>>>>> =========================================
>>>>> 46c46
>>>>> <  44 KSP Residual norm 5.15089e-05
>>>>> ---
>>>>>>  44 KSP Residual norm 5.1509e-05
>>>>> /home/balay/petsc.clone/src/ksp/ksp/examples/tutorials
>>>>> Possible problem with ex49_1, diffs above
>>>>> =========================================
>>>>> 2,14c2,16
>>>>> <   0 KSP Residual norm 4.5784
>>>>> <   1 KSP Residual norm 0.264677
>>>>> <   2 KSP Residual norm 0.067428
>>>>> <   3 KSP Residual norm 0.0188871
>>>>> <   4 KSP Residual norm 0.00960792
>>>>> <   5 KSP Residual norm 0.00522938
>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>

More information about the petsc-dev mailing list