[petsc-users] unsorted local columns in 3.8?

[Randy Michael Churchill]

Doing some additional testing, the issue goes away when removing the gamg
preconditioner line from the petsc.rc:
-pc_type gamg


On Wed, Nov 1, 2017 at 8:23 PM, Hong <hzhang at mcs.anl.gov> wrote:

> Randy:
> Thanks, I'll check it tomorrow.
> Hong
>
> OK, this might not be completely satisfactory, because it doesn't show the
>> partitioning or how the matrix is created, but this reproduces the problem.
>> I wrote out my matrix, Amat, from the larger simulation, and load it in
>> this script. This must be run with MPI rank greater than 1. This may be
>> some combination of my petsc.rc, because when I use the PetscInitialize
>> with it, it throws the error, but when using default (PETSC_NULL_CHARACTER)
>> it runs fine.
>>
>>
>> On Tue, Oct 31, 2017 at 9:58 AM, Hong <hzhang at mcs.anl.gov> wrote:
>>
>>> Randy:
>>> It could be a bug or a missing feature in our new
>>> MatCreateSubMatrix_MPIAIJ_SameRowDist().
>>> It would be helpful if you can provide us a simple example that produces
>>> this example.
>>> Hong
>>>
>>> I'm running a Fortran code that was just changed over to using petsc 3.8
>>>> (previously petsc 3.7.6). An error was thrown during a KSPSetUp() call. The
>>>> error is "unsorted iscol_local is not implemented yet" (see full error
>>>> below). I tried to trace down the difference in the source files, but where
>>>> the error occurs (MatCreateSubMatrix_MPIAIJ_SameRowDist()) doesn't
>>>> seem to have existed in v3.7.6, so I'm unsure how to compare. It seems the
>>>> error is that the order of the columns locally are unsorted, though I don't
>>>> think I specify a column order in the creation of the matrix:
>>>>      call MatCreate(this%comm,AA,ierr)
>>>>      call MatSetSizes(AA,npetscloc,npetscloc,nreal,nreal,ierr)
>>>>      call MatSetType(AA,MATAIJ,ierr)
>>>>      call MatSetup(AA,ierr)
>>>>      call MatGetOwnershipRange(AA,low,high,ierr)
>>>>      allocate(d_nnz(npetscloc),o_nnz(npetscloc))
>>>>      call getNNZ(grid,npetscloc,low,high,d_nnz,o_nnz,this%xgc_petsc,nr
>>>> eal,ierr)
>>>>      call MatSeqAIJSetPreallocation(AA,PETSC_NULL_INTEGER,d_nnz,ierr)
>>>>      call MatMPIAIJSetPreallocation(AA,PETSC_NULL_INTEGER,d_nnz,PETSC_
>>>> NULL_INTEGER,o_nnz,ierr)
>>>>      deallocate(d_nnz,o_nnz)
>>>>      call MatSetOption(AA,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE,ierr)
>>>>      call MatSetOption(AA,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE,ierr)
>>>>      call MatSetup(AA,ierr)
>>>>
>>>>
>>>> [62]PETSC ERROR: --------------------- Error Message
>>>> --------------------------------------------------------------
>>>> [62]PETSC ERROR: No support for this operation for this object type
>>>> [62]PETSC ERROR: unsorted iscol_local is not implemented yet
>>>> [62]PETSC ERROR: See http://www.mcs.anl.gov/petsc/d
>>>> ocumentation/faq.html for trouble shooting.
>>>> [62]PETSC ERROR: Petsc Release Version 3.8.0, unknown[62]PETSC ERROR:
>>>> #1 MatCreateSubMatrix_MPIAIJ_SameRowDist() line 3418 in
>>>> /global/u1/r/rchurchi/petsc/3.8.0/src/mat/impls/aij/mpi/mpiaij.c
>>>> [62]PETSC ERROR: #2 MatCreateSubMatrix_MPIAIJ() line 3247 in
>>>> /global/u1/r/rchurchi/petsc/3.8.0/src/mat/impls/aij/mpi/mpiaij.c
>>>> [62]PETSC ERROR: #3 MatCreateSubMatrix() line 7872 in
>>>> /global/u1/r/rchurchi/petsc/3.8.0/src/mat/interface/matrix.c
>>>> [62]PETSC ERROR: #4 PCGAMGCreateLevel_GAMG() line 383 in
>>>> /global/u1/r/rchurchi/petsc/3.8.0/src/ksp/pc/impls/gamg/gamg.c
>>>> [62]PETSC ERROR: #5 PCSetUp_GAMG() line 561 in
>>>> /global/u1/r/rchurchi/petsc/3.8.0/src/ksp/pc/impls/gamg/gamg.c
>>>> [62]PETSC ERROR: #6 PCSetUp() line 924 in /global/u1/r/rchurchi/petsc/3.
>>>> 8.0/src/ksp/pc/interface/precon.c
>>>> [62]PETSC ERROR: #7 KSPSetUp() line 378 in
>>>> /global/u1/r/rchurchi/petsc/3.8.0/src/ksp/ksp/interface/itfunc.c
>>>>
>>>> --
>>>> R. Michael Churchill
>>>>
>>>
>>>
>>
>>
>> --
>> R. Michael Churchill
>>
>
>


-- 
R. Michael Churchill
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20171101/0ec13bf6/attachment.html>