[petsc-users] unsorted local columns in 3.8?

[Randy Michael Churchill]

OK, this might not be completely satisfactory, because it doesn't show the
partitioning or how the matrix is created, but this reproduces the problem.
I wrote out my matrix, Amat, from the larger simulation, and load it in
this script. This must be run with MPI rank greater than 1. This may be
some combination of my petsc.rc, because when I use the PetscInitialize
with it, it throws the error, but when using default (PETSC_NULL_CHARACTER)
it runs fine.


On Tue, Oct 31, 2017 at 9:58 AM, Hong <hzhang at mcs.anl.gov> wrote:

> Randy:
> It could be a bug or a missing feature in our new
> MatCreateSubMatrix_MPIAIJ_SameRowDist().
> It would be helpful if you can provide us a simple example that produces
> this example.
> Hong
>
> I'm running a Fortran code that was just changed over to using petsc 3.8
>> (previously petsc 3.7.6). An error was thrown during a KSPSetUp() call. The
>> error is "unsorted iscol_local is not implemented yet" (see full error
>> below). I tried to trace down the difference in the source files, but where
>> the error occurs (MatCreateSubMatrix_MPIAIJ_SameRowDist()) doesn't seem
>> to have existed in v3.7.6, so I'm unsure how to compare. It seems the error
>> is that the order of the columns locally are unsorted, though I don't think
>> I specify a column order in the creation of the matrix:
>>      call MatCreate(this%comm,AA,ierr)
>>      call MatSetSizes(AA,npetscloc,npetscloc,nreal,nreal,ierr)
>>      call MatSetType(AA,MATAIJ,ierr)
>>      call MatSetup(AA,ierr)
>>      call MatGetOwnershipRange(AA,low,high,ierr)
>>      allocate(d_nnz(npetscloc),o_nnz(npetscloc))
>>      call getNNZ(grid,npetscloc,low,high,d_nnz,o_nnz,this%xgc_petsc,nr
>> eal,ierr)
>>      call MatSeqAIJSetPreallocation(AA,PETSC_NULL_INTEGER,d_nnz,ierr)
>>      call MatMPIAIJSetPreallocation(AA,PETSC_NULL_INTEGER,d_nnz,PETSC_
>> NULL_INTEGER,o_nnz,ierr)
>>      deallocate(d_nnz,o_nnz)
>>      call MatSetOption(AA,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE,ierr)
>>      call MatSetOption(AA,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE,ierr)
>>      call MatSetup(AA,ierr)
>>
>>
>> [62]PETSC ERROR: --------------------- Error Message
>> --------------------------------------------------------------
>> [62]PETSC ERROR: No support for this operation for this object type
>> [62]PETSC ERROR: unsorted iscol_local is not implemented yet
>> [62]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
>> for trouble shooting.
>> [62]PETSC ERROR: Petsc Release Version 3.8.0, unknown[62]PETSC ERROR: #1
>> MatCreateSubMatrix_MPIAIJ_SameRowDist() line 3418 in
>> /global/u1/r/rchurchi/petsc/3.8.0/src/mat/impls/aij/mpi/mpiaij.c
>> [62]PETSC ERROR: #2 MatCreateSubMatrix_MPIAIJ() line 3247 in
>> /global/u1/r/rchurchi/petsc/3.8.0/src/mat/impls/aij/mpi/mpiaij.c
>> [62]PETSC ERROR: #3 MatCreateSubMatrix() line 7872 in
>> /global/u1/r/rchurchi/petsc/3.8.0/src/mat/interface/matrix.c
>> [62]PETSC ERROR: #4 PCGAMGCreateLevel_GAMG() line 383 in
>> /global/u1/r/rchurchi/petsc/3.8.0/src/ksp/pc/impls/gamg/gamg.c
>> [62]PETSC ERROR: #5 PCSetUp_GAMG() line 561 in
>> /global/u1/r/rchurchi/petsc/3.8.0/src/ksp/pc/impls/gamg/gamg.c
>> [62]PETSC ERROR: #6 PCSetUp() line 924 in /global/u1/r/rchurchi/petsc/3.
>> 8.0/src/ksp/pc/interface/precon.c
>> [62]PETSC ERROR: #7 KSPSetUp() line 378 in /global/u1/r/rchurchi/petsc/3.
>> 8.0/src/ksp/ksp/interface/itfunc.c
>>
>> --
>> R. Michael Churchill
>>
>
>


-- 
R. Michael Churchill
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20171101/2cd17198/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Amat.dat
Type: application/octet-stream
Size: 7795408 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20171101/2cd17198/attachment-0004.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Amat.dat.info
Type: application/octet-stream
Size: 21 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20171101/2cd17198/attachment-0005.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: petsc.rc
Type: application/octet-stream
Size: 10457 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20171101/2cd17198/attachment-0006.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_petsc38.F90
Type: application/octet-stream
Size: 1407 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20171101/2cd17198/attachment-0007.obj>