[petsc-users] Modify matrix nonzero structure

Tue May 28 11:33:40 CDT 2024

   Adrian,

   I could reproduce with 3 MPI ranks.

   Another error I had to fix. I also added a test example 

   SInce I rebased the branch you will need to do something like

   git fetch
   git checkout main
   git branch -D barry/2024-05-27/fix-bug-baij-setvaluesblocked/release
  git checkout barry/2024-05-27/fix-bug-baij-setvaluesblocked/release

  Thanks for your patience 

   Barry

> On May 27, 2024, at 10:42 PM, Adrian Croucher <a.croucher at auckland.ac.nz> wrote:
> 
> Hmm, that's a bit weird. I haven't modified the test code - I checked the file date to make sure. I also tried deleting my PETSc build dir and rebuilding it, then rebuilding the test code. I still get the error if I run on 2 ranks with -dm_mat_type (or -mat_type) baij or mpibaij, but it's fine on aij or mpiaij.
> 
> My actual (non-test) code is however working ok now, in serial or parallel. So I don't think this should hold up merging your bugfix.
> 
> - Adrian
> 
> On 28/05/24 2:13 pm, Barry Smith wrote:
>>  
>>   When I run the exact code you sent with two ranks and—mat_type mpibaij, it runs as expected. If you modified the code in any way to demonstrate the bug, please send the modified code.
>> 
>> 
>> 
>>> On May 27, 2024, at 9:37 PM, Adrian Croucher <a.croucher at auckland.ac.nz> <mailto:a.croucher at auckland.ac.nz> wrote:
>>> 
>>> hi Barry,
>>> 
>>> On 28/05/24 7:46 am, Barry Smith wrote:
>>>> 
>>>>    Thanks for reporting this. It is a bug. I have a fixed branch barry/2024-05-27/fix-bug-baij-setvaluesblocked/release  and associated merge request https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/merge_requests/7578__;!!G_uCfscf7eWS!doaVfskPKURsqYgA0bpocyW5cbPLsGwTjv41wqfhw_nlPc6a7scHMULs9UqHfAz74BFdHtbPJ-LBlRkYElrUKjE$ 
>>>> 
>>> Thanks very much, that does appear to fix the bug when I run my test program in serial.
>>> 
>>> If I run it on 2 processes, it is OK with AIJ matrix type, but with BAIJ I get an error (see below). Is there another problem, or I am doing something else wrong?
>>> 
>>> - Adrian
>>> 
>>> [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
>>> [0]PETSC ERROR: Petsc has generated inconsistent data
>>> [0]PETSC ERROR: Incorrect colmap
>>> [0]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!doaVfskPKURsqYgA0bpocyW5cbPLsGwTjv41wqfhw_nlPc6a7scHMULs9UqHfAz74BFdHtbPJ-LBlRkY0Lao4Fs$  for trouble shooting.
>>> [0]PETSC ERROR: Petsc Release Version 3.21.1, unknown 
>>> [0]PETSC ERROR: ./matmodify on a main-debug named EN438880 by acro018 Tue May 28 13:33:46 2024
>>> [0]PETSC ERROR: Configure options --with-x --download-hdf5 --download-zlib --download-netcdf --download-pnetcdf --download-exodusii --download-triangle --download-ptscotch --download-chaco --download-hypre
>>> [0]PETSC ERROR: #1 MatSetValuesBlocked_MPIBAIJ() at /home/acro018/software/PETSc/code/src/mat/impls/baij/mpi/mpibaij.c:448
>>> [0]PETSC ERROR: #2 MatSetValuesBlocked() at /home/acro018/software/PETSc/code/src/mat/interface/matrix.c:2030
>>> [0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
>>> [0]PETSC ERROR:   It appears a new error in the code was triggered after a previous error, possibly because:
>>> [0]PETSC ERROR:   -  The first error was not properly handled via (for example) the use of
>>> [0]PETSC ERROR:      PetscCall(TheFunctionThatErrors()); or
>>> [0]PETSC ERROR:   -  The second error was triggered while handling the first error.
>>> [0]PETSC ERROR:   Above is the traceback for the previous unhandled error, below the traceback for the next error
>>> [0]PETSC ERROR:   ALL ERRORS in the PETSc libraries are fatal, you should add the appropriate error checking to the code
>>> [0]PETSC ERROR: Petsc has generated inconsistent data
>>> [0]PETSC ERROR: Incorrect colmap
>>> [0]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!doaVfskPKURsqYgA0bpocyW5cbPLsGwTjv41wqfhw_nlPc6a7scHMULs9UqHfAz74BFdHtbPJ-LBlRkY0Lao4Fs$  for trouble shooting.
>>> [0]PETSC ERROR: Petsc Release Version 3.21.1, unknown 
>>> [0]PETSC ERROR: ./matmodify on a main-debug named EN438880 by acro018 Tue May 28 13:33:46 2024
>>> [0]PETSC ERROR: Configure options --with-x --download-hdf5 --download-zlib --download-netcdf --download-pnetcdf --download-exodusii --download-triangle --download-ptscotch --download-chaco --download-hypre
>>> [0]PETSC ERROR: #1 MatSetValuesBlocked_MPIBAIJ() at /home/acro018/software/PETSc/code/src/mat/impls/baij/mpi/mpibaij.c:448
>>> [0]PETSC ERROR: #2 MatAssemblyEnd_MPIBAIJ() at /home/acro018/software/PETSc/code/src/mat/impls/baij/mpi/mpibaij.c:943
>>> [0]PETSC ERROR: #3 MatAssemblyEnd() at /home/acro018/software/PETSc/code/src/mat/interface/matrix.c:5820
>>> [0]PETSC ERROR: #4 matmodify.F90:40
>>> 
>>> 
>>> 
>>>>   Barry
>>>> 
>>>> 
>>>>> On May 26, 2024, at 10:45 PM, Adrian Croucher <a.croucher at auckland.ac.nz> <mailto:a.croucher at auckland.ac.nz> wrote:
>>>>> 
>>>>> hi,
>>>>> 
>>>>> I've been trying creating a matrix with DMCreateMatrix() and then adding extra blocks of nonzeros into it using MatSetValuesBlocked(), but getting some unexpected results if I set the matrix type to BAIJ. It seems to behave as expected if I use matrix type AIJ.
>>>>> 
>>>>> I've attached a minimal example program. It reads in the DMPlex from file, sets up a section on it, creates a matrix (blocksize 2) and then inserts a single 2x2 block at global block indices (0,7). It views the matrix before and after the insertion.
>>>>> 
>>>>> If I run with "-dm_mat_type aij" it gives the expected results, but with "-dm_mat_type baij" it doesn't - e.g. if run in serial, it adds the new nonzeros in the right place but also adds a whole lot of other duplicated entries in block row 0.
>>>>> 
>>>>> Is there something I'm not understanding about BAIJ, or about MatSetValuesBlocked()? or possibly some other mistake?
>>>>> 
>>>>> - Adrian
>>>>> 
>>>>> On 20/05/24 12:24 pm, Barry Smith wrote:
>>>>>> 
>>>>>>    You can call MatSetOption(mat,MAT_NEW_NONZERO_LOCATION_ERR) then insert the new values. If it is just a handful of new insertions the extra time should be small.
>>>>>> 
>>>>>>     Making a copy of the matrix won't give you a new matrix that is any faster to insert into so best to just use the same matrix.
>>>>>> 
>>>>>>   Barry
>>>>>> 
>>>>>> 
>>>>>>> On May 19, 2024, at 7:44 PM, Adrian Croucher <a.croucher at auckland.ac.nz> <mailto:a.croucher at auckland.ac.nz> wrote:
>>>>>>> 
>>>>>>> This Message Is From an External Sender
>>>>>>> This message came from outside your organization.
>>>>>>> hi,
>>>>>>> 
>>>>>>> I have a Jacobian matrix created using DMCreateMatrix(). What would be 
>>>>>>> the best way to add extra nonzero entries into it?
>>>>>>> 
>>>>>>> I'm guessing that DMCreateMatrix() allocates the storage so the nonzero 
>>>>>>> structure can't really be easily modified. Would it be a case of 
>>>>>>> creating a new matrix, copying the nonzero entries from the original one 
>>>>>>> and then adding the extra ones, before calling MatSetUp() or similar? If 
>>>>>>> so, how exactly would you copy the nonzero structure from the original 
>>>>>>> matrix?
>>>>>>> 
>>>>>>> Background: the flow problem I'm solving (on a DMPlex with finite volume 
>>>>>>> method) has complex source terms that depend on the solution (e.g. 
>>>>>>> pressure), and can also depend on other source terms. A simple example 
>>>>>>> is when fluid is extracted from one location, with a pressure-dependent 
>>>>>>> flow rate, and some of it is then reinjected in another location. This 
>>>>>>> can result in poor nonlinear solver convergence. I think the reason is 
>>>>>>> that there are effectively missing Jacobian entries in the row for the 
>>>>>>> reinjection cell, which should have an additional dependence on the 
>>>>>>> solution in the cell where fluid is extracted.
>>>>>>> 
>>>>>>> - Adrian
>>>>>>> 
>>>>>>> -- 
>>>>>>> Dr Adrian Croucher
>>>>>>> Senior Research Fellow
>>>>>>> Department of Engineering Science
>>>>>>> Waipapa Taumata Rau / University of Auckland, New Zealand
>>>>>>> email: a.croucher at auckland.ac.nz <mailto:a.croucher at auckland.ac.nz>
>>>>>>> tel: +64 (0)9 923 4611
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> -- 
>>>>> Dr Adrian Croucher
>>>>> Senior Research Fellow
>>>>> Department of Engineering Science
>>>>> Waipapa Taumata Rau / University of Auckland, New Zealand
>>>>> email: a.croucher at auckland.ac.nz <mailto:a.croucher at auckland.ac.nz>
>>>>> tel: +64 (0)9 923 4611
>>>>> <3x3grid.exo><matmodify.F90>
>>>> 
>>> -- 
>>> Dr Adrian Croucher
>>> Senior Research Fellow
>>> Department of Engineering Science
>>> Waipapa Taumata Rau / University of Auckland, New Zealand
>>> email: a.croucher at auckland.ac.nz <mailto:a.croucher at auckland.ac.nz>
>>> tel: +64 (0)9 923 4611
>> 
> -- 
> Dr Adrian Croucher
> Senior Research Fellow
> Department of Engineering Science
> Waipapa Taumata Rau / University of Auckland, New Zealand
> email: a.croucher at auckland.ac.nz <mailto:a.croucher at auckland.ac.nz>
> tel: +64 (0)9 923 4611

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240528/236a6c28/attachment-0001.html>