[petsc-users] Modify matrix nonzero structure
Adrian Croucher
a.croucher at auckland.ac.nz
Tue May 28 20:33:38 CDT 2024
Thanks again Barry, it's working fine for me now too.
- Adrian
On 29/05/24 1:27 pm, Barry Smith wrote:
>
> There was a bug in my fix for parallel which I have fixed. You will
> need to
>
> git fetch
> git checkout main
> git branch -D barry/2024-05-27/fix-bug-baij-setvaluesblocked/release
> git checkout barry/2024-05-27/fix-bug-baij-setvaluesblocked/release
>
> I get the same results with your example with 1,2, and 3 ranks
>
>
>> On May 28, 2024, at 7:45 PM, Adrian Croucher
>> <a.croucher at auckland.ac.nz> wrote:
>>
>> hi Barry,
>>
>> Thanks, that change has fixed the error on 2 ranks for me.
>>
>> When I run on 3 ranks, there is no error, but it doesn't actually add
>> the extra values in to the matrix. Do you see that behaviour too?
>>
>> - Adrian
>>
>> On 29/05/24 4:33 am, Barry Smith wrote:
>>>
>>> Adrian,
>>>
>>> I could reproduce with 3 MPI ranks.
>>>
>>> Another error I had to fix. I also added a test example
>>>
>>> SInce I rebased the branch you will need to do something like
>>>
>>> git fetch
>>> git checkout main
>>> git branch -D barry/2024-05-27/fix-bug-baij-setvaluesblocked/release
>>> git checkout barry/2024-05-27/fix-bug-baij-setvaluesblocked/release
>>>
>>> Thanks for your patience
>>>
>>> Barry
>>>
>>>
>>>> On May 27, 2024, at 10:42 PM, Adrian Croucher
>>>> <a.croucher at auckland.ac.nz> wrote:
>>>>
>>>> Hmm, that's a bit weird. I haven't modified the test code - I
>>>> checked the file date to make sure. I also tried deleting my PETSc
>>>> build dir and rebuilding it, then rebuilding the test code. I still
>>>> get the error if I run on 2 ranks with -dm_mat_type (or -mat_type)
>>>> baij or mpibaij, but it's fine on aij or mpiaij.
>>>>
>>>> My actual (non-test) code is however working ok now, in serial or
>>>> parallel. So I don't think this should hold up merging your bugfix.
>>>>
>>>> - Adrian
>>>>
>>>> On 28/05/24 2:13 pm, Barry Smith wrote:
>>>>> When I run the exact code you sent with two ranks and—mat_type
>>>>> mpibaij, it runs as expected. If you modified the code in any way
>>>>> to demonstrate the bug, please send the modified code.
>>>>>
>>>>>
>>>>>
>>>>>> On May 27, 2024, at 9:37 PM, Adrian Croucher
>>>>>> <a.croucher at auckland.ac.nz> wrote:
>>>>>>
>>>>>> hi Barry,
>>>>>>
>>>>>> On 28/05/24 7:46 am, Barry Smith wrote:
>>>>>>>
>>>>>>> Thanks for reporting this. It is a bug. I have a fixed branch
>>>>>>> *barry/2024-05-27/fix-bug-baij-setvaluesblocked/release * and
>>>>>>> associated merge request
>>>>>>> https://urldefense.us/v3/__https://gitlab.com/petsc/petsc/-/merge_requests/7578__;!!G_uCfscf7eWS!dyTRr3EmobjcjKs0kzURm7EjPlyR-L6yBosZe_dky8HADNvWRJjBCsqt6Quq20DTsiHcW255AIkxd7mdGIssOXoFXVK_OG1L$
>>>>>>>
>>>>>> Thanks very much, that does appear to fix the bug when I run my
>>>>>> test program in serial.
>>>>>>
>>>>>> If I run it on 2 processes, it is OK with AIJ matrix type, but
>>>>>> with BAIJ I get an error (see below). Is there another problem,
>>>>>> or I am doing something else wrong?
>>>>>>
>>>>>> - Adrian
>>>>>>
>>>>>> [0]PETSC ERROR: --------------------- Error Message
>>>>>> --------------------------------------------------------------
>>>>>> [0]PETSC ERROR: Petsc has generated inconsistent data
>>>>>> [0]PETSC ERROR: Incorrect colmap
>>>>>> [0]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!dyTRr3EmobjcjKs0kzURm7EjPlyR-L6yBosZe_dky8HADNvWRJjBCsqt6Quq20DTsiHcW255AIkxd7mdGIssOXoFXe8LPGgS$ for trouble
>>>>>> shooting.
>>>>>> [0]PETSC ERROR: Petsc Release Version 3.21.1, unknown
>>>>>> [0]PETSC ERROR: ./matmodify on a main-debug named EN438880 by
>>>>>> acro018 Tue May 28 13:33:46 2024
>>>>>> [0]PETSC ERROR: Configure options --with-x --download-hdf5
>>>>>> --download-zlib --download-netcdf --download-pnetcdf
>>>>>> --download-exodusii --download-triangle --download-ptscotch
>>>>>> --download-chaco --download-hypre
>>>>>> [0]PETSC ERROR: #1 MatSetValuesBlocked_MPIBAIJ() at
>>>>>> /home/acro018/software/PETSc/code/src/mat/impls/baij/mpi/mpibaij.c:448
>>>>>> [0]PETSC ERROR: #2 MatSetValuesBlocked() at
>>>>>> /home/acro018/software/PETSc/code/src/mat/interface/matrix.c:2030
>>>>>> [0]PETSC ERROR: --------------------- Error Message
>>>>>> --------------------------------------------------------------
>>>>>> [0]PETSC ERROR: It appears a new error in the code was
>>>>>> triggered after a previous error, possibly because:
>>>>>> [0]PETSC ERROR: - The first error was not properly handled via
>>>>>> (for example) the use of
>>>>>> [0]PETSC ERROR: PetscCall(TheFunctionThatErrors()); or
>>>>>> [0]PETSC ERROR: - The second error was triggered while
>>>>>> handling the first error.
>>>>>> [0]PETSC ERROR: Above is the traceback for the previous
>>>>>> unhandled error, below the traceback for the next error
>>>>>> [0]PETSC ERROR: ALL ERRORS in the PETSc libraries are fatal,
>>>>>> you should add the appropriate error checking to the code
>>>>>> [0]PETSC ERROR: Petsc has generated inconsistent data
>>>>>> [0]PETSC ERROR: Incorrect colmap
>>>>>> [0]PETSC ERROR: See https://urldefense.us/v3/__https://petsc.org/release/faq/__;!!G_uCfscf7eWS!dyTRr3EmobjcjKs0kzURm7EjPlyR-L6yBosZe_dky8HADNvWRJjBCsqt6Quq20DTsiHcW255AIkxd7mdGIssOXoFXe8LPGgS$ for trouble
>>>>>> shooting.
>>>>>> [0]PETSC ERROR: Petsc Release Version 3.21.1, unknown
>>>>>> [0]PETSC ERROR: ./matmodify on a main-debug named EN438880 by
>>>>>> acro018 Tue May 28 13:33:46 2024
>>>>>> [0]PETSC ERROR: Configure options --with-x --download-hdf5
>>>>>> --download-zlib --download-netcdf --download-pnetcdf
>>>>>> --download-exodusii --download-triangle --download-ptscotch
>>>>>> --download-chaco --download-hypre
>>>>>> [0]PETSC ERROR: #1 MatSetValuesBlocked_MPIBAIJ() at
>>>>>> /home/acro018/software/PETSc/code/src/mat/impls/baij/mpi/mpibaij.c:448
>>>>>> [0]PETSC ERROR: #2 MatAssemblyEnd_MPIBAIJ() at
>>>>>> /home/acro018/software/PETSc/code/src/mat/impls/baij/mpi/mpibaij.c:943
>>>>>> [0]PETSC ERROR: #3 MatAssemblyEnd() at
>>>>>> /home/acro018/software/PETSc/code/src/mat/interface/matrix.c:5820
>>>>>> [0]PETSC ERROR: #4 matmodify.F90:40
>>>>>>
>>>>>>
>>>>>>> Barry
>>>>>>>
>>>>>>>
>>>>>>>> On May 26, 2024, at 10:45 PM, Adrian Croucher
>>>>>>>> <a.croucher at auckland.ac.nz> wrote:
>>>>>>>>
>>>>>>>> hi,
>>>>>>>>
>>>>>>>> I've been trying creating a matrix with DMCreateMatrix() and
>>>>>>>> then adding extra blocks of nonzeros into it using
>>>>>>>> MatSetValuesBlocked(), but getting some unexpected results if I
>>>>>>>> set the matrix type to BAIJ. It seems to behave as expected if
>>>>>>>> I use matrix type AIJ.
>>>>>>>>
>>>>>>>> I've attached a minimal example program. It reads in the DMPlex
>>>>>>>> from file, sets up a section on it, creates a matrix (blocksize
>>>>>>>> 2) and then inserts a single 2x2 block at global block indices
>>>>>>>> (0,7). It views the matrix before and after the insertion.
>>>>>>>>
>>>>>>>> If I run with "-dm_mat_type aij" it gives the expected results,
>>>>>>>> but with "-dm_mat_type baij" it doesn't - e.g. if run in
>>>>>>>> serial, it adds the new nonzeros in the right place but also
>>>>>>>> adds a whole lot of other duplicated entries in block row 0.
>>>>>>>>
>>>>>>>> Is there something I'm not understanding about BAIJ, or about
>>>>>>>> MatSetValuesBlocked()? or possibly some other mistake?
>>>>>>>>
>>>>>>>> - Adrian
>>>>>>>>
>>>>>>>> On 20/05/24 12:24 pm, Barry Smith wrote:
>>>>>>>>>
>>>>>>>>> You can call MatSetOption(mat,MAT_NEW_NONZERO_LOCATION_ERR)
>>>>>>>>> then insert the new values. If it is just a handful of new
>>>>>>>>> insertions the extra time should be small.
>>>>>>>>>
>>>>>>>>> Making a copy of the matrix won't give you a new matrix
>>>>>>>>> that is any faster to insert into so best to just use the same
>>>>>>>>> matrix.
>>>>>>>>>
>>>>>>>>> Barry
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On May 19, 2024, at 7:44 PM, Adrian Croucher
>>>>>>>>>> <a.croucher at auckland.ac.nz> wrote:
>>>>>>>>>>
>>>>>>>>>> This Message Is From an External Sender
>>>>>>>>>> This message came from outside your organization.
>>>>>>>>>> hi,
>>>>>>>>>>
>>>>>>>>>> I have a Jacobian matrix created using DMCreateMatrix(). What would be
>>>>>>>>>> the best way to add extra nonzero entries into it?
>>>>>>>>>>
>>>>>>>>>> I'm guessing that DMCreateMatrix() allocates the storage so the nonzero
>>>>>>>>>> structure can't really be easily modified. Would it be a case of
>>>>>>>>>> creating a new matrix, copying the nonzero entries from the original one
>>>>>>>>>> and then adding the extra ones, before calling MatSetUp() or similar? If
>>>>>>>>>> so, how exactly would you copy the nonzero structure from the original
>>>>>>>>>> matrix?
>>>>>>>>>>
>>>>>>>>>> Background: the flow problem I'm solving (on a DMPlex with finite volume
>>>>>>>>>> method) has complex source terms that depend on the solution (e.g.
>>>>>>>>>> pressure), and can also depend on other source terms. A simple example
>>>>>>>>>> is when fluid is extracted from one location, with a pressure-dependent
>>>>>>>>>> flow rate, and some of it is then reinjected in another location. This
>>>>>>>>>> can result in poor nonlinear solver convergence. I think the reason is
>>>>>>>>>> that there are effectively missing Jacobian entries in the row for the
>>>>>>>>>> reinjection cell, which should have an additional dependence on the
>>>>>>>>>> solution in the cell where fluid is extracted.
>>>>>>>>>>
>>>>>>>>>> - Adrian
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Dr Adrian Croucher
>>>>>>>>>> Senior Research Fellow
>>>>>>>>>> Department of Engineering Science
>>>>>>>>>> Waipapa Taumata Rau / University of Auckland, New Zealand
>>>>>>>>>> email:a.croucher at auckland.ac.nz
>>>>>>>>>> tel: +64 (0)9 923 4611
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Dr Adrian Croucher
>>>>>>>> Senior Research Fellow
>>>>>>>> Department of Engineering Science
>>>>>>>> Waipapa Taumata Rau / University of Auckland, New Zealand
>>>>>>>> email:a.croucher at auckland.ac.nz
>>>>>>>> tel: +64 (0)9 923 4611
>>>>>>>> <3x3grid.exo><matmodify.F90>
>>>>>>>
>>>>>> --
>>>>>> Dr Adrian Croucher
>>>>>> Senior Research Fellow
>>>>>> Department of Engineering Science
>>>>>> Waipapa Taumata Rau / University of Auckland, New Zealand
>>>>>> email:a.croucher at auckland.ac.nz
>>>>>> tel: +64 (0)9 923 4611
>>>>>
>>>> --
>>>> Dr Adrian Croucher
>>>> Senior Research Fellow
>>>> Department of Engineering Science
>>>> Waipapa Taumata Rau / University of Auckland, New Zealand
>>>> email:a.croucher at auckland.ac.nz
>>>> tel: +64 (0)9 923 4611
>>>
>> --
>> Dr Adrian Croucher
>> Senior Research Fellow
>> Department of Engineering Science
>> Waipapa Taumata Rau / University of Auckland, New Zealand
>> email:a.croucher at auckland.ac.nz
>> tel: +64 (0)9 923 4611
>
--
Dr Adrian Croucher
Senior Research Fellow
Department of Engineering Science
Waipapa Taumata Rau / University of Auckland, New Zealand
email:a.croucher at auckland.ac.nz
tel: +64 (0)9 923 4611
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240529/b0d574cb/attachment-0001.html>
More information about the petsc-users
mailing list