[petsc-users] Why use MATMPIBAIJ?
Hoang Giang Bui
hgbk2008 at gmail.com
Mon Jan 25 11:13:58 CST 2016
OK, let's come back to my problem. I got your point about the interaction
between components in one block. In my case, the interaction is strong.
As you said, I try this:
ierr = KSPSetFromOptions(ksp); CHKERRQ(ierr);
ierr = PCFieldSplitGetSubKSP(pc, &nsplits, &sub_ksp); CHKERRQ(ierr);
ksp_U = sub_ksp[0];
ierr = KSPGetOperators(ksp_U, &A_U, &P_U); CHKERRQ(ierr);
ierr = MatSetBlockSize(A_U, 3); CHKERRQ(ierr);
ierr = MatSetBlockSize(P_U, 3); CHKERRQ(ierr);
ierr = PetscFree(sub_ksp); CHKERRQ(ierr);
But it seems doesn't work. The output from -ksp_view shows that matrix
passed to Hypre still has bs=1
KSP Object: (fieldsplit_u_) 8 MPI processes
type: preonly
maximum iterations=10000, initial guess is zero
tolerances: relative=1e-05, absolute=1e-50, divergence=10000
left preconditioning
using NONE norm type for convergence test
PC Object: (fieldsplit_u_) 8 MPI processes
type: hypre
HYPRE BoomerAMG preconditioning
HYPRE BoomerAMG: Cycle type V
HYPRE BoomerAMG: Maximum number of levels 25
HYPRE BoomerAMG: Maximum number of iterations PER hypre call 1
HYPRE BoomerAMG: Convergence tolerance PER hypre call 0
HYPRE BoomerAMG: Threshold for strong coupling 0.25
HYPRE BoomerAMG: Interpolation truncation factor 0
HYPRE BoomerAMG: Interpolation: max elements per row 0
HYPRE BoomerAMG: Number of levels of aggressive coarsening 0
HYPRE BoomerAMG: Number of paths for aggressive coarsening 1
HYPRE BoomerAMG: Maximum row sums 0.9
HYPRE BoomerAMG: Sweeps down 1
HYPRE BoomerAMG: Sweeps up 1
HYPRE BoomerAMG: Sweeps on coarse 1
HYPRE BoomerAMG: Relax down symmetric-SOR/Jacobi
HYPRE BoomerAMG: Relax up symmetric-SOR/Jacobi
HYPRE BoomerAMG: Relax on coarse Gaussian-elimination
HYPRE BoomerAMG: Relax weight (all) 1
HYPRE BoomerAMG: Outer relax weight (all) 1
HYPRE BoomerAMG: Using CF-relaxation
HYPRE BoomerAMG: Measure type local
HYPRE BoomerAMG: Coarsen type PMIS
HYPRE BoomerAMG: Interpolation type classical
linear system matrix = precond matrix:
Mat Object: (fieldsplit_u_) 8 MPI processes
type: mpiaij
rows=792333, cols=792333
total: nonzeros=1.39004e+08, allocated nonzeros=1.39004e+08
total number of mallocs used during MatSetValues calls =0
using I-node (on process 0) routines: found 30057 nodes, limit
used is 5
In other test, I can see the block size bs=3 in the section of Mat Object
Regardless the setup cost of Hypre AMG, I saw it gives quite a radical
performance, providing that the material parameters does not vary strongly,
and the geometry is regular enough.
Giang
On Fri, Jan 22, 2016 at 2:57 PM, Matthew Knepley <knepley at gmail.com> wrote:
> On Fri, Jan 22, 2016 at 7:27 AM, Hoang Giang Bui <hgbk2008 at gmail.com>
> wrote:
>
>> DO you mean the option pc_fieldsplit_block_size? In this thread:
>>
>> http://petsc-users.mcs.anl.narkive.com/qSHIOFhh/fieldsplit-error
>>
>
> No. "Block Size" is confusing on PETSc since it is used to do several
> things. Here block size
> is being used to split the matrix. You do not need this since you are
> prescribing your splits. The
> matrix block size is used two ways:
>
> 1) To indicate that matrix values come in logically dense blocks
>
> 2) To change the storage to match this logical arrangement
>
> After everything works, we can just indicate to the submatrix which is
> extracted that it has a
> certain block size. However, for the Laplacian I expect it not to matter.
>
>
>> It assumes you have a constant number of fields at each grid point, am I
>> right? However, my field split is not constant, like
>> [u1_x u1_y u1_z p_1 u2_x u2_y u2_z u3_x u3_y
>> u3_z p_3 u4_x u4_y u4_z]
>>
>> Subsequently the fieldsplit is
>> [u1_x u1_y u1_z u2_x u2_y u2_z u3_x u3_y u3_z
>> u4_x u4_y u4_z]
>> [p_1 p_3]
>>
>> Then what is the option to set block size 3 for split 0?
>>
>> Sorry, I search several forum threads but cannot figure out the options
>> as you said.
>>
>>
>>
>>> You can still do that. It can be done with options once the
>>> decomposition is working. Its true that these solvers
>>> work better with the block size set. However, if its the P2 Laplacian it
>>> does not really matter since its uncoupled.
>>>
>>> Yes, I agree it's uncoupled with the other field, but the crucial factor
>> defining the quality of the block preconditioner is the approximate
>> inversion of individual block. I would merely try block Jacobi first,
>> because it's quite simple. Nevertheless, fieldsplit implements other nice
>> things, like Schur complement, etc.
>>
>
> I think concepts are getting confused here. I was talking about the
> interaction of components in one block (the P2 block). You
> are talking about interaction between blocks.
>
> Thanks,
>
> Matt
>
>
>> Giang
>>
>>
>>
>> On Fri, Jan 22, 2016 at 11:15 AM, Matthew Knepley <knepley at gmail.com>
>> wrote:
>>
>>> On Fri, Jan 22, 2016 at 3:40 AM, Hoang Giang Bui <hgbk2008 at gmail.com>
>>> wrote:
>>>
>>>> Hi Matt
>>>> I would rather like to set the block size for block P2 too. Why?
>>>>
>>>> Because in one of my test (for problem involves only [u_x u_y u_z]),
>>>> the gmres + Hypre AMG converges in 50 steps with block size 3, whereby it
>>>> increases to 140 if block size is 1 (see attached files).
>>>>
>>>
>>> You can still do that. It can be done with options once the
>>> decomposition is working. Its true that these solvers
>>> work better with the block size set. However, if its the P2 Laplacian it
>>> does not really matter since its uncoupled.
>>>
>>> This gives me the impression that AMG will give better inversion for
>>>> "P2" block if I can set its block size to 3. Of course it's still an
>>>> hypothesis but worth to try.
>>>>
>>>> Another question: In one of the Petsc presentation, you said the Hypre
>>>> AMG does not scale well, because set up cost amortize the iterations. How
>>>> is it quantified? and what is the memory overhead?
>>>>
>>>
>>> I said the Hypre setup cost is not scalable, but it can be amortized
>>> over the iterations. You can quantify this
>>> just by looking at the PCSetUp time as your increase the number of
>>> processes. I don't think they have a good
>>> model for the memory usage, and if they do, I do not know what it is.
>>> However, generally Hypre takes more
>>> memory than the agglomeration MG like ML or GAMG.
>>>
>>> Thanks,
>>>
>>> Matt
>>>
>>>
>>>>
>>>> Giang
>>>>
>>>> On Mon, Jan 18, 2016 at 5:25 PM, Jed Brown <jed at jedbrown.org> wrote:
>>>>
>>>>> Hoang Giang Bui <hgbk2008 at gmail.com> writes:
>>>>>
>>>>> > Why P2/P2 is not for co-located discretization?
>>>>>
>>>>> Matt typed "P2/P2" when me meant "P2/P1".
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> -- Norbert Wiener
>>>
>>
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160125/1f174f25/attachment-0001.html>
More information about the petsc-users
mailing list