<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
Mark's suggestion will definitely help a lot. Remove the
displacement bc equations or include them in the matrix by zeroing
out the row and putting a 1 on the diagonal. The Lagrange
multiplier will cause grief.<br>
<pre class="moz-signature" cols="72">
</pre>
<div class="moz-cite-prefix">On 8/20/21 11:21 AM, Mark Adams wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CADOhEh7KhcaJrEfcT95HYv_=WH6OQ4u__8QaZtAJvLG_1fVGDw@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">Constraints are a pain with scalable/iterative
solvers. If you order the constraints last then ILU should work
as well as it can work, but AMG gets confused by the constraint
equations.
<div>You could look at PETSc's Stokes solvers, but it would be
best if you could remove the constrained equations from your
system if they are just simple point wise BC's.</div>
<div>Mark</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Aug 20, 2021 at 8:53
AM Наздрачёв Виктор <<a href="mailto:numbersixvs@gmail.com"
moz-do-not-send="true">numbersixvs@gmail.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<p class="MsoNormal"
style="margin:0cm;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"><b><span
style="font-size:14pt" lang="EN-US">Hello, dear PETSc
team!</span></b></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">I have a 3D elasticity with heterogeneous
properties problem. There is unstructured grid with
aspect ratio varied from 4 to 25. Dirichlet BCs (bottom
zero displacements) are imposed via linear constraint
equations using Lagrange multipliers. Also, Neumann
(traction) BCs are imposed on side edges of mesh.
Gravity load is also accounted for.</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">I can solve this problem with <b>dgmres
solver</b> and <b>ILU</b> as a <b>preconditioner</b>.
But ILU doesn`t support parallel computing, so I decided
to use Euclid or Boomeramg as a preconditioner. The
issue is in slow convergence and high memory
consumption, much higher, than for ILU.</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">E.g., for source matrix size 2.14 GB with <b>ILU-0
preconditioning</b> memory consumption is about 5.9
GB, and the process converges due to 767 iterations, and
with <b>Euclid-0 preconditioning</b> memory consumption
is about 8.7 GB, and the process converges due to 1732
iterations.</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">One of the following preconditioners is
currently in use: <b>ILU-0, ILU-1, Hypre (Euclid), Hypre
(boomeramg)</b>.</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">As a result of computations <b><i>(logs and
memory logs are attached)</i></b>, the following is
established for preconditioners:</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">1. <b>ILU-0</b>: does not always provide
convergence (or provides, but slow); uses an acceptable
amount of RAM; does not support parallel computing.</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">2. <b>ILU-1</b>: stable; memory consumption
is much higher than that of ILU-0; does not support
parallel computing.</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">3. <b>Euclid</b>: provides very slow
convergence, calculations are performed several times
slower than for ILU-0; memory consumption greatly
exceeds both ILU-0 and ILU-1; supports parallel
computing. Also “drop tolerance” doesn’t provide enough
accuracy in some cells, so I don’t use it.</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">4. <b>Boomeramg</b>: provides very slow
convergence, calculations are performed several times
slower than for ILU-0; memory consumption greatly
exceeds both ILU-0 and ILU-1; supports parallel
computing.</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US"> </span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">In this regard, the following questions
arose:</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">1. Is this behavior expected for HYPRE in
computations with 1 MPI process? If not, is that problem
can be related to <b>PETSc</b> or <b>HYPRE</b>?</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">2. Hypre (Euclid) has much fewer parameters
than ILU. Among them is the factorization level <b>"-pc_hypre_euclid_level
<now -2: formerly -2>: Factorization levels
(None)"</b> and its default value looks very strange,
moreover, it doesn’t matter what factor is chosen -2, -1
or 0. Could it be that the parameter is confused with
Column pivot tolerance in ILU - <b>"-pc_factor_column_pivot
<-2.: -2.>: Column pivot tolerance (used only
for some factorization) (PCFactorSetColumnPivot)"</b>?</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">3. What preconditioner would you recommend
to: optimize <b>convergence</b>, <b>memory</b> consumption,
add <b>parallel computing</b>?</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">4. How can we theoretically estimate memory
costs with <b>ILU, Euclid, Boomeramg</b>?</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US">5. At what stage are memory leaks most
likely?</span></p>
<p class="MsoNormal" style="margin:0cm 0cm
8pt;line-height:15.6933px;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt;line-height:19.9733px"
lang="EN-US"> </span></p>
<p class="MsoNormal"
style="margin:0cm;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"><span
style="font-size:14pt" lang="EN-US">In any case, thank
you so much for your attention! Will be grateful for any
response.<br>
</span><span style="font-size:10.5pt" lang="EN-US"><br>
</span><span style="font-size:14pt" lang="EN-US">Kind
regards,<br>
Viktor Nazdrachev<br>
R&D senior researcher<br>
Geosteering Technologies LLC</span></p>
</div>
</blockquote>
</div>
</blockquote>
<br>
</body>
</html>