<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Matt<br>
<br>
<div class="moz-cite-prefix">On 4/6/2015 6:49 AM, Matthew Knepley
wrote:<br>
</div>
<blockquote
cite="mid:CAMYG4GmD2FO5XFTxWm_kkaXrxeZeP1EAye0dkqmw8h8NcZbE=Q@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Mon, Apr 6, 2015 at 12:16 AM, Eric
Mittelstaedt <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:emittelstaedt@uidaho.edu" target="_blank">emittelstaedt@uidaho.edu</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Hello
all,<br>
We are working on implementing restriction and
prolongation operators for algebraic multigrid in our
finite difference (staggered grid) code that solves the
incompressible Stokes problem. So far, the restriction
and prolongation operators we have assembled in serial can
successfully solve on a simple 2D grid with multiple-level
v-cycles. However, we are running into problems when we
attempt to progress to parallel solves. One of the
primary errors we run into is an incompatibility in matrix
types for MatMatMatMult and similar calls:<br>
<br>
[0]PETSC ERROR: [1]PETSC ERROR: MatMatMatMult() line 9173
in /opt/petsc/src/mat/interface/matrix.c MatMatMatMult
requires A, seqaij, to be compatible with B, mpiaij, C,
seqaij<br>
MatMatMatMult() line 9173 in
/opt/petsc/src/mat/interface/matrix.c MatMatMatMult
requires A, seqaij, to be compatible with B, mpiaij, C,
seqaij<br>
<br>
<br>
The root cause of this is likely the matrix types we chose
for restriction and prolongation. To ease calculation of
our restriction (R) and prolongation (P) matrices, we have
created them as sparse, sequential matrices (SEQAIJ) that
are identical on all procs. Will this matrix type
actually work for R and P operators in parallel?<br>
</blockquote>
<div><br>
</div>
<div>You need to create them as parallel matrices, MPIAIJ
for example, that have the same layout as the system
matrix.</div>
</div>
</div>
</div>
</blockquote>
<br>
Okay, this is what I thought was the problem. It is really a matter
of making sure we have the correct information on each processor. <br>
<blockquote
cite="mid:CAMYG4GmD2FO5XFTxWm_kkaXrxeZeP1EAye0dkqmw8h8NcZbE=Q@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"> It seem
that setting this matrix type correctly is the heart of
our problem. Originally, we chose SEQAIJ because we want
to let Petsc deal with partitioning of the grid on
multiple processors. If we do this, however, we don't
know a priori whether Petsc will place a processor
boundary on an even or odd nodal point. This makes
determining the processor bounds on the subsequent coarse
levels difficult. One method for resolving this may be to
handle the partitioning ourselves, but it would be nice to
avoid this if possible. Is there a better way to set up
these operators?<br>
</blockquote>
<div><br>
</div>
<div>You use the same partitioning as the system matrices on
the two levels. Does this makes sense?</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
It does, but I am unsure how to get the partitioning of the coarser
grids since we don't deal with them directly. Is it safe to assume
that the partitioning on the coarser grid will always overlap with
the same part of the domain as the fine level? In other words, the
current processor will receive the coarse grid that is formed by the
portion of the restriction matrix that it owns, right?<br>
<br>
Thanks!<br>
Eric<br>
<br>
<br>
<blockquote
cite="mid:CAMYG4GmD2FO5XFTxWm_kkaXrxeZeP1EAye0dkqmw8h8NcZbE=Q@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div><br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"> Thank
you,<br>
Eric and Arthur<br>
<br>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div class="gmail_signature">What most experimenters take for
granted before they begin their experiments is infinitely
more interesting than any results to which their experiments
lead.<br>
-- Norbert Wiener</div>
</div>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
*************************************
Eric Mittelstaedt, Ph.D.
Assistant Professor
Dept. of Geological Sciences
University of Idaho
email: <a class="moz-txt-link-abbreviated" href="mailto:emittelstaedt@uidaho.edu">emittelstaedt@uidaho.edu</a>
phone: (208) 885 2045
<a class="moz-txt-link-freetext" href="http://scholar.google.com/citations?user=DvRzEqkAAAAJ">http://scholar.google.com/citations?user=DvRzEqkAAAAJ</a>
*************************************
</pre>
</body>
</html>