<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Oct 1, 2015 at 3:32 AM, Eike Mueller <span dir="ltr"><<a href="mailto:E.Mueller@bath.ac.uk" target="_blank">E.Mueller@bath.ac.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;background-color:rgb(255,255,255)">
<p>Hi Matt,</p>
<p><br>
</p>
<p>thanks a lot for your answer. Assembling the matrices into one large matrix should be ok, since I have control over that.</p>
<p><br>
</p>
<p>But am I right in assuming that the vectors have to be stored as one large PETSc vector? So when X = (x^(1),x^(2),...,x^(k)), where x^(i) are the individual vectors of length n_i, then I have to store X as one PETSc vector of length n_1+n_2+...+n_k? In the
data structure I use, storage for the vectors is already allocated, so currently I use</p>
<p>VecCreateSeqWithArray() to construct k separate PETSc vectors. Sounds like I can't do that, but have to create one big PETSc vector, and copy the data into that? That's ok, if it's the only option, since the overhead should not be too large, but I wanted
to check before doing something unnecessary.<br></p></div></blockquote><div> You should be able to use <a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecGetSubVector.html">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Vec/VecGetSubVector.html</a></div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;background-color:rgb(255,255,255)">
<p><span>Thanks,</span></p>
<p><br>
</p>
<p>Eike<br>
</p>
<p><br>
</p>
<div style="color:rgb(33,33,33)">
<hr style="display:inline-block;width:98%">
<div dir="ltr"><font style="font-size:11pt" color="#000000" face="Calibri, sans-serif"><b>From:</b> Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br>
<b>Sent:</b> 30 September 2015 15:27<br>
<b>To:</b> Eike Mueller<br>
<b>Cc:</b> <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><br>
<b>Subject:</b> Re: [petsc-users] fieldsplit-preconditioner for block-system of linear equations</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Wed, Sep 30, 2015 at 3:00 AM, Eike Mueller <span dir="ltr">
<<a href="mailto:E.Mueller@bath.ac.uk" target="_blank">E.Mueller@bath.ac.uk</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Dear PETSc,<br>
<br>
I am solving a linear system of equations<br>
<br>
(1) A.X = B<br>
<br>
where the vector X is divided into chunks, such that X= (x_1,x_2,…,x_k) and each of the vectors x_i has length n_i (same for the vector B=(b_1,b_2,…,b_k)). k=5 in my case, and n_i >> 1. This partitioning implies that the matrix has a block-structure, where
the submatrices A_{ij} are of size n_i x n_j. So explicitly for k=3:<br>
<br>
A_{1,1}.x_1 + A_{1,2}.x_2 + A_{1,3}.x_3 = b_1<br>
(2) A_{2,1}.x_1 + A_{2,2}.x_2 + A_{2,3}.x_3 = b_2<br>
A_{3,1}.x_1 + A_{3,2}.x_2 + A_{3,3}.x_3 = b_3<br>
<br>
I now want to solve this system with a Krylov-method (say, GMRES) and precondition it with a field-split preconditioner, such that the Schur-complement is formed in the 1-block. I know how to do this if I have assembled the big matrix A, and I store all x_i
in one big vector X (I construct the index-sets corresponding to the vectors x_1 and (x_2,x_3), and then call PCFieldSplitSetIS()). This all works without any problems.<br>
<br>
However, I now want to do this for an existing code which<br>
<br>
1. Assembles the matrices A_{ij} separately (i.e. they are not stored as part of a big matrix A, but are stored in independent memory locations)<br>
2. Stores the vectors x_i separately, i.e. they are not stored as parts of one big chunk of memory of size n_1+…+n_k<br>
<br>
I can of course implement the matrix application via a matrix shell, but is there still an easy way of using the fieldsplit preconditioner?<br>
</blockquote>
<div><br>
</div>
<div>From your description, it sounds like you could use</div>
<div><br>
</div>
<div> <a href="http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetLocalSubMatrix.html" target="_blank">http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatGetLocalSubMatrix.html</a></div>
<div><br>
</div>
<div>to assemble your separate matrices directly into a large matrix. Furthermore, if it really benefits you</div>
<div>to have separate memory, you can change the matrix type from MATIJ to MATNEST dynamically,</div>
<div>but none of your code would change. This would let you check things with LU, but still optimize when</div>
<div>necessary.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
The naive way I can think of is to allocate a new big matrix A, and copy the A_{ij} into the corresponding blocks. Then allocate big vectors x and b, and also copy in/out the data before and after the solve. This, however, seems to be wasteful, so before I
go down this route I wanted to double check if there is a way around it, since this seems to be a very common problem?<br>
<br>
Thanks a lot,<br>
<br>
Eike<br>
<br><span class=""><font color="#888888">
</font></span></blockquote><span class=""><font color="#888888">
</font></span></div><span class=""><font color="#888888">
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener</div>
</font></span></div>
</div>
</div>
</div>
</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>