Matt,<br><br>Alright, that means I need to continue learning how to use MatSetValues(). With my 6x6 example I tried filling it with four 3x3 sub matrices, but when I do that I get the error 'sum of local sizes 12 does not equal global size.' I had 4 processors each calling MatSetValues for their own 3x3. Graphically, I arranged the nodes 0 1<br>
2 3 where process 0 had global rows 0-2 and global columns 0-2; process 1 had 0-2, 3-5; process 2 had 3-5, 0-2; and process 3 had 3-5, 3-5. From the documentation, I think this should be correct, but I'm not sure. Also, which format would you recommend for storing the matrix?<br>
<br><br>Jack,<br><br>I'm a summer intern just getting started with this project, so I don't know all the details yet (I can ask though). I know I need to find the Green's function which will involve the trace of the inverted Hamiltonian, as well as the rest of the matrix. I have inquired about avoiding the inversion altogether, but my instructor doesn't believe there is a way around it. Once I've worked through the math I want to explore other options though.<br>
<br>Respectfully,<br>Adam<br><br><div class="gmail_quote">On Tue, Jun 28, 2011 at 6:08 PM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com">knepley@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im">On Tue, Jun 28, 2011 at 5:01 PM, Adam Byrd <span dir="ltr"><<a href="mailto:adam1.byrd@gmail.com" target="_blank">adam1.byrd@gmail.com</a>></span> wrote:<br></div><div class="gmail_quote"><div class="im">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Actually, it's quite sparse. In the 3600x3600 there are only just 4 nonzero entries in each row. This means it's 99.9% empty. My smaller 6x6 example is dense, but it's only practice building and manipulating matrices.<br>
</blockquote><div><br></div></div><div>Ah, then its easy. Just call MatSetValues() with each block. Then use MUMPS to do a sparse direct solve.</div><div><br></div><div> Matt</div><div><div></div><div class="h5"><div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Respectfully,<br><font color="#888888">Adam</font><div><div></div><div><br><br><div class="gmail_quote">On Tue, Jun 28, 2011 at 5:55 PM, Matthew Knepley <span dir="ltr"><<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
It sounds like you have a dense matrix (from your example). Is this true? If so, you should use Elemental (on Google Code).<div><br></div><div> Thanks,</div><div><br></div><div> Matt<br><br><div class="gmail_quote">
<div>
On Tue, Jun 28, 2011 at 8:55 AM, Adam Byrd <span dir="ltr"><<a href="mailto:adam1.byrd@gmail.com" target="_blank">adam1.byrd@gmail.com</a>></span> wrote:<br>
</div><div><div></div><div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote">Hi,<div><br></div><div>I'm rather new to PETSc and trying to work out the best way to create and fill a large sparse matrix distributed over many processors. Currently, my goal is to create a 3600x3600 matrix in units of 12x12 blocks with several blocks on any given node. I'd like to create the matrix in such a way that each node only holds the information in it's handful of blocks and not the entire matrix. Eventually, this matrix is to be inverted (I know, inversion should be avoided, but as this is a Hamiltonian matrix from which I need the Green's function, I'm unaware of a way to forgo carrying out the inversion). Additionally, the values will be changed slightly and the matrix will be repeatedly inverted. It's structure will remain the same. In order to learn how to do this is I am starting with a small 6x6 matrix broken into four 3x3 blocks and distributed one block per node. I've been able to create a local 3x3 matrix on each node, with it's own values, and with the global row/column IDs correctly set to [0, 1, 2] or [3, 4, 5] depending on where the block is in the matrix. My problem manifests when I try to create the larger matrix from the individual smaller ones. When the matrix is constructed I'm trying to use MatSetValues and having each node pass in it's 3x3 block. I end up with an error that the sum of local lengths 12x12 does not match the global length 6x6. It appears as though this is from passing in four 3x3s and the program interpreting that as a 12x12 instead of as a 6x6 with the blocks in a grid.</div>
<div><br></div><div>My question is then: is it possible to fill a matrix as a grid of blocks, or can I only fill it in groups of rows or columns? Also, am I approaching this problem the correct way, or are there more efficient ways of building this matrix with the ultimate goal of inverting it?</div>
<div><br></div><div>I have included my copy of a modified example if it helps. I do apologize if this is answered somewhere in the documentation, I have been unable to find a solution.</div><div><br></div><div>Respectfully,</div>
<div>Adam</div></div>
</blockquote></div></div></div><br><br clear="all"><br>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>
</div>
</blockquote></div><br>
</div></div></blockquote></div></div></div><div><div></div><div class="h5"><br><br clear="all"><br>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>
</div></div></blockquote></div><br>