<div dir="ltr"><div dir="ltr">On Fri, Jan 10, 2020 at 7:23 AM Lukas Razinkovas <<a href="mailto:lukasrazinkovas@gmail.com">lukasrazinkovas@gmail.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Thank you very much! <br><div><br></div><div>I already checked that its MPIAIJ matrix and for size I use MatGetInfo routine.</div><div>You are right. With matrix of dim 100000x10000 i get sizes:</div><div><br></div><div>- serial: 5.603 Mb</div><div>- 4 proc.: 7.626 Mb</div><div>- 36 proc.: 7.834 Mb</div><div><br></div><div>That looks fine to me. Thank you again for such quick response.</div><div><br></div><div>I am really impressed with python interface to petsc and slepc. </div><div>I think it is missing detailed documentation and that discouraged me to use it initially,</div><div>so I was writing C code and then wrapping it with python. I am still confused, </div><div>how for example to set MUMPS parameters from python code, but that is different topic.</div></div></blockquote><div><br></div><div>We would discourage you from setting parameters in the code, and rather use the command line interface to MUMPS parameters.</div><div>However, you can put that in the code itself using PetscOptionSetValue().</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Lukas</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jan 10, 2020 at 5:21 PM Smith, Barry F. <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
Yes, with, for example, MATMPAIJ, the matrix entries are distributed among the processes; first verify that you are using a MPI matrix, not Seq, since Seq will keep an entire copy on each process.<br>
<br>
But the parallel matrices do come with some overhead for meta data. So for small matrices like yours it can seem the memory grows unrealistically. Try a much bigger matrix, say 100 times as big and look at the memory usage then. You should see that the meta data is now a much smaller percentage of the memory usage. <br>
<br>
Also be careful if you use top or other such tools for determining memory usage; since malloced() is often not returned to the OS, the can indicate much higher memory usage than is really taking place. You can run PETSc with -log_view -log_view_memory to get a good idea of where PETSc is allocating memory and how much<br>
<br>
Barry<br>
<br>
<br>
> On Jan 10, 2020, at 7:52 AM, Lukas Razinkovas <<a href="mailto:lukasrazinkovas@gmail.com" target="_blank">lukasrazinkovas@gmail.com</a>> wrote:<br>
> <br>
> Hello,<br>
> <br>
> I am trying to use petsc4py and slepc4py for parallel sparse matrix diagonalization.<br>
> However I am a bit confused about matrix size increase when I switch from single processor to multiple processors. For example 100 x 100 matrix with 298 nonzero elements consumes <br>
> 8820 bytes of memory (mat.getInfo()["memory"]), however on two processes it consumes 20552 bytes of memory and on four 33528. My matrix is taken from the slepc4py/demo/ex1.py,<br>
> where nonzero elements are on three diagonals.<br>
> <br>
> Why memory usage increases with MPI processes number?<br>
> I thought that each process stores its own rows and it should stay the same. Or some elements are stored globally?<br>
> <br>
> Lukas<br>
<br>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>