<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /></head><body style='font-size: 10pt; font-family: Verdana,Geneva,sans-serif'>
<p>Hi,</p>
<p>So I'm loading two matrices from files, both 1000000 by 10000000. I ran the program with -mat_view::ascii_info and I got:</p>
<p>Mat Object: 1 MPI processes<br /> type: seqaij<br /> rows=1000000, cols=1000000<br /> total: nonzeros=7000000, allocated nonzeros=7000000<br /> total number of mallocs used during MatSetValues calls =0<br /> not using I-node routines</p>
<p>20 times, and then</p>
<p>Mat Object: 1 MPI processes<br /> type: seqaij<br /> rows=1000000, cols=1000000<br /> total: nonzeros=1000000, allocated nonzeros=1000000<br /> total number of mallocs used during MatSetValues calls =0<br /> not using I-node routines</p>
<p>20 times as well, and then</p>
<p>Mat Object: 1 MPI processes<br /> type: seqaij<br /> rows=1000000, cols=1000000<br /> total: nonzeros=7000000, allocated nonzeros=7000000<br /> total number of mallocs used during MatSetValues calls =0<br /> not using I-node routines</p>
<p>20 times as well before crashing.</p>
<p>I realized it might be because I am setting up 20 krylov schur partitions which may be too much. I tried running the code again with only 2 partitions and now the code runs but I have speed issues.</p>
<p>I have one version of the code where my first matrix has 5 non-zero diagonals (so 5000000 non-zero entries), and the set up time is quite fast (8 seconds) and solving is also quite fast. The second version is the same but I have two extra non-zero diagonals (7000000 non-zero entries) and the set up time is a lot slower (2900 seconds ~ 50 minutes) and solving is also a lot slower. Is it normal that adding two extra diagonals increases solve and set up time so much?</p>
<p>Thanks again,</p>
<p>Best regards,</p>
<p>Perceval,</p>
<p><br /></p>
<p><br /></p>
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><!-- html ignored --><!-- head ignored --><!-- meta ignored -->
<div class="pre" style="margin: 0; padding: 0; font-family: monospace">Then I guess it is the factorization that is failing. How many nonzero entries do you have? Run with<br /> -mat_view ::ascii_info<br /> <br /> Jose<br /> <br /> <br />
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0">El 22 nov 2019, a las 19:56, Perceval Desforges <<a href="mailto:perceval.desforges@polytechnique.edu">perceval.desforges@polytechnique.edu</a>> escribió:<br /> <br /> Hi,<br /> <br /> Thanks for your answer. I tried looking at the inertias before solving, but the problem is that the program crashes when I call EPSSetUp with this error:<br /> <br /> slurmstepd: error: Step 2140.0 exceeded virtual memory limit (313526508 > 107317760), being killed<br /> <br /> I get this error even when there are no eigenvalues in the interval.<br /> <br /> I've started using BVMAT instead of BVVECS by the way.<br /> <br /> Thanks,<br /> <br /> Perceval,<br /> <br /> <br /> <br /> <br /> <br />
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0">Don't use -mat_mumps_icntl_14 to reduce the memory used by MUMPS.<br /> <br /> Most likely the problem is that the interval you gave is too large and contains too many eigenvalues (SLEPc needs to allocate at least one vector per each eigenvalue). You can count the eigenvalues in the interval with the inertias, which are available at EPSSetUp (no need to call EPSSolve). See this example:<br /> <a href="http://slepc.upv.es/documentation/current/src/eps/examples/tutorials/ex25.c.html" target="_blank" rel="noopener noreferrer">http://slepc.upv.es/documentation/current/src/eps/examples/tutorials/ex25.c.html</a><br /> You can comment out the call to EPSSolve() and run with the option -show_inertias<br /> For example, the output<br /> Shift 0.1 Inertia 3 <br /> Shift 0.35 Inertia 11 <br /> means that the interval [0.1,0.35] contains 8 eigenvalues (=11-3).<br /> <br /> By the way, I would suggest using BVMAT instead of BVVECS (the latter is slower).<br /> <br /> Jose<br /> <br /> <br />
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0">El 21 nov 2019, a las 18:13, Perceval Desforges via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> escribió:<br /> <br /> Hello all,<br /> <br /> I am trying to obtain all the eigenvalues in a certain interval for a fairly large matrix (1000000 * 1000000). I therefore use the spectrum slicing method detailed in section 3.4.5 of the manual. The calculations are run on a processor with 20 cores and 96 Go of RAM.<br /> <br /> The options I use are :<br /> <br /> -bv_type vecs -eps_krylovschur_detect_zeros 1 -mat_mumps_icntl_13 1 -mat_mumps_icntl_24 1 -mat_mumps_cntl_3 1e-12<br /> <br /> <br /> <br /> However the program quickly crashes with this error:<br /> <br /> slurmstepd: error: Step 2115.0 exceeded virtual memory limit (312121084 > 107317760), being killed<br /> <br /> I've tried reducing the amount of memory used by slepc with the -mat_mumps_icntl_14 option by setting it at -70 for example but then I get this error:<br /> <br /> [1]PETSC ERROR: Error in external library<br /> [1]PETSC ERROR: Error reported by MUMPS in numerical factorization phase: INFOG(1)=-9, INFO(2)=82733614<br /> <br /> which is an error due to setting the mumps icntl option so low from what I've gathered.<br /> <br /> Is there any other way I can reduce memory usage?<br /> <br /> <br /> <br /> Thanks,<br /> <br /> Regards,<br /> <br /> Perceval,<br /> <br /> <br /> <br /> P.S. I sent the same email a few minutes ago but I think I made a mistake in the address, I'm sorry if I've sent it twice.</blockquote>
</blockquote>
<br /> </blockquote>
</div>
</blockquote>
<p><br /></p>
</body></html>