<div dir="ltr">Hi Hong,<div><br></div><div>You can find the matrices here:</div><div><div><a href="https://www.dropbox.com/s/ejpa9owkv8tjnwi/A.petsc?dl=0" target="_blank">https://www.dropbox.com/s/ejpa9owkv8tjnwi/A.petsc?dl=0</a><br></div><div><a href="https://www.dropbox.com/s/urjtxaezl0cv3om/B.petsc?dl=0" target="_blank">https://www.dropbox.com/s/urjtxaezl0cv3om/B.petsc?dl=0</a><br></div><br class="gmail-Apple-interchange-newline"></div><div>Changing the target value leads to the same error. What is strange is that this works without a problem on two other machines. But in my main workstation (the one I use for developing and testing) it fails :(</div><div><br></div><div>Thanks so much for your help!</div><div>Santiago </div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, Oct 31, 2018 at 2:48 AM Zhang, Hong <<a href="mailto:hzhang@mcs.anl.gov">hzhang@mcs.anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div dir="ltr">
<div><span style="color:rgb(136,136,136)">Santiago,</span><br>
</div>
The shift '-eps_target -2e-3+1.01i' is very close to the eigenvalues. What happens if you pick a target little away from your eigenvalues?
<div>I suspect mumps encounters a zero pivot during numerical factorization. There are options to handle it, but I need matrices A and B to investigate. </div>
<div>I am not sure if the problem comes from memory bug.</div>
<div>Anyway, I'm cc'ing mumps developers here.</div>
<div><br>
</div>
<div>Hong<br>
<br>
<div class="gmail_quote">
<div dir="ltr">On Tue, Oct 30, 2018 at 8:09 PM Smith, Barry F. via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Yeah this doesn't look good for MUMPS but isn't for sure the problem either. <br>
<br>
The valgrind output should be sent to the MUMPS developers.<br>
<br>
Hong, <br>
<br>
Can you send this to the MUMPS developers and see what they say?<br>
<br>
Thanks<br>
<br>
Barry<br>
<br>
<br>
> On Oct 30, 2018, at 2:04 PM, Santiago Andres Triana <<a href="mailto:repepo@gmail.com" target="_blank">repepo@gmail.com</a>> wrote:<br>
> <br>
> This is the output of<br>
> mpiexec -n 2 valgrind --tool=memcheck -q --num-callers=20 --log-file=valgrind.log.%p ./ex7 -malloc off -f1 A.petsc -f2 B.petsc -eps_nev 4 -eps_target -2e-3+1.01i -st_type sinvert<br>
> <br>
> Generalized eigenproblem stored in file.<br>
> <br>
> Reading COMPLEX matrices from binary files...<br>
> [1]PETSC ERROR: ------------------------------------------------------------------------<br>
> [1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range<br>
> [1]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger<br>
> [1]PETSC ERROR: or see <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind" rel="noreferrer" target="_blank">
http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a><br>
> [1]PETSC ERROR: or try <a href="http://valgrind.org" rel="noreferrer" target="_blank">
http://valgrind.org</a> on GNU/linux and Apple Mac OS X to find memory corruption errors<br>
> [1]PETSC ERROR: likely location of problem given in stack below<br>
> [1]PETSC ERROR: --------------------- Stack Frames ------------------------------------<br>
> [1]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,<br>
> [1]PETSC ERROR: INSTEAD the line number of the start of the function<br>
> [1]PETSC ERROR: is given.<br>
> [1]PETSC ERROR: [1] MatFactorNumeric_MUMPS line 1205 /home/spin2/petsc-3.10.2/src/mat/impls/aij/mpi/mumps/mumps.c<br>
> [1]PETSC ERROR: [1] MatLUFactorNumeric line 3054 /home/spin2/petsc-3.10.2/src/mat/interface/matrix.c<br>
> [1]PETSC ERROR: [1] PCSetUp_LU line 59 /home/spin2/petsc-3.10.2/src/ksp/pc/impls/factor/lu/lu.c<br>
> [1]PETSC ERROR: [1] PCSetUp line 894 /home/spin2/petsc-3.10.2/src/ksp/pc/interface/precon.c<br>
> [1]PETSC ERROR: [1] KSPSetUp line 304 /home/spin2/petsc-3.10.2/src/ksp/ksp/interface/itfunc.c<br>
> [1]PETSC ERROR: [1] STSetUp_Sinvert line 96 /home/spin2/slepc-3.10.1/src/sys/classes/st/impls/sinvert/sinvert.c<br>
> [1]PETSC ERROR: [1] STSetUp line 233 /home/spin2/slepc-3.10.1/src/sys/classes/st/interface/stsolve.c<br>
> [1]PETSC ERROR: [1] EPSSetUp line 104 /home/spin2/slepc-3.10.1/src/eps/interface/epssetup.c<br>
> [1]PETSC ERROR: [1] EPSSolve line 129 /home/spin2/slepc-3.10.1/src/eps/interface/epssolve.c<br>
> [1]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>
> [1]PETSC ERROR: Signal received<br>
> [1]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" rel="noreferrer" target="_blank">
http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>
> [1]PETSC ERROR: Petsc Release Version 3.10.2, Oct, 09, 2018 <br>
> [1]PETSC ERROR: ./ex7 on a arch-linux2-c-opt named wobble-wkst-as by spin2 Tue Oct 30 19:42:18 2018<br>
> [1]PETSC ERROR: Configure options --download-mpich -with-scalar-type=complex --download-mumps --download-parmetis --download-metis --download-scalapack --download-fblaslapack --with-debugging=1 --download-superlu_dist --download-ptscotch<br>
> [1]PETSC ERROR: #1 User provided function() line 0 in unknown file<br>
> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 1<br>
> <br>
> <br>
> <br>
> and one of the two valgrind logs (the other was empty):<br>
> <br>
> ==63004== Use of uninitialised value of size 8<br>
> ==63004== at 0x694F8FF: zmumps_redistribution_ (zfac_distrib_distentry.F:367)<br>
> ==63004== by 0x68E1266: zmumps_fac_driver_ (zfac_driver.F:1777)<br>
> ==63004== by 0x6869F63: zmumps_ (zmumps_driver.F:1686)<br>
> ==63004== by 0x6861B64: zmumps_f77_ (zmumps_f77.F:267)<br>
> ==63004== by 0x685FB43: zmumps_c (mumps_c.c:417)<br>
> ==63004== by 0x5B741CD: MatFactorNumeric_MUMPS (mumps.c:1227)<br>
> ==63004== by 0x53C3DDB: MatLUFactorNumeric (matrix.c:3065)<br>
> ==63004== by 0x626E652: PCSetUp_LU (lu.c:131)<br>
> ==63004== by 0x6387B8D: PCSetUp (precon.c:932)<br>
> ==63004== by 0x649CD41: KSPSetUp (itfunc.c:391)<br>
> ==63004== by 0x4A0E3F7: STSetUp_Sinvert (sinvert.c:132)<br>
> ==63004== by 0x4A4033F: STSetUp (stsolve.c:271)<br>
> ==63004== by 0x4B586FB: EPSSetUp (epssetup.c:263)<br>
> ==63004== by 0x4B5D43A: EPSSolve (epssolve.c:135)<br>
> ==63004== by 0x10B6FD: main (ex7.c:134)<br>
> ==63004== <br>
> ==63004== Invalid read of size 4<br>
> ==63004== at 0x694F8FF: zmumps_redistribution_ (zfac_distrib_distentry.F:367)<br>
> ==63004== by 0x68E1266: zmumps_fac_driver_ (zfac_driver.F:1777)<br>
> ==63004== by 0x6869F63: zmumps_ (zmumps_driver.F:1686)<br>
> ==63004== by 0x6861B64: zmumps_f77_ (zmumps_f77.F:267)<br>
> ==63004== by 0x685FB43: zmumps_c (mumps_c.c:417)<br>
> ==63004== by 0x5B741CD: MatFactorNumeric_MUMPS (mumps.c:1227)<br>
> ==63004== by 0x53C3DDB: MatLUFactorNumeric (matrix.c:3065)<br>
> ==63004== by 0x626E652: PCSetUp_LU (lu.c:131)<br>
> ==63004== by 0x6387B8D: PCSetUp (precon.c:932)<br>
> ==63004== by 0x649CD41: KSPSetUp (itfunc.c:391)<br>
> ==63004== by 0x4A0E3F7: STSetUp_Sinvert (sinvert.c:132)<br>
> ==63004== by 0x4A4033F: STSetUp (stsolve.c:271)<br>
> ==63004== by 0x4B586FB: EPSSetUp (epssetup.c:263)<br>
> ==63004== by 0x4B5D43A: EPSSolve (epssolve.c:135)<br>
> ==63004== by 0x10B6FD: main (ex7.c:134)<br>
> ==63004== Address 0xe5ffda3101296ca0 is not stack'd, malloc'd or (recently) free'd<br>
> ==63004== <br>
> <br>
> <br>
> Hope it gives enough info. Thanks!<br>
> <br>
> <br>
> <br>
> On Tue, Oct 30, 2018 at 6:50 PM Smith, Barry F. <<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>> wrote:<br>
> <br>
> Can you run the code on the "failing" machine using valgrind? <a href="https://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind" rel="noreferrer" target="_blank">
https://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a><br>
> <br>
> Barry<br>
> <br>
> <br>
> > On Oct 30, 2018, at 12:10 PM, Santiago Andres Triana via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br>
> > <br>
> > Hi petsc-users,<br>
> > <br>
> > I am solving a generalized eigenvalue problem using ex7 in $SLEPC_DIR/src/eps/examples/tutorials/. I provide the A and B matrices.<br>
> > The program runs fine, with correct solutions on 12-core node and also on a mac laptop.<br>
> > <br>
> > However, on a 16-core workstation running Debian testing (fresh install) and also a fresh install of petsc and slepc I get the following error:<br>
> > <br>
> > $ mpiexec -n 2 ./ex7 -f1 A.petsc -f2 B.petsc -st_type sinvert -eps_nev 4 -eps_target -2e-3+1.01i<br>
> > <br>
> > Generalized eigenproblem stored in file.<br>
> > <br>
> > Reading COMPLEX matrices from binary files...<br>
> > [1]PETSC ERROR: ------------------------------------------------------------------------<br>
> > [1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range<br>
> > [1]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger<br>
> > [1]PETSC ERROR: or see <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind" rel="noreferrer" target="_blank">
http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a><br>
> > [1]PETSC ERROR: or try <a href="http://valgrind.org" rel="noreferrer" target="_blank">
http://valgrind.org</a> on GNU/linux and Apple Mac OS X to find memory corruption errors<br>
> > [1]PETSC ERROR: likely location of problem given in stack below<br>
> > [1]PETSC ERROR: --------------------- Stack Frames ------------------------------------<br>
> > [1]PETSC ERROR: Note: The EXACT line numbers in the stack are not available,<br>
> > [1]PETSC ERROR: INSTEAD the line number of the start of the function<br>
> > [1]PETSC ERROR: is given.<br>
> > [1]PETSC ERROR: [1] MatFactorNumeric_MUMPS line 1205 /home/spin2/petsc-3.10.2/src/mat/impls/aij/mpi/mumps/mumps.c<br>
> > [1]PETSC ERROR: [1] MatLUFactorNumeric line 3054 /home/spin2/petsc-3.10.2/src/mat/interface/matrix.c<br>
> > [1]PETSC ERROR: [1] PCSetUp_LU line 59 /home/spin2/petsc-3.10.2/src/ksp/pc/impls/factor/lu/lu.c<br>
> > [1]PETSC ERROR: [1] PCSetUp line 894 /home/spin2/petsc-3.10.2/src/ksp/pc/interface/precon.c<br>
> > [1]PETSC ERROR: [1] KSPSetUp line 304 /home/spin2/petsc-3.10.2/src/ksp/ksp/interface/itfunc.c<br>
> > [1]PETSC ERROR: [1] STSetUp_Sinvert line 96 /home/spin2/slepc-3.10.1/src/sys/classes/st/impls/sinvert/sinvert.c<br>
> > [1]PETSC ERROR: [1] STSetUp line 233 /home/spin2/slepc-3.10.1/src/sys/classes/st/interface/stsolve.c<br>
> > [1]PETSC ERROR: [1] EPSSetUp line 104 /home/spin2/slepc-3.10.1/src/eps/interface/epssetup.c<br>
> > [1]PETSC ERROR: [1] EPSSolve line 129 /home/spin2/slepc-3.10.1/src/eps/interface/epssolve.c<br>
> > [1]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>
> > [1]PETSC ERROR: Signal received<br>
> > [1]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" rel="noreferrer" target="_blank">
http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>
> > [1]PETSC ERROR: Petsc Release Version 3.10.2, Oct, 09, 2018 <br>
> > [1]PETSC ERROR: ./ex7 on a arch-linux2-c-opt named wobble-wkst-as by spin2 Tue Oct 30 17:40:51 2018<br>
> > [1]PETSC ERROR: Configure options --download-mpich -with-scalar-type=complex --download-mumps --download-parmetis --download-metis --download-scalapack --download-fblaslapack --with-debugging=1 --download-superlu_dist --download-ptscotch<br>
> > [1]PETSC ERROR: #1 User provided function() line 0 in unknown file<br>
> > application called MPI_Abort(MPI_COMM_WORLD, 59) - process 1<br>
> > <br>
> > <br>
> > <br>
> > the expected output is the following (on a compute node running petsc-3.9.2 and also on a mac laptop running petsc-3.10.2):<br>
> > <br>
> > $ mpiexec -n 2 ./ex7 -f1 A.petsc -f2 B.petsc -st_type sinvert -eps_nev 4 -eps_target -2e-3+1.01i<br>
> > <br>
> > Generalized eigenproblem stored in file.<br>
> > <br>
> > Reading COMPLEX matrices from binary files...<br>
> > Number of iterations of the method: 2<br>
> > Number of linear iterations of the method: 27<br>
> > Solution method: krylovschur<br>
> > <br>
> > Number of requested eigenvalues: 4<br>
> > Stopping condition: tol=1e-08, maxit=63157<br>
> > Linear eigensolve converged (4 eigenpairs) due to CONVERGED_TOL; iterations 2<br>
> > ---------------------- --------------------<br>
> > k ||Ax-kBx||/||kx||<br>
> > ---------------------- --------------------<br>
> > -0.002806+1.009827i 2.00821e-19<br>
> > -0.002980+1.008417i 8.08359e-17<br>
> > -0.002676+1.011755i 9.49342e-18<br>
> > -0.003201+1.007367i 1.50869e-16<br>
> > ---------------------- --------------------<br>
> > <br>
> > <br>
> > Just in case, the matrices can be downloaded from here if any one wants to give them a try<br>
> > <a href="https://www.dropbox.com/s/ejpa9owkv8tjnwi/A.petsc?dl=0" rel="noreferrer" target="_blank">
https://www.dropbox.com/s/ejpa9owkv8tjnwi/A.petsc?dl=0</a><br>
> > <a href="https://www.dropbox.com/s/urjtxaezl0cv3om/B.petsc?dl=0" rel="noreferrer" target="_blank">
https://www.dropbox.com/s/urjtxaezl0cv3om/B.petsc?dl=0</a><br>
> > <br>
> > <br>
> > I tried different petsc/slepc versions to no avail, including an OS reinstall. So any help would be highly appreciated. Thanks in advance!<br>
> > <br>
> > Santiago<br>
> <br>
<br>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote></div>