<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<div>
<div id="x_compose-container" itemscope="" itemtype="https://schema.org/EmailMessage" style="direction:ltr">
<span itemprop="creator" itemscope="" itemtype="https://schema.org/Organization"><span itemprop="name"></span></span>
<div>
<div style="direction:ltr">I think this problem arose because I did not reset the ksp for solving a different problem! It's not giving me an error anymore now that I added the reset, so it's all good :)</div>
<div><br>
</div>
<div style="direction:ltr">Thanks,</div>
<div style="direction:ltr">Yuyun</div>
<div><br>
</div>
<div class="x_acompli_signature">Get <a href="https://aka.ms/o0ukef">Outlook for iOS</a></div>
</div>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Smith, Barry F. <bsmith@mcs.anl.gov><br>
<b>Sent:</b> Thursday, April 11, 2019 9:21:11 PM<br>
<b>To:</b> Yuyun Yang<br>
<b>Cc:</b> petsc-users@mcs.anl.gov<br>
<b>Subject:</b> Re: [petsc-users] Question about KSP, and makefile linking MPICH</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:11pt;">
<div class="PlainText"><br>
Ahh, I just realized one other thing we can try. Run the program that crashes with -ksp_mat_view binary this will produce a file called binaryoutput, send that file to petsc-maint@mcs.anl.gov and we'll see if we can get MUMPS to mis-behave with it also.<br>
<br>
Barry<br>
<br>
<br>
<br>
> On Apr 11, 2019, at 11:17 PM, Yuyun Yang <yyang85@stanford.edu> wrote:<br>
> <br>
> Thanks Barry for the detailed answers!<br>
> <br>
> Regarding the problem with valgrind, this is the only error produced, and if I allow it to run further, the program would break (at a later function I get NaN for some of the values being calculated, and I've put an assert to prevent NaN results). I will
take a look at it in the debugger. This is for testing, but for bigger problems I won't end up using Cholesky, so it's not really a big issue.<br>
> <br>
> Thanks again for the timely help!<br>
> Yuyun<br>
> <br>
> Get Outlook for iOS<br>
> From: Smith, Barry F. <bsmith@mcs.anl.gov><br>
> Sent: Thursday, April 11, 2019 6:44:54 PM<br>
> To: Yuyun Yang<br>
> Cc: petsc-users@mcs.anl.gov<br>
> Subject: Re: [petsc-users] Question about KSP, and makefile linking MPICH<br>
> <br>
> <br>
> <br>
> > On Apr 11, 2019, at 5:44 PM, Yuyun Yang via petsc-users <petsc-users@mcs.anl.gov> wrote:<br>
> > <br>
> > Hello team,<br>
> > <br>
> > I’d like to check if it’s ok to use the same ksp object and change its operator (the matrix A) later on in the code to solve a different problem?<br>
> <br>
> Do you mean call KSPSetOperators() with one matrix and then later call it with a different matrix? This is ok if the two matrices are the same size and have the same parallel layout. But if the matrices are different size, have different parallel layout
then you need to destroy the KSP and create a new one or call KSPReset() in between for example<br>
> <br>
> KSPSetFromOptions(ksp);<br>
> KSPSetOperators(ksp,A,A);<br>
> KSPSolve(ksp,b,x); <br>
> KSPReset(ksp);<br>
> KSPSetOperators(ksp,B,B);<br>
> KSPSolve(ksp,newb,newx);<br>
> <br>
> > <br>
> > Also, I know I’ve asked this before about linking to MPICH when I call mpirun, instead of using my computer’s default MPI, but I want to check again. The same problem was solved on my cluster by using a different CLINKER (called mpiicc) in the Makefile
and a different intel compiler, which will link my compiled code with MPICH. Is there a similar thing I can do on my own computer, instead of having to use a very long path to locate the MPICH I configured with PETSc, and then calling the executable? (I tried
making CLINKER = mpiicc on my own computer but that didn’t work.)<br>
> <br>
> Are you asking how you can avoid something like<br>
> <br>
> /home/me/petsc/arch-myarch/bin/mpiexec -n 2 ./mycode ?<br>
> <br>
> You can add /home/me/petsc/arch-myarch/bin to the beginning of your PATH, for example with bash put the following into your ~/.bashrc file<br>
> <br>
> export PATH=/home/me/petsc/arch-myarch/bin:$PATH<br>
> mpiexec -n 2 ./mycode<br>
> <br>
> > <br>
> > The final question is related to valgrind. I have defined a setupKSP function to do all the solver/pc setup. It seems like there is a problem with memory allocation but I don’t really understand why. This only happens for MUMPSCHOLESKY though (running CG,
AMG etc. was fine):<br>
> > <br>
> > ==830== Invalid read of size 8<br>
> > ==830== at 0x6977C95: dmumps_ana_o_ (dana_aux.F:2054)<br>
> > ==830== by 0x6913B5A: dmumps_ana_driver_ (dana_driver.F:390)<br>
> > ==830== by 0x68C152C: dmumps_ (dmumps_driver.F:1213)<br>
> > ==830== by 0x68BBE1C: dmumps_f77_ (dmumps_f77.F:267)<br>
> > ==830== by 0x68BA4EB: dmumps_c (mumps_c.c:417)<br>
> > ==830== by 0x5A070D6: MatCholeskyFactorSymbolic_MUMPS (mumps.c:1654)<br>
> > ==830== by 0x54071F2: MatCholeskyFactorSymbolic (matrix.c:3179)<br>
> > ==830== by 0x614AFE9: PCSetUp_Cholesky (cholesky.c:88)<br>
> > ==830== by 0x62BA574: PCSetUp (precon.c:932)<br>
> > ==830== by 0x640BB29: KSPSetUp (itfunc.c:391)<br>
> > ==830== by 0x4A1192: PressureEq::setupKSP(_p_KSP*&, _p_PC*&, _p_Mat*&) (pressureEq.cpp:834)<br>
> > ==830== by 0x4A1258: PressureEq::computeInitialSteadyStatePressure(Domain&) (pressureEq.cpp:862)<br>
> > <br>
> > ==830== Address 0xb8149c0 is 0 bytes after a block of size 7,872 alloc'd<br>
> > <br>
> > ==830== at 0x4C2FFC6: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)<br>
> > ==830== by 0x500E7E0: PetscMallocAlign (mal.c:41)<br>
> > ==830== by 0x59F8A16: MatConvertToTriples_seqaij_seqsbaij (mumps.c:402)<br>
> > ==830== by 0x5A06B53: MatCholeskyFactorSymbolic_MUMPS (mumps.c:1618)<br>
> > ==830== by 0x54071F2: MatCholeskyFactorSymbolic (matrix.c:3179)<br>
> > ==830== by 0x614AFE9: PCSetUp_Cholesky (cholesky.c:88)<br>
> > ==830== by 0x62BA574: PCSetUp (precon.c:932)<br>
> > ==830== by 0x640BB29: KSPSetUp (itfunc.c:391)<br>
> > ==830== by 0x4A1192: PressureEq::setupKSP(_p_KSP*&, _p_PC*&, _p_Mat*&) (pressureEq.cpp:834)<br>
> > ==830== by 0x4A1258: PressureEq::computeInitialSteadyStatePressure(Domain&) (pressureEq.cpp:862)<br>
> > ==830== by 0x49B809: PressureEq::PressureEq(Domain&) (pressureEq.cpp:62)<br>
> > ==830== by 0x4A88E9: StrikeSlip_LinearElastic_qd::StrikeSlip_LinearElastic_qd(Domain&) (strikeSlip_linearElastic_qd.cpp:57)<br>
> <br>
> This is curious. The line in the MUMPS code where valgrind detects a problem is
<br>
> <br>
> K = 1_8<br>
> THEMIN = ZERO<br>
> DO<br>
> IF(THEMIN .NE. ZERO) EXIT<br>
> THEMIN = abs(id%A(K)) <<<<<<< this line<br>
> K = K+1_8<br>
> <br>
> So it has a problem accessing id%A(1) the very first entry in numerical values of the sparse matrix. Meanwhile it states
<br>
> > 0 bytes after a block of size 7,872 alloc'd MatConvertToTriples_seqaij_seqsbaij (mumps.c:402) which is where PETSc allocates<br>
> the values passed to MUMPS. So it almost as if MUMPS never allocated any space for id%A(), I can't imagine why that would ever happen (the<br>
> problem size is super small so its not like it might have run out of memory)<br>
> <br>
> What happens if you allow the valgrind to continue? Do you get more valgrind errors?<br>
> <br>
> What happens if run without valgrind? Does it crash at this point in the code? At some later point? Does it run to completion and seem to
<br>
> produce the correct answer? If it crashes, you could run it in the debugger and when it crashes print the value of id, id%A etc and see if they look<br>
> reasonable. <br>
> <br>
> Barry<br>
> <br>
> <br>
> <br>
> <br>
> > <br>
> > Thank you!<br>
> > Yuyun<br>
<br>
</div>
</span></font>
</body>
</html>