<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<div>
<div id="x_compose-container" itemscope="" itemtype="https://schema.org/EmailMessage" style="direction:ltr">
<span itemprop="creator" itemscope="" itemtype="https://schema.org/Organization"><span itemprop="name"></span></span>
<div>
<div style="direction:ltr">Thanks Barry for the detailed answers!</div>
<div><br>
</div>
<div style="direction:ltr">Regarding the problem with valgrind, this is the only error produced, and if I allow it to run further, the program would break (at a later function I get NaN for some of the values being calculated, and I've put an assert to prevent
NaN results). I will take a look at it in the debugger. This is for testing, but for bigger problems I won't end up using Cholesky, so it's not really a big issue.</div>
<div><br>
</div>
<div style="direction:ltr">Thanks again for the timely help!</div>
<div style="direction:ltr">Yuyun</div>
<div><br>
</div>
<div class="x_acompli_signature">Get <a href="https://aka.ms/o0ukef">Outlook for iOS</a></div>
</div>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Smith, Barry F. <bsmith@mcs.anl.gov><br>
<b>Sent:</b> Thursday, April 11, 2019 6:44:54 PM<br>
<b>To:</b> Yuyun Yang<br>
<b>Cc:</b> petsc-users@mcs.anl.gov<br>
<b>Subject:</b> Re: [petsc-users] Question about KSP, and makefile linking MPICH</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:11pt;">
<div class="PlainText"><br>
<br>
> On Apr 11, 2019, at 5:44 PM, Yuyun Yang via petsc-users <petsc-users@mcs.anl.gov> wrote:<br>
> <br>
> Hello team,<br>
> <br>
> I’d like to check if it’s ok to use the same ksp object and change its operator (the matrix A) later on in the code to solve a different problem?<br>
<br>
Do you mean call KSPSetOperators() with one matrix and then later call it with a different matrix? This is ok if the two matrices are the same size and have the same parallel layout. But if the matrices are different size, have different parallel layout
then you need to destroy the KSP and create a new one or call KSPReset() in between for example<br>
<br>
KSPSetFromOptions(ksp);<br>
KSPSetOperators(ksp,A,A);<br>
KSPSolve(ksp,b,x); <br>
KSPReset(ksp);<br>
KSPSetOperators(ksp,B,B);<br>
KSPSolve(ksp,newb,newx);<br>
<br>
> <br>
> Also, I know I’ve asked this before about linking to MPICH when I call mpirun, instead of using my computer’s default MPI, but I want to check again. The same problem was solved on my cluster by using a different CLINKER (called mpiicc) in the Makefile and
a different intel compiler, which will link my compiled code with MPICH. Is there a similar thing I can do on my own computer, instead of having to use a very long path to locate the MPICH I configured with PETSc, and then calling the executable? (I tried
making CLINKER = mpiicc on my own computer but that didn’t work.)<br>
<br>
Are you asking how you can avoid something like<br>
<br>
/home/me/petsc/arch-myarch/bin/mpiexec -n 2 ./mycode ?<br>
<br>
You can add /home/me/petsc/arch-myarch/bin to the beginning of your PATH, for example with bash put the following into your ~/.bashrc file<br>
<br>
export PATH=/home/me/petsc/arch-myarch/bin:$PATH<br>
mpiexec -n 2 ./mycode<br>
<br>
> <br>
> The final question is related to valgrind. I have defined a setupKSP function to do all the solver/pc setup. It seems like there is a problem with memory allocation but I don’t really understand why. This only happens for MUMPSCHOLESKY though (running CG,
AMG etc. was fine):<br>
> <br>
> ==830== Invalid read of size 8<br>
> ==830== at 0x6977C95: dmumps_ana_o_ (dana_aux.F:2054)<br>
> ==830== by 0x6913B5A: dmumps_ana_driver_ (dana_driver.F:390)<br>
> ==830== by 0x68C152C: dmumps_ (dmumps_driver.F:1213)<br>
> ==830== by 0x68BBE1C: dmumps_f77_ (dmumps_f77.F:267)<br>
> ==830== by 0x68BA4EB: dmumps_c (mumps_c.c:417)<br>
> ==830== by 0x5A070D6: MatCholeskyFactorSymbolic_MUMPS (mumps.c:1654)<br>
> ==830== by 0x54071F2: MatCholeskyFactorSymbolic (matrix.c:3179)<br>
> ==830== by 0x614AFE9: PCSetUp_Cholesky (cholesky.c:88)<br>
> ==830== by 0x62BA574: PCSetUp (precon.c:932)<br>
> ==830== by 0x640BB29: KSPSetUp (itfunc.c:391)<br>
> ==830== by 0x4A1192: PressureEq::setupKSP(_p_KSP*&, _p_PC*&, _p_Mat*&) (pressureEq.cpp:834)<br>
> ==830== by 0x4A1258: PressureEq::computeInitialSteadyStatePressure(Domain&) (pressureEq.cpp:862)<br>
> <br>
> ==830== Address 0xb8149c0 is 0 bytes after a block of size 7,872 alloc'd<br>
> <br>
> ==830== at 0x4C2FFC6: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)<br>
> ==830== by 0x500E7E0: PetscMallocAlign (mal.c:41)<br>
> ==830== by 0x59F8A16: MatConvertToTriples_seqaij_seqsbaij (mumps.c:402)<br>
> ==830== by 0x5A06B53: MatCholeskyFactorSymbolic_MUMPS (mumps.c:1618)<br>
> ==830== by 0x54071F2: MatCholeskyFactorSymbolic (matrix.c:3179)<br>
> ==830== by 0x614AFE9: PCSetUp_Cholesky (cholesky.c:88)<br>
> ==830== by 0x62BA574: PCSetUp (precon.c:932)<br>
> ==830== by 0x640BB29: KSPSetUp (itfunc.c:391)<br>
> ==830== by 0x4A1192: PressureEq::setupKSP(_p_KSP*&, _p_PC*&, _p_Mat*&) (pressureEq.cpp:834)<br>
> ==830== by 0x4A1258: PressureEq::computeInitialSteadyStatePressure(Domain&) (pressureEq.cpp:862)<br>
> ==830== by 0x49B809: PressureEq::PressureEq(Domain&) (pressureEq.cpp:62)<br>
> ==830== by 0x4A88E9: StrikeSlip_LinearElastic_qd::StrikeSlip_LinearElastic_qd(Domain&) (strikeSlip_linearElastic_qd.cpp:57)<br>
<br>
This is curious. The line in the MUMPS code where valgrind detects a problem is
<br>
<br>
K = 1_8<br>
THEMIN = ZERO<br>
DO<br>
IF(THEMIN .NE. ZERO) EXIT<br>
THEMIN = abs(id%A(K)) <<<<<<< this line<br>
K = K+1_8<br>
<br>
So it has a problem accessing id%A(1) the very first entry in numerical values of the sparse matrix. Meanwhile it states
<br>
> 0 bytes after a block of size 7,872 alloc'd MatConvertToTriples_seqaij_seqsbaij (mumps.c:402) which is where PETSc allocates<br>
the values passed to MUMPS. So it almost as if MUMPS never allocated any space for id%A(), I can't imagine why that would ever happen (the<br>
problem size is super small so its not like it might have run out of memory)<br>
<br>
What happens if you allow the valgrind to continue? Do you get more valgrind errors?<br>
<br>
What happens if run without valgrind? Does it crash at this point in the code? At some later point? Does it run to completion and seem to
<br>
produce the correct answer? If it crashes, you could run it in the debugger and when it crashes print the value of id, id%A etc and see if they look<br>
reasonable. <br>
<br>
Barry<br>
<br>
<br>
<br>
<br>
> <br>
> Thank you!<br>
> Yuyun<br>
<br>
</div>
</span></font>
</body>
</html>