<div dir="ltr"><div dir="ltr">On Mon, Nov 2, 2020 at 12:31 AM Hien HN- <<a href="mailto:hoangnhanhien@gmail.com">hoangnhanhien@gmail.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:"trebuchet ms",sans-serif;font-size:small"><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Dear PETSc developers,</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"> </p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b>I have used KSP solver to solve a linear equation system in parallel with 5 processors.</b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"> </p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b>The linear system consists of a pressure matrix (pressure_coeff_mat) and its RHS (p_coeff_mat) whose values are set every time step using MatSetValue and VecSetValues functions and then assembled using:</b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call MatAssemblyBegin(pressure_coeff_mat,mat_final_assembly,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call MatAssemblyEnd(pressure_coeff_mat,mat_final_assembly,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call VecAssemblyBegin(p_coeff_rhs,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call VecAssemblyEnd(p_coeff_rhs,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> </p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"> </p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b>One processor solves this linear system using KSBCGS Krylov solver with ILU preconditioner, and the result is scattered to all processors</b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call KSPGetPC(petsc_ksp,petsc_pc,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> if (preconditioner==0) then</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call PCSetType(petsc_pc,PCJACOBI,petsc_err</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call PetscOptionsSetValue(PETSC_NULL_OPTIONS,"sub_ksp_type","preonly",petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call PetscOptionsSetValue(PETSC_NULL_OPTIONS,"sub_pc_type","ilu",petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call PCSetFromOptions(petsc_pc,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> else</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call PCSetType(petsc_pc,PCBJACOBI,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> end if</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call KSPSetFromOptions(petsc_ksp,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call KSPSetUp(petsc_ksp,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call KSPSolve(petsc_ksp,p_coeff_rhs,petsc_solution,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call VecScatterCreateToAll(petsc_solution,ctx,sol_to_all,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call VecScatterBegin(ctx,petsc_solution,sol_to_all,INSERT_VALUES,SCATTER_FORWARD,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call VecScatterEnd(ctx,petsc_solution,sol_to_all, INSERT_VALUES,SCATTER_FORWARD,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call VecGetValues(sol_to_all,ncells, loc_id_to_global-1,sol, petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call VecScatterDestroy(ctx,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> call VecDestroy(sol_to_all,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"> </p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"> </p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b>However, I got the following errors with vector scattering, i.e., VecScatterBegin, VecScatterEnd, and also VecGetValues. I have checked the pressure matrix, its RHS, and the solution for the first call of KSPSolve. Their values seem to be okay (see attached file). So, could you please tell me what does this error mean and how can I fix it.</b></p></div></div></blockquote><div><br></div><div>1) The error message says that '<span style="font-family:Calibri,sans-serif;font-size:14.6667px">sol_to_all' has not been created</span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px"><br></span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px">2) Is this system the Laplacian? If so, ILU is a bad preconditioner. You should try gamg.</span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px"><br></span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px"> Thanks,</span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px"><br></span></div><div><span style="font-family:Calibri,sans-serif;font-size:14.6667px"> Matt</span></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:"trebuchet ms",sans-serif;font-size:small"><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b style="font-size:11pt"> </b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b>Thank you,</b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"> </p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b>Error message:</b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">-----------------------------------------------------------------------------</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Configure options --with-cc="win32fe cl" --with-fc="win32fe ifort" --with-cxx="win32fe cl" --with-blaslapack-lib="[/cygdrive/c/PROGRA~2/Intel/COMPOS~1/mkl/lib/ia32/mkl_intel_c_dll.lib,/cygdrive/c/PROGRA~2/Intel/COMPOS~1/mkl/lib/ia32/mkl_sequential_dll.lib,/cygdrive/c/PROGRA~2/Intel/COMPOS~1/mkl/lib/ia32/mkl_core_dll.lib]" --with-mpi-lib="[/cygdrive/c/PROGRA~2/MICROS~1/MPI/Lib/x86/msmpifec.lib,/cygdrive/c/PROGRA~2/MICROS~1/MPI/Lib/x86/msmpi.lib]" --with-mpi-include="[/cygdrive/c/PROGRA~2/MICROS~1/MPI/Include,/cygdrive/c/PROGRA~2/MICROS~1/MPI/Include/x86]" --with-mpiexec=/cygdrive/c/PROGRA~1/MI564C~1/bin/mpiexec.exe --with-debugging=1 --with-shared-libraries=0 --CFLAGS=-Od --FFLAGS=-Od CXXFLAGS=-Od --with-clean=1</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">#4930 VecScatterBegin() line 83 in C:\petsc\PE4D24~1.13-\src\vec\vscat\INTERF~1\vscatfce.c</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[1]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[2]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[3]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[0]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[4]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">#4930 VecScatterBegin() line 83 in C:\petsc\PE4D24~1.13-\src\vec\vscat\INTERF~1\vscatfce.c</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">#4930 VecScatterBegin() line 83 in C:\petsc\PE4D24~1.13-\src\vec\vscat\INTERF~1\vscatfce.c</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">#4930 VecScatterBegin() line 83 in C:\petsc\PE4D24~1.13-\src\vec\vscat\INTERF~1\vscatfce.c</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">#4930 VecScatterBegin() line 83 in C:\petsc\PE4D24~1.13-\src\vec\vscat\INTERF~1\vscatfce.c</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">--------------------- Error Message --------------------------------------------------------------</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[1]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[2]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[3]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[0]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[4]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">--------------------- Error Message --------------------------------------------------------------</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">--------------------- Error Message --------------------------------------------------------------</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">--------------------- Error Message --------------------------------------------------------------</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">--------------------- Error Message --------------------------------------------------------------</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null argument, when expecting valid pointer</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[1]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[2]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[3]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[0]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[4]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null argument, when expecting valid pointer</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null argument, when expecting valid pointer</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null argument, when expecting valid pointer</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null argument, when expecting valid pointer</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null Object: Parameter # 3</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[1]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[2]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[3]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[0]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[4]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null Object: Parameter # 3</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null Object: Parameter # 3</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null Object: Parameter # 3</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null Object: Parameter # 3</p></div><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr">Hoang Nhan Hien, Ph.D.<div><br></div><div>1) Nuclear Safety Department, Institute for Nuclear Science and Technology, Vietnam Atomic Energy Insitute</div><div><br></div><div>2) Nuclear Safety Analysis Method Development, Enviroment & Energy Technology, Inc.</div><div><br></div><div>Email: <a href="mailto:hoangnhanhien@gmail.com" target="_blank">hoangnhanhien@gmail.com</a>; <a href="mailto:hnhien@kaeri.re.kr" target="_blank">h</a><a href="mailto:ien@en2t.com" target="_blank">ien@en2t.com</a><br></div><div>Phone: +81-10-2635-2411</div><div><br></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>