<div dir="ltr"><div class="gmail_default" style="font-family:trebuchet ms,sans-serif;font-size:small"><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Dear PETSc developers,</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"> </p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b>I have used KSP solver to solve a linear equation system in parallel with 5 processors.</b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"> </p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b>The linear system consists of a pressure matrix (pressure_coeff_mat) and its RHS (p_coeff_mat) whose values are set every time step using MatSetValue and VecSetValues functions and then assembled using:</b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   call MatAssemblyBegin(pressure_coeff_mat,mat_final_assembly,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   call MatAssemblyEnd(pressure_coeff_mat,mat_final_assembly,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   call VecAssemblyBegin(p_coeff_rhs,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   call VecAssemblyEnd(p_coeff_rhs,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"> </p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"> </p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b>One processor solves this linear system using KSBCGS Krylov solver with ILU preconditioner, and the result is scattered to all processors</b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">  call KSPGetPC(petsc_ksp,petsc_pc,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   if (preconditioner==0) then</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">      call PCSetType(petsc_pc,PCJACOBI,petsc_err</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">      call PetscOptionsSetValue(PETSC_NULL_OPTIONS,"sub_ksp_type","preonly",petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">      call PetscOptionsSetValue(PETSC_NULL_OPTIONS,"sub_pc_type","ilu",petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">      call PCSetFromOptions(petsc_pc,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   else</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">      call PCSetType(petsc_pc,PCBJACOBI,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   end if</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   call KSPSetFromOptions(petsc_ksp,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   call KSPSetUp(petsc_ksp,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   call KSPSolve(petsc_ksp,p_coeff_rhs,petsc_solution,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   call VecScatterCreateToAll(petsc_solution,ctx,sol_to_all,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   call VecScatterBegin(ctx,petsc_solution,sol_to_all,INSERT_VALUES,SCATTER_FORWARD,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   call VecScatterEnd(ctx,petsc_solution,sol_to_all, INSERT_VALUES,SCATTER_FORWARD,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   call VecGetValues(sol_to_all,ncells, loc_id_to_global-1,sol, petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   call VecScatterDestroy(ctx,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif">   call VecDestroy(sol_to_all,petsc_err)</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"> </p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"> </p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b>However, I got the following errors with vector scattering, i.e., VecScatterBegin, VecScatterEnd, and also VecGetValues. I have checked the pressure matrix, its RHS, and the solution for the first call of KSPSolve. Their values seem to be okay (see attached file). So, could you please tell me what does this error mean and how can I fix it.</b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b> </b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b>Thank you,</b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"> </p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b>Error message:</b></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">-----------------------------------------------------------------------------</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Configure options --with-cc="win32fe cl" --with-fc="win32fe ifort" --with-cxx="win32fe cl" --with-blaslapack-lib="[/cygdrive/c/PROGRA~2/Intel/COMPOS~1/mkl/lib/ia32/mkl_intel_c_dll.lib,/cygdrive/c/PROGRA~2/Intel/COMPOS~1/mkl/lib/ia32/mkl_sequential_dll.lib,/cygdrive/c/PROGRA~2/Intel/COMPOS~1/mkl/lib/ia32/mkl_core_dll.lib]" --with-mpi-lib="[/cygdrive/c/PROGRA~2/MICROS~1/MPI/Lib/x86/msmpifec.lib,/cygdrive/c/PROGRA~2/MICROS~1/MPI/Lib/x86/msmpi.lib]" --with-mpi-include="[/cygdrive/c/PROGRA~2/MICROS~1/MPI/Include,/cygdrive/c/PROGRA~2/MICROS~1/MPI/Include/x86]" --with-mpiexec=/cygdrive/c/PROGRA~1/MI564C~1/bin/mpiexec.exe --with-debugging=1 --with-shared-libraries=0 --CFLAGS=-Od --FFLAGS=-Od CXXFLAGS=-Od --with-clean=1</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">#4930 VecScatterBegin() line 83 in C:\petsc\PE4D24~1.13-\src\vec\vscat\INTERF~1\vscatfce.c</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[1]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[2]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[3]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[0]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[4]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">#4930 VecScatterBegin() line 83 in C:\petsc\PE4D24~1.13-\src\vec\vscat\INTERF~1\vscatfce.c</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">#4930 VecScatterBegin() line 83 in C:\petsc\PE4D24~1.13-\src\vec\vscat\INTERF~1\vscatfce.c</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">#4930 VecScatterBegin() line 83 in C:\petsc\PE4D24~1.13-\src\vec\vscat\INTERF~1\vscatfce.c</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">#4930 VecScatterBegin() line 83 in C:\petsc\PE4D24~1.13-\src\vec\vscat\INTERF~1\vscatfce.c</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">--------------------- Error Message --------------------------------------------------------------</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[1]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[2]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[3]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[0]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[4]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">--------------------- Error Message --------------------------------------------------------------</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">--------------------- Error Message --------------------------------------------------------------</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">--------------------- Error Message --------------------------------------------------------------</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">--------------------- Error Message --------------------------------------------------------------</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null argument, when expecting valid pointer</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[1]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[2]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[3]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[0]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[4]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null argument, when expecting valid pointer</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null argument, when expecting valid pointer</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null argument, when expecting valid pointer</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null argument, when expecting valid pointer</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null Object: Parameter # 3</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[1]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[2]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[3]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[0]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">[4]PETSC ERROR:</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null Object: Parameter # 3</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null Object: Parameter # 3</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null Object: Parameter # 3</p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif">Null Object: Parameter # 3</p></div><div><br></div>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">Hoang Nhan Hien, Ph.D.<div><br></div><div>1) Nuclear Safety Department, Institute for Nuclear Science and Technology, Vietnam Atomic Energy Insitute</div><div><br></div><div>2) Nuclear Safety Analysis Method Development, Enviroment & Energy Technology, Inc.</div><div><br></div><div>Email: <a href="mailto:hoangnhanhien@gmail.com" target="_blank">hoangnhanhien@gmail.com</a>; <a href="mailto:hnhien@kaeri.re.kr" target="_blank">h</a><a href="mailto:ien@en2t.com" target="_blank">ien@en2t.com</a><br></div><div>Phone: +81-10-2635-2411</div><div><br></div></div></div></div></div></div>