[petsc-users] MatAssemblyBegin freezes during MPI communication

[Junchao Zhang]

On Thu, Jan 18, 2024 at 1:47 AM 袁煕 <yuanxi at advancesoft.jp> wrote:

> Dear PETSc Experts,
>
> My FEM program works well generally, but in some specific cases with
> multiple CPUs are used, it freezes when calling MatAssemblyBegin where
> PMPI_Allreduce is called (see attached file).
>
> After some investigation, I found that it is most probably due to
>
> ・ MatSetValue is not called from all CPUs before MatAssemblyBegin
>
> For example, when 4 CPUs are used, if there are elements in CPU 0,1,2 but
> no elements in CPU 3, then all CPUs other than CPU 3 would call
> MatSetValue  function. I want to know
>
> 1. If my conjecture could be right? And If so
>
No.  All processes do MPI_Allreduce to know if there are incoming values
set by others.  To know why hanging, you can attach gdb to all MPI
processes to see where they are.

>
>
2. Are there any convenient means to avoid this problem?
>
> Thanks,
> Xi YUAN, PhD Solid Mechanics
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20240118/e4a02e03/attachment.html>