[petsc-users] VecAssembly gets stuck

[José Lorenzo]

I'm using PETSc 3.10.2, I guess it is the master branch but I do not know
for sure as I didn't install it myself.

You are right, each processor provides data for all the boundary entries.

I have carried out a few more tests and apparently it gets stuck during
VecAssemblyBegin. I don't know whether I can be more preciseabout this.

El vie., 13 sept. 2019 a las 20:14, Smith, Barry F. (<bsmith at mcs.anl.gov>)
escribió:

>
>   What version of PETSc is this? The master branch?
>
> > call VecSetValues(H, nedge_all, edgappglo(diredg_app) - 1, Hself,
> ADD_VALUES, ierr)
>
> So each process is providing all data for all the boundary entries in the
> vector?
>
> I don't think there is anything wrong with what you are doing but the
> mechanism that does the communication inside the VecAssembly cannot know
> about the structure of the communication and so will do it inefficiently.
> It would be useful to know where it is "stuck" that might help us improve
> the assembly process for your case.
>
> But I think it would better for you to just use MPI directly to put the
> data where it is needed.
>
>
> > On Sep 13, 2019, at 11:30 AM, José Lorenzo via petsc-users <
> petsc-users at mcs.anl.gov> wrote:
> >
> > Hello,
> >
> > I am solving a finite element problem with Dirichlet boundary conditions
> using PETSC. In the boundary conditions there are two terms: a first one
> that is known before hand (normally zero) and a second term that depends
> linearly on the unknown variable itself in the whole domain. Therefore, at
> every time step I need to iterate as the boundary condition depends on the
> field and the latter depends on the BC. Moreover, the problem is nonlinear
> and I use a ghosted vector to represent the field.
> >
> > Every processor manages a portion of the domain and a portion of the
> boundary (if not interior). At every Newton iteration within the time loop
> the way I set the boundary conditions is as follows:
> >
> > First, each processor computes the known term of the BC (first term) and
> inserts the values into the vector
> >
> > call VecSetValues(H, nedge_own, edglocglo(diredg_loc) - 1, Hdir,
> INSERT_VALUES, ierr)
> > call VecAssemblyBegin(H, ierr)
> > call VecAssemblyEnd(H, ierr)
> >
> > As far as I understand, at this stage VecAssembly will not need to
> communicate to other processors as each processor only sets values to
> components that belong to it.
> >
> > Then, each processor computes its own contribution to the
> field-dependent term of the BC for the whole domain boundary as
> >
> > call VecSetValues(H, nedge_all, edgappglo(diredg_app) - 1, Hself,
> ADD_VALUES, ierr)
> > call VecAssemblyBegin(H, ierr)
> > call VecAssemblyEnd(H, ierr)
> >
> > In this case communication will be needed as each processor will add
> values to vector components that are not stored by it, and I guess it might
> get very busy as all the processors will need to communicate with each
> other.
> >
> > When using this strategy I don't find any issue for problems using a
> small amount of processors, but recently I've been solving using 90
> processors and the simulation always hangs at the second VecSetValues at
> some random time step. It works fine for some time steps but at some point
> it just gets stuck and I have to cancel the simulation.
> >
> > I have managed to overcome this by making each processor contribute to
> its own components using first MPI_Reduce and then doing
> >
> > call VecSetValues(H, nedge_own, edgappglo(diredg_app_loc), Hself_own,
> ADD_VALUES, ierr)
> > call VecAssemblyBegin(H, ierr)
> > call VecAssemblyEnd(H, ierr)
> >
> > However I would like to understand whether there is something wrong in
> the code above.
> >
> > Thank you.
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190913/cf089882/attachment.html>