[petsc-users] MatAssemblyEnd taking too long

[Junchao Zhang]

I will have a look and report back to you. Thanks.
--Junchao Zhang


On Thu, Aug 20, 2020 at 5:23 PM Manav Bhatia <bhatiamanav at gmail.com> wrote:

> I have created a standalone test that demonstrates the problem at my end.
> I have stored the indices, etc.  from my problem in a text file for each
> rank, which I use to initialize the matrix.
> Please note that the test is specifically for 8 ranks.
>
> The .tgz file is on my google drive:
> https://drive.google.com/file/d/1R-WjS36av3maXX3pUyiR3ndGAxteTVj-/view?usp=sharing
>
>
> This contains a README file with instructions on running. Please note that
> the work directory needs the index files.
>
> Please let me know if I can provide any further information.
>
> Thank you all for your help.
>
> Regards,
> Manav
>
> On Aug 20, 2020, at 12:54 PM, Jed Brown <jed at jedbrown.org> wrote:
>
> Matthew Knepley <knepley at gmail.com> writes:
>
> On Thu, Aug 20, 2020 at 11:09 AM Manav Bhatia <bhatiamanav at gmail.com>
> wrote:
>
>
>
> On Aug 20, 2020, at 8:31 AM, Stefano Zampini <stefano.zampini at gmail.com>
> wrote:
>
> Can you add a MPI_Barrier before
>
> ierr = MatAssemblyBegin(aij->A,mode);CHKERRQ(ierr);
>
>
> With a MPI_Barrier before this function call:
> —  three of the processes have already hit this barrier,
> —  the other 5 are inside MatStashScatterGetMesg_Private ->
> MatStashScatterGetMesg_BTS -> MPI_Waitsome(2 processes)/MPI_Waitall(3
> processes)
>
>
> This is not itself evidence of inconsistent state.  You can use
>
>  -build_twosided allreduce
>
> to avoid the nonblocking sparse algorithm.
>
>
> Okay, you should run this with -matstash_legacy just to make sure it is not
> a bug in your MPI implementation. But it looks like
> there is inconsistency in the parallel state. This can happen because we
> have a bug, or it could be that you called a collective
> operation on a subset of the processes. Is there any way you could cut down
> the example (say put all 1s in the matrix, etc) so
> that you could give it to us to run?
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200820/6338eb74/attachment-0001.html>