[petsc-dev] (no subject)

[Zhang, Junchao]

We should be able to overlap PetscSFReduce. I will have a look. Thanks.
--Junchao Zhang


On Mon, Jul 22, 2019 at 5:23 AM Stefano Zampini <stefano.zampini at gmail.com<mailto:stefano.zampini at gmail.com>> wrote:
Junchao,

I found an issue with PetscSFReduceBegin/End. It seems that we can no longer do (always worked before)

    ierr = PetscSFReduceBegin(sf,MPIU_INT,leaf,root,MPI_MAX);CHKERRQ(ierr);
    ierr = PetscSFReduceBegin(sf,MPIU_INT,leaf2,root2,MPI_MAX);CHKERRQ(ierr);
    ierr = PetscSFReduceEnd(sf,MPIU_INT,leaf,root,MPI_MAX);CHKERRQ(ierr);
    ierr = PetscSFReduceEnd(sf,MPIU_INT,leaf2,root2,MPI_MAX);CHKERRQ(ierr);

You can reproduce with the currrent master
$ cd $PETSC_DIR/src/ksp/ksp/examples/tutorials
$ make ex71
$ mpiexec -n 4  ./ex71 -pde_type Elasticity -cells 7,9 -dim 2 -pc_bddc_levels 1 -pc_bddc_coarsening_ratio 2 -ksp_error_if_not_converged -pc_bddc_monolithic -pc_bddc_use_faces -pc_bddc_coarse_pc_bddc_corner_selection -pc_bddc_coarse_l1_pc_bddc_corner_selection -mat_partitioning_type average  -pc_bddc_coarse_pc_bddc_use_deluxe_scaling -pc_bddc_coarse_sub_schurs_mat_solver_type petsc

The attached patch shows that if we instead do

    ierr = PetscSFReduceBegin(sf,MPIU_INT,leaf,root,MPI_MAX);CHKERRQ(ierr);
    ierr = PetscSFReduceEnd(sf,MPIU_INT,leaf,root,MPI_MAX);CHKERRQ(ierr);
    ierr = PetscSFReduceBegin(sf,MPIU_INT,leaf2,root2,MPI_MAX);CHKERRQ(ierr);
    ierr = PetscSFReduceEnd(sf,MPIU_INT,leaf2,root2,MPI_MAX);CHKERRQ(ierr);

everything works properly.
I think the issue is with the SF GetPack mechanism, but I couldn't figure out where.

--
Stefano
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20190722/8b228640/attachment-0001.html>