[petsc-dev] Expensive VecDuplicate_MPI_DA

[Smith, Barry F.]

   Jed,

     You added these two lines. Any idea why they are needed?

    Thanks

    Barry

3e08d2bebf99 src/dm/impls/da/dadist.c (Barry Smith          2012-09-18  14)   PetscFunctionBegin;
c688c0463f6c src/dm/impls/da/dadist.c (Matthew Knepley      2012-11-11  15)   ierr = VecGetDM(g, &da);CHKERRQ(ierr);
2dcb2ebcaafe src/dm/impls/da/dadist.c (Barry Smith          2010-10-31  16)   ierr = DMCreateGlobalVector(da,gg);CHKERRQ(ierr);
077aedaf8cb5 src/dm/impls/da/dadist.c (Jed Brown            2013-02-11  17)   ierr = VecGetLayout(g,&map);CHKERRQ(ierr);
077aedaf8cb5 src/dm/impls/da/dadist.c (Jed Brown            2013-02-11  18)   ierr = VecSetLayout(*gg,map);CHKERRQ(ierr);
2dcb2ebcaafe src/dm/impls/da/dadist.c (Barry Smith          2010-10-31  19)   PetscFunctionReturn(0)


   If we were a bit more clever we could probably avoid all communication in VecDuplicate() but is it worth bothering unless we know it is problematic

> On Jun 19, 2018, at 4:30 PM, Junchao Zhang <jczhang at mcs.anl.gov> wrote:
> 
> I met an expensive VecDuplicate. See the attached call stack. VecDuplicate_MPI_DA indirectly calls MPI_Allreduce/Allgather to build the vector's layout, but it then immediately destroys it with VecGetLayout & VecSetLayout. Is it wrong?
> src/dm/impls/da/dadist.c
> PetscErrorCode  VecDuplicate_MPI_DA(Vec g,Vec *gg)
> {
>   PetscErrorCode ierr;
>   DM             da;
>   PetscLayout    map;
> 
>   PetscFunctionBegin;
>   ierr = VecGetDM(g, &da);CHKERRQ(ierr);
>   ierr = DMCreateGlobalVector(da,gg);CHKERRQ(ierr);
>   ierr = VecGetLayout(g,&map);CHKERRQ(ierr);
>   ierr = VecSetLayout(*gg,map);CHKERRQ(ierr);
>   PetscFunctionReturn(0);
> }
> 
> --Junchao Zhang
> <VecDup.png>