[petsc-users] What is the right way to use split mode asynchronous reduction?

[teivml at gmail.com]

Dear Petsc users,

I would like to confirm that the asynchronous calculation of the vector
norm is faster than the synchronous calculation with the following code.


PetscLogDouble tt1,tt2;
ierr = VecSet(c,one);
ierr = VecSet(u,one);
ierr = VecSet(b,one);

ierr = KSPCreate(PETSC_COMM_WORLD,&ksp); CHKERRQ(ierr);
ierr = KSP_MatMult(ksp,A,x,Ax); CHKERRQ(ierr);


ierr = PetscTime(&tt1);CHKERRQ(ierr);

ierr = VecNormBegin(u,NORM_2,&norm1);
ierr = PetscCommSplitReductionBegin(PetscObjectComm((PetscObject)Ax));
ierr = KSP_MatMult(ksp,A,c,Ac);
ierr = VecNormEnd(u,NORM_2,&norm1);


 ierr = PetscTime(&tt2);CHKERRQ(ierr);
 ierr = PetscPrintf(PETSC_COMM_WORLD, "The time used for the asynchronous
calculation: %f\n",tt2-tt1); CHKERRQ(ierr);
ierr = PetscPrintf(PETSC_COMM_WORLD,"+ |u| =  %g\n",(double) norm1);
CHKERRQ(ierr);


 ierr = PetscTime(&tt1);CHKERRQ(ierr);
ierr = VecNorm(b,NORM_2,&norm2); CHKERRQ(ierr);
ierr = KSP_MatMult(ksp,A,c,Ac);


ierr = PetscTime(&tt2);CHKERRQ(ierr);
ierr = PetscPrintf(PETSC_COMM_WORLD, "The time used for the synchronous
calculation: %f\n",tt2-tt1); CHKERRQ(ierr);
ierr = PetscPrintf(PETSC_COMM_WORLD,"+ |b| =  %g\n",(double) norm2);
CHKERRQ(ierr);


This code computes a matrix-vector product and a vector norm asynchronously
and synchronously.

The calculation is carried out on a single node PC with a Xeon CPU.
The result of the code above shows that the synchronous calculation is
faster than the asynchronous calculation. The MPI library is MPICH 3.3 and
the parallel number is n = 20.

The time used for the asynchronous calculation: 0.001622
+ |u| =  100.
The time used for the synchronous calculation: 0.000062
+ |b| =  100.


Is there anything I should consider in order to properly take advantage of
the Petsc's asynchronous progress?

Thank you for any help you can provide.
Sincerely,
Teiv.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200914/328c62b0/attachment.html>