[petsc-users] Collect Trajectories components

[Francesco Brarda]

I created a for cycle where I call TSTrajectoryGetVecs, but only the 0 rank seems to enter in this cycle and I do not know why.
I thought the following might be a solution, but it is not working as I would like to, i.e. the final vector has the same local parts, a copy of the values obtained with the 0-rank. How should I change this, please?

Vec            U, partial, Uloc;
PetscScalar    *Ui, *partlocal;
PetscInt       i;
ierr = VecCreateMPI(PETSC_COMM_WORLD,PETSC_DECIDE,14,&partial);CHKERRQ(ierr);
      for(i=0; i<40; i++) {
      PetscReal ttime = i+1;
      ierr = TSTrajectoryGetVecs(appctx.tj,appctx.ts,PETSC_DECIDE,&ttime,U,NULL);CHKERRQ(ierr);
      ierr = DMGetLocalVector(appctx.da,&Uloc);CHKERRQ(ierr);
      ierr = DMGlobalToLocalBegin(appctx.da,U,INSERT_VALUES,Uloc);CHKERRQ(ierr);
      ierr = DMGlobalToLocalEnd(appctx.da,U,INSERT_VALUES,Uloc);CHKERRQ(ierr);
      ierr = DMDAVecGetArray(appctx.da,Uloc,&Ui);CHKERRQ(ierr);
      ierr = VecGetArray(partial,&partlocal);CHKERRQ(ierr);
      partlocal[i] = Ui[1];
      ierr = DMDAVecRestoreArray(appctx.da,U,&Ui);CHKERRQ(ierr);
      ierr = VecRestoreArray(partial,&partlocal);CHKERRQ(ierr);
      ierr = DMRestoreLocalVector(appctx.da,&Uloc);CHKERRQ(ierr); 
}


> Il giorno 27 mag 2021, alle ore 01:15, Barry Smith <bsmith at petsc.dev> ha scritto:
> 
> 
> 
>> On May 26, 2021, at 10:39 AM, Francesco Brarda <brardafrancesco at gmail.com> wrote:
>> 
>> Thank you very much.
>>>  Based on the error message it appears that your code is requesting different times on different MPI ranks. Is that what you intend to do?
>> Yes. I want to save different times across a vector built with multiple MPI ranks (PETSC_DECIDE for the local length).
>> The function is called only by the first proc (rank=0) and not from the others. Is there a way to force also other ranks to call that routine?
> 
>    Yes, just have all ranks call it and ignore the result on the other ranks. 
> 
>> Should I build everything into an external function outside the main?
> 
>    It can be called in main, does not need to be in a different function.
> 
>> 
>> Francesco
>> 
>> 
>>> Il giorno 26 mag 2021, alle ore 16:20, Barry Smith <bsmith at petsc.dev> ha scritto:
>>> 
>>> 
>>>  
>>> 
>>>   TSTrajectoryGetVecs() is listed as Collective on TS. This means all ranks must call it with the same times in the same order of operations on all ranks that share the TS. 
>>> 
>>>   You do not need to use VecScatter. Each process must call TSTrajectoryGetVecs with the same time but then you can have only the rank you care about select the entries from the resulting vectors you care about while the other ranks for that time just ignore the vectors since they do not need to values from it. 
>>> 
>>>   Barry
>>> 
>>>   
>>> 
>>>> On May 26, 2021, at 5:20 AM, Francesco Brarda <brardafrancesco at gmail.com> wrote:
>>>> 
>>>> Hi!
>>>> 
>>>> I solved an ODE system with TS. Now I would like to save one of the trajectories in specific times. To do so, I used TSTrajectoryGetVecs. 
>>>> The values of the variable I am interested in is on one processor. I want to collect these values in a parallel vector, but I had the error:
>>>> 
>>>> [0]PETSC ERROR: Invalid argument
>>>> [0]PETSC ERROR: Real value must be same on all processes, argument # 2
>>>> [0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
>>>> [0]PETSC ERROR: Petsc Release Version 3.14.4, unknown 
>>>> [0]PETSC ERROR: ./petsc_sir on a arch-debug named srvulx13 by fbrarda Wed May 26 12:00:42 2021
>>>> [0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --with-openblas-dir=/opt/packages/openblas/0.2.13-gcc --download-mpich PETSC_ARCH=arch-debug
>>>> [0]PETSC ERROR: #1 TSHistoryGetLocFromTime() line 134 in /home/fbrarda/petsc/src/ts/interface/tshistory.c
>>>> [0]PETSC ERROR: #2 TSTrajectoryReconstruct_Private() line 55 in /home/fbrarda/petsc/src/ts/trajectory/utils/reconstruct.c
>>>> [0]PETSC ERROR: #3 TSTrajectoryGetVecs() line 239 in /home/fbrarda/petsc/src/ts/trajectory/interface/traj.c
>>>> 
>>>> Is there any specific routine I can use to overcome this issue? Should I use VecScatter?
>>>> 
>>>> I hope I made myself clear.
>>>> Best,
>>>> Francesco
>>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20210527/fc170848/attachment-0001.html>