[petsc-dev] Possible bugs when using TS with ViennaCL

[Mani Chandra]

Hi Karl,

Thanks for looking into it. Do let me know if there is anything I can do to
help you debug this. Attached is the code.

Cheers,
Mani


On Sat, Jan 25, 2014 at 3:14 PM, Karl Rupp <rupp at mcs.anl.gov> wrote:

> Hi Mani,
>
> could you please send me the code including the error checks?
>
> Thanks and best regards,
> Karli
>
>
>
> On 01/25/2014 10:07 PM, Mani Chandra wrote:
>
>> Hi Karl,
>>
>> I now checked the error flags of all petsc functions in all functions. I
>> also recompiled petsc with debug. Here's the report:
>>
>> Case 1) Still works. All good.
>>
>> Case 2) This is the case with ComputeResidual using VecGetArrays but
>> with DMSetVecType(da, VECVIENNACL) and DMSetMatType(da, MATAIJVIENNACL).
>> The flags don't show any errors. *The programs proceeds smoothly but the
>> solution is wrong even though SNES is converging.*
>>
>>
>> Case 3) This is the case with ComputeResidualViennaCL using
>> VecViennaCLGetArrays with and without DMSetVecType(da, VECVIENNACL) and
>> DMSetMatType(da, MATAIJVIENNACL). In both cases I get the following error:
>>
>> [0]PETSC ERROR: TSComputeIFunction() line 676 in
>> /home/mc/Downloads/petsc/src/ts/interface/ts.c
>> [0]PETSC ERROR: SNESTSFormFunction_Theta() line 284 in
>> /home/mc/Downloads/petsc/src/ts/impls/implicit/theta/theta.c
>> [0]PETSC ERROR: SNESTSFormFunction() line 3499 in
>> /home/mc/Downloads/petsc/src/ts/interface/ts.c
>> [0]PETSC ERROR: SNESComputeFunction() line 2089 in
>> /home/mc/Downloads/petsc/src/snes/interface/snes.c
>> [0]PETSC ERROR: SNESSolve_NEWTONLS() line 175 in
>> /home/mc/Downloads/petsc/src/snes/impls/ls/ls.c
>> [0]PETSC ERROR: SNESSolve() line 3812 in
>> /home/mc/Downloads/petsc/src/snes/interface/snes.c
>> [0]PETSC ERROR: TSStep_Theta() line 183 in
>> /home/mc/Downloads/petsc/src/ts/impls/implicit/theta/theta.c
>> [0]PETSC ERROR: TSStep() line 2625 in
>> /home/mc/Downloads/petsc/src/ts/interface/ts.c
>> [0]PETSC ERROR: TSSolve() line 2741 in
>> /home/mc/Downloads/petsc/src/ts/interface/ts.c
>> [0]PETSC ERROR: main() line 83 in
>> /home/mc/PhD/opencl_tests/petsc_opencl/petsc_opencl.cpp
>> ------------------------------------------------------------
>> --------------
>> MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
>> with errorcode -473550369.
>>
>> NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
>> You may or may not see output from other processes, depending on
>> exactly when Open MPI kills them.
>> ------------------------------------------------------------
>> --------------
>>
>> The error occured at VecViennaCLGetArrayRead/Write in my
>> ComputeResidualViennaCL function. Note that with the petsc debug mode,
>> the code crashes with the above error even if I don't catch the error
>> codes. Might be because of some VecViennaCLGetArray calls inside petsc.
>>
>> Cheers,
>> Mani
>>
>>
>> On Sat, Jan 25, 2014 at 2:48 AM, Karl Rupp <rupp at mcs.anl.gov
>> <mailto:rupp at mcs.anl.gov>> wrote:
>>
>>     Hi Mani,
>>
>>     please check the return value of *all* function calls from PETSc, e.g.
>>       ierr = DMCreateGlobalVector(da, &soln);CHKERRQ(ierr);
>>     instead of just
>>       DMCreateGlobalVector(da, &soln);
>>     Most likely one of the routines threw an error, but your code just
>>     kept going, producing wrong results.
>>
>>     Best regards,
>>     Karli
>>
>>
>>
>>     On 01/25/2014 05:35 AM, Mani Chandra wrote:
>>
>>         Hi Everyone,
>>
>>         I'm trying to use TS with ViennaCL vecs/mats and residual
>>         evaluation on
>>         device and have encountered some problems. I have attached a
>>         small test
>>         code that illustrates the issue.
>>
>>         The code simply advects a blob diagonally using TS. I have
>>         written the
>>         residual evaluation function using 1) the usual Petsc vectors
>>         (VecGetArray) and 2) using ViennaCL vectors
>>         (VecViennaCLGetArrayRead/__Write).
>>
>>
>>         Run the code using the following:
>>         ./petsc_opencl -ts_monitor -snes_monitor -ts_max_steps 1000
>> -ts_type
>>         theta -ts_dt 10 -snes_rtol 1e-4 -ts_final_time 1000
>>         -ts_monitor_draw_solution
>>
>>         Case 1) No ViennaCL anywhere. I simply use the usual Petsc
>>         vectors and
>>         set the residual evaluation function as ComputeResidual (line
>>         no. 55).
>>         This case works and the blob is indeed advected as can be seen. (I
>>         haven't bothered with the boundaries. The simulation just stops
>>         before
>>         the blob hits the boundaries).
>>
>>         Case 2) We again use the ComputeResidual but now enable ViennaCL
>>         vecs
>>         and mats (line nos. 48, 49). This case does NOT work. The SNES
>>         monitor
>>         shows convergence but the solution makes no sense.
>>
>>         Case 3) We now use ComputeResidualViennaCL (line no. 56). This
>>         does NOT
>>         work either with or without enabling the ViennaCL vecs (line
>>         nos. 48, 49).
>>
>>         Cheers,
>>         Mani
>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20140125/20bc70a1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: petsc_opencl.tar.gz
Type: application/x-gzip
Size: 4013 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20140125/20bc70a1/attachment.gz>