[petsc-dev] Possible bugs when using TS with ViennaCL

[Mani Chandra]

Hi Karl,

I now checked the error flags of all petsc functions in all functions. I
also recompiled petsc with debug. Here's the report:

Case 1) Still works. All good.

Case 2) This is the case with ComputeResidual using VecGetArrays but with
DMSetVecType(da, VECVIENNACL) and DMSetMatType(da, MATAIJVIENNACL). The
flags don't show any errors. *The programs proceeds smoothly but the
solution is wrong even though SNES is converging.*

Case 3) This is the case with ComputeResidualViennaCL using
VecViennaCLGetArrays with and without DMSetVecType(da, VECVIENNACL) and
DMSetMatType(da, MATAIJVIENNACL). In both cases I get the following error:

[0]PETSC ERROR: TSComputeIFunction() line 676 in
/home/mc/Downloads/petsc/src/ts/interface/ts.c
[0]PETSC ERROR: SNESTSFormFunction_Theta() line 284 in
/home/mc/Downloads/petsc/src/ts/impls/implicit/theta/theta.c
[0]PETSC ERROR: SNESTSFormFunction() line 3499 in
/home/mc/Downloads/petsc/src/ts/interface/ts.c
[0]PETSC ERROR: SNESComputeFunction() line 2089 in
/home/mc/Downloads/petsc/src/snes/interface/snes.c
[0]PETSC ERROR: SNESSolve_NEWTONLS() line 175 in
/home/mc/Downloads/petsc/src/snes/impls/ls/ls.c
[0]PETSC ERROR: SNESSolve() line 3812 in
/home/mc/Downloads/petsc/src/snes/interface/snes.c
[0]PETSC ERROR: TSStep_Theta() line 183 in
/home/mc/Downloads/petsc/src/ts/impls/implicit/theta/theta.c
[0]PETSC ERROR: TSStep() line 2625 in
/home/mc/Downloads/petsc/src/ts/interface/ts.c
[0]PETSC ERROR: TSSolve() line 2741 in
/home/mc/Downloads/petsc/src/ts/interface/ts.c
[0]PETSC ERROR: main() line 83 in
/home/mc/PhD/opencl_tests/petsc_opencl/petsc_opencl.cpp
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode -473550369.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------

The error occured at VecViennaCLGetArrayRead/Write in my
ComputeResidualViennaCL function. Note that with the petsc debug mode, the
code crashes with the above error even if I don't catch the error codes.
Might be because of some VecViennaCLGetArray calls inside petsc.

Cheers,
Mani


On Sat, Jan 25, 2014 at 2:48 AM, Karl Rupp <rupp at mcs.anl.gov> wrote:

> Hi Mani,
>
> please check the return value of *all* function calls from PETSc, e.g.
>  ierr = DMCreateGlobalVector(da, &soln);CHKERRQ(ierr);
> instead of just
>  DMCreateGlobalVector(da, &soln);
> Most likely one of the routines threw an error, but your code just kept
> going, producing wrong results.
>
> Best regards,
> Karli
>
>
>
> On 01/25/2014 05:35 AM, Mani Chandra wrote:
>
>> Hi Everyone,
>>
>> I'm trying to use TS with ViennaCL vecs/mats and residual evaluation on
>> device and have encountered some problems. I have attached a small test
>> code that illustrates the issue.
>>
>> The code simply advects a blob diagonally using TS. I have written the
>> residual evaluation function using 1) the usual Petsc vectors
>> (VecGetArray) and 2) using ViennaCL vectors (VecViennaCLGetArrayRead/
>> Write).
>>
>> Run the code using the following:
>> ./petsc_opencl -ts_monitor -snes_monitor -ts_max_steps 1000 -ts_type
>> theta -ts_dt 10 -snes_rtol 1e-4 -ts_final_time 1000
>> -ts_monitor_draw_solution
>>
>> Case 1) No ViennaCL anywhere. I simply use the usual Petsc vectors and
>> set the residual evaluation function as ComputeResidual (line no. 55).
>> This case works and the blob is indeed advected as can be seen. (I
>> haven't bothered with the boundaries. The simulation just stops before
>> the blob hits the boundaries).
>>
>> Case 2) We again use the ComputeResidual but now enable ViennaCL vecs
>> and mats (line nos. 48, 49). This case does NOT work. The SNES monitor
>> shows convergence but the solution makes no sense.
>>
>> Case 3) We now use ComputeResidualViennaCL (line no. 56). This does NOT
>> work either with or without enabling the ViennaCL vecs (line nos. 48, 49).
>>
>> Cheers,
>> Mani
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20140125/7240042a/attachment.html>