[petsc-dev] Petsc+ViennaCL usage

[Mani Chandra]

Hi Karl,

A couple more questions:

1) In ViennaCL, queue.finish() seems to be called only if we enable the
flags VIENNACL_DEBUG_ALL or VIENNACL_DEBUG_KERNEL. How do I ensure that my
custom kernel finishes when the debug mode is not enabled? Should I be
doing one of the following:

viennacl::ocl::command_queue queue =
viennacl::ocl::current_context().get_queue();
viennacl::ocl::enqueue(kernel(*vars, *dvars_dt, *Fvars));
queue.finish();

or

viennacl::ocl::enqueue(kernel(*vars, *dvars_dt, *Fvars));
viennacl::backend::finish();

2) My platform 0 has only a GPU. So when I launch my custom kernel inside
the residual function it indeed does evaluate on the GPU. In particular,
suppose my residual function (for seq case) is like this:

PetscErrorCode ComputeResidual(TS ts,
                               PetscScalar t,
                               Vec X, Vec dX_dt,
                               Vec F, void *ptr)
{
    VecViennaCLGetArrayRead(X, &x);
    VecViennaCLGetArrayWrite(F, &f);

    viennacl::ocl::enqueue(myKernel(*x, *f));
// Put something here to finish the kernel.

    VecViennaCLRestoreArrayRead(X, &x);
    VecViennaCLRestoreArrayWrite(F, &f);
}

and I execute as given below:

./program

then the code inside the ComputeResidual function runs inside the GPU but
everything else runs on the CPU, right? (since I did not specify
-dm_vec_type viennacl  -dm_mat_type aijviennacl). Now suppose I execute as
given below:

./program -dm_vec_type viennacl -dm_mat_type aijviennacl

then every vector operation occurs using the viennacl code
in vecviennacl.cxx. And since my default platform is 0 (only having a
NVIDIA GPU), I thought everything will run on the GPU. However with the
ViennaCL debug mode, I get the following messages for the vector operations:

ViennaCL: Starting 1D-kernel 'assign_cpu'...
ViennaCL: Global work size: '16384'...
ViennaCL: Local work size: '128'...
ViennaCL: Kernel assign_cpu finished!

How is it possible that part of the ViennaCL code is using my CPU (which is
on a completely different platform, #1) and the custom kernel is launched
on my GPU (platform #0).

Cheers,
Mani



On Wed, Jan 22, 2014 at 3:43 PM, Karl Rupp <rupp at mcs.anl.gov> wrote:

> Hi Mani,
>
>
> > Thanks Karl. Is there anyway I can get the build info?
>
> For the moment only in case of an error...
>
>
>
>  Also, my system is such that I have the GPU on one platform (0) and the
>> CPU on another (1). When I try using -viennalcl_device_cpu, it still
>> uses the GPU cause I think that it defaults to the first platform it
>> finds which only has the GPU. Is there some way to toggle between the
>> platforms or do you suggest that I manually pass in the contexts into
>> viennacl?
>>
>
> Ah, so you have two OpenCL platforms available on your machine. Call
>   viennacl::ocl::set_context_platform_index(0, platform_index_here);
> at the beginning of your program in order to specify the default platform
> index. In your case this should be
>   viennacl::ocl::set_context_platform_index(0, 1);
> (cf. http://viennacl.sourceforge.net/viennacl-manual-current.pdf)
> I'll also make this configurable directly through PETSc in the next update
> of the ViennaCL bindings.
>
> Best regards,
> Karli
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20140124/3f253674/attachment.html>