[petsc-dev] Slow ViennaCL performance on KSP ex12

[Mani Chandra]

Hi,

Following http://www.mcs.anl.gov/petsc/petsc-20/conference/Rupp_K.pdf (page
16), I ran KSP ex12 for two cases:

1) time ./ex12 -m 100 -n 100 -log_summary > log_summary_no_viennacl

real    0m0.213s
user    0m0.206s
sys     0m0.004s

2) ./ex12 -m 100 -n 100 -vec_type viennacl -mat_type aijviennacl
-log_summary > log_summary_with_viennacl

real    0m20.296s
user    0m46.025s
sys     0m1.435s

The runs have been performed on a CPU : AMD A10-5800K, with OpenCL from
AMD-APP-SDK-v3.0.

Attached are:
1) configure.log for the petsc build
2) log summary without viennacl
3) log summary with viennacl
4) OpenCL info for the system on which the runs were performed

Perhaps the reason for the slow performance are superfluous copies being
performed, which need not occur when running ViennaCL on the CPU. Looking
at
http://www.mcs.anl.gov/petsc/petsc-dev/src/vec/vec/impls/seq/seqviennacl/vecviennacl.cxx
:

/* Copies a vector from the CPU to the GPU unless we already have an
up-to-date copy on the GPU */
PetscErrorCode VecViennaCLCopyToGPU(Vec v)
{
  PetscErrorCode ierr;

  PetscFunctionBegin;
  ierr = VecViennaCLAllocateCheck(v);CHKERRQ(ierr);
  if (v->map->n > 0) {
    if (v->valid_GPU_array == PETSC_VIENNACL_CPU) {
      ierr = PetscLogEventBegin(VEC_ViennaCLCopyToGPU,v,0,0,0);CHKERRQ(ierr);
      try {
        ViennaCLVector *vec = ((Vec_ViennaCL*)v->spptr)->GPUarray;
        viennacl::fast_copy(*(PetscScalar**)v->data,
*(PetscScalar**)v->data + v->map->n, vec->begin());
        ViennaCLWaitForGPU();
      } catch(std::exception const & ex) {
        SETERRQ1(PETSC_COMM_SELF,PETSC_ERR_LIB,"ViennaCL error: %s", ex.what());
      }
      ierr = PetscLogEventEnd(VEC_ViennaCLCopyToGPU,v,0,0,0);CHKERRQ(ierr);
      v->valid_GPU_array = PETSC_VIENNACL_BOTH;
    }
  }
  PetscFunctionReturn(0);
}

When running ViennaCL with OpenCL on the CPU, the above function should
maybe be modified?

Cheers,
Mani
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20151010/059d79f6/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: configure.log
Type: text/x-log
Size: 5938960 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20151010/059d79f6/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: log_summary_no_viennacl
Type: application/octet-stream
Size: 9240 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20151010/059d79f6/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: log_summary_with_viennacl
Type: application/octet-stream
Size: 9640 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20151010/059d79f6/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: opencl_info
Type: application/octet-stream
Size: 1822 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20151010/059d79f6/attachment-0002.obj>