On Sun, Jan 29, 2012 at 12:53 PM, recrusader <span dir="ltr"><<a href="mailto:recrusader@gmail.com">recrusader@gmail.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Dear PETSc developers,<br>
<br>
With your help, I can successfully PETSc-deve with enabling GPU and<br>
complex number.<br>
However, when I compiled the codes, I met some errors. I also tried to<br>
use simple codes to realize the same function. However, the errors<br>
disappear. One example is as follows:<br>
<br>
for the function "VecScale_SeqCUSP"<br>
"#undef __FUNCT__<br>
#define __FUNCT__ "VecScale_SeqCUSP"<br>
PetscErrorCode VecScale_SeqCUSP(Vec xin, PetscScalar alpha)<br>
{<br>
CUSPARRAY *xarray;<br>
PetscErrorCode ierr;<br>
<br>
PetscFunctionBegin;<br>
if (alpha == 0.0) {<br>
ierr = VecSet_SeqCUSP(xin,alpha);CHKERRQ(ierr);<br>
} else if (alpha != 1.0) {<br>
ierr = VecCUSPGetArrayReadWrite(xin,&xarray);CHKERRQ(ierr);<br>
try {<br>
cusp::blas::scal(*xarray,alpha);<br>
} catch(char* ex) {<br>
SETERRQ1(PETSC_COMM_SELF,PETSC_ERR_LIB,"CUSP error: %s", ex);<br>
}<br>
ierr = VecCUSPRestoreArrayReadWrite(xin,&xarray);CHKERRQ(ierr);<br>
}<br>
ierr = WaitForGPU();CHKERRCUSP(ierr);<br>
ierr = PetscLogFlops(xin->map->n);CHKERRQ(ierr);<br>
PetscFunctionReturn(0);<br>
} "<br>
<br>
When I compiled PETSc-dev, I met the following errors:<br>
" /opt/apps/cuda/4.0/cuda/include/cusp/detail/blas.inl(134): warning:<br>
calling a __host__ function from a __host__ __device__ function is not<br>
allowed<br>
detected during:<br>
instantiation of "void<br>
cusp::blas::detail::SCAL<T>::operator()(T2 &) [with<br>
T=std::complex<double>, T2=PetscScalar]"<br>
/opt/apps/cuda/4.0/cuda/include/thrust/detail/device/cuda/for_each.inl(72): here<br>
instantiation of "void<br>
thrust::detail::device::cuda::for_each_n_closure<RandomAccessIterator,<br>
Size, UnaryFunction>::operator()() [with<br>
RandomAccessIterator=thrust::detail::normal_iterator<thrust::device_ptr<PetscScalar>>,<br>
Size=long, UnaryFunction=cusp::blas::detail::SCAL<std::complex<double>>]"<br>
/opt/apps/cuda/4.0/cuda/include/thrust/detail/device/cuda/detail/launch_closure.inl(51):<br>
here<br>
instantiation of "void<br>
thrust::detail::device::cuda::detail::launch_closure_by_value(NullaryFunction)<br>
[with NullaryFunction=thrust::detail::device::cuda::for_each_n_closure<thrust::detail::normal_iterator<thrust::device_ptr<PetscScalar>>,<br>
long, cusp::blas::detail::SCAL<std::complex<double>>>]"<br>
/opt/apps/cuda/4.0/cuda/include/thrust/detail/device/cuda/detail/launch_closure.inl(71):<br>
here<br>
instantiation of "size_t<br>
thrust::detail::device::cuda::detail::closure_launcher_base<NullaryFunction,<br>
launch_by_value>::block_size_with_maximal_occupancy(size_t) [with<br>
NullaryFunction=thrust::detail::device::cuda::for_each_n_closure<thrust::detail::normal_iterator<thrust::device_ptr<PetscScalar>>,<br>
long, cusp::blas::detail::SCAL<std::complex<double>>>,<br>
launch_by_value=true]"<br>
/opt/apps/cuda/4.0/cuda/include/thrust/detail/device/cuda/detail/launch_closure.inl(136):<br>
here<br>
instantiation of "thrust::pair<size_t, size_t><br>
thrust::detail::device::cuda::detail::closure_launcher<NullaryFunction>::configuration_with_maximal_occupancy(Size)<br>
[with NullaryFunction=thrust::detail::device::cuda::for_each_n_closure<thrust::detail::normal_iterator<thrust::device_ptr<PetscScalar>>,<br>
long, cusp::blas::detail::SCAL<std::complex<double>>>, Size=long]"<br>
/opt/apps/cuda/4.0/cuda/include/thrust/detail/device/cuda/detail/launch_closure.inl(145):<br>
here<br>
[ 6 instantiation contexts not shown ]<br>
instantiation of "InputIterator<br>
thrust::detail::dispatch::for_each(InputIterator, InputIterator,<br>
UnaryFunction, thrust::device_space_tag) [with<br>
InputIterator=thrust::detail::normal_iterator<thrust::device_ptr<PetscScalar>>,<br>
UnaryFunction=cusp::blas::detail::SCAL<std::complex<double>>]"<br>
/opt/apps/cuda/4.0/cuda/include/thrust/detail/for_each.inl(51): here<br>
instantiation of "InputIterator<br>
thrust::detail::for_each(InputIterator, InputIterator, UnaryFunction)<br>
[with InputIterator=thrust::detail::normal_iterator<thrust::device_ptr<PetscScalar>>,<br>
UnaryFunction=cusp::blas::detail::SCAL<std::complex<double>>]"<br>
/opt/apps/cuda/4.0/cuda/include/thrust/detail/for_each.inl(67): here<br>
instantiation of "void thrust::for_each(InputIterator,<br>
InputIterator, UnaryFunction) [with<br>
InputIterator=thrust::detail::normal_iterator<thrust::device_ptr<PetscScalar>>,<br>
UnaryFunction=cusp::blas::detail::SCAL<std::complex<double>>]"<br>
(367): here<br>
instantiation of "void<br>
cusp::blas::detail::scal(ForwardIterator, ForwardIterator, ScalarType)<br>
[with ForwardIterator=thrust::detail::normal_iterator<thrust::device_ptr<PetscScalar>>,<br>
ScalarType=std::complex<double>]"<br>
(748): here<br>
instantiation of "void cusp::blas::scal(Array &,<br>
ScalarType) [with Array=cusp::array1d<PetscScalar,<br>
cusp::device_memory>, ScalarType=std::complex<double>]"<br>
<a href="http://veccusp.cu" target="_blank">veccusp.cu</a>(1185): here<br>
<br>
/opt/apps/cuda/4.0/cuda/include/thrust/detail/device/cuda/detail/launch_closure.inl(51):<br>
error: a value of type "int" cannot be assigned to an entity of type<br>
"_ZNSt7complexIdE9_ComplexTE"<br>
<br>
"<br>
However, I further realize simiar codes as<br>
"<br>
#include <thrust/version.h><br>
#include <cusp/version.h><br>
#include <iostream><br>
#include <cusp/blas.h><br>
#include <cusp/array1d.h><br>
#include <complex><br>
<br>
int main(void)<br>
{<br>
cusp::array1d<std::complex<double>, cusp::host_memory> *x;<br>
<br>
x=new cusp::array1d<std::complex<double>, cusp::host_memory>(2,0.0);<br>
<br>
std::complex<double> alpha(1,2.0);<br>
cusp::blas::scal(*x,alpha);<br>
<br>
return 0;<br>
}<br>
"<br>
<br>
When I complied it using "nvcc <a href="http://gputest.cu" target="_blank">gputest.cu</a> -o gputest", I only meet<br>
warning information as follows:<br>
"<br>
/opt/apps/cuda/4.0/cuda/bin/../include/cusp/detail/blas.inl(134):<br>
warning: calling a __host__ function from a __host__ __device__<br>
function is not allowed<br>
detected during:<br>
instantiation of "void<br>
cusp::blas::detail::SCAL<T>::operator()(T2 &) [with<br>
T=std::complex<double>, T2=std::complex<double>]"<br>
/opt/apps/cuda/4.0/cuda/bin/../include/thrust/detail/host/for_each.inl(37): here<br>
instantiation of "InputIterator<br>
thrust::detail::host::for_each(InputIterator, InputIterator,<br>
UnaryFunction) [with<br>
InputIterator=thrust::detail::normal_iterator<std::complex<double> *>,<br>
UnaryFunction=cusp::blas::detail::SCAL<std::complex<double>>]"<br>
/opt/apps/cuda/4.0/cuda/bin/../include/thrust/detail/dispatch/for_each.h(46):<br>
here<br>
instantiation of "InputIterator<br>
thrust::detail::dispatch::for_each(InputIterator, InputIterator,<br>
UnaryFunction, thrust::host_space_tag) [with<br>
InputIterator=thrust::detail::normal_iterator<std::complex<double> *>,<br>
UnaryFunction=cusp::blas::detail::SCAL<std::complex<double>>]"<br>
/opt/apps/cuda/4.0/cuda/bin/../include/thrust/detail/for_each.inl(51): here<br>
instantiation of "InputIterator<br>
thrust::detail::for_each(InputIterator, InputIterator, UnaryFunction)<br>
[with InputIterator=thrust::detail::normal_iterator<std::complex<double><br>
*>, UnaryFunction=cusp::blas::detail::SCAL<std::complex<double>>]"<br>
/opt/apps/cuda/4.0/cuda/bin/../include/thrust/detail/for_each.inl(67): here<br>
instantiation of "void thrust::for_each(InputIterator,<br>
InputIterator, UnaryFunction) [with<br>
InputIterator=thrust::detail::normal_iterator<std::complex<double> *>,<br>
UnaryFunction=cusp::blas::detail::SCAL<std::complex<double>>]"<br>
(367): here<br>
instantiation of "void<br>
cusp::blas::detail::scal(ForwardIterator, ForwardIterator, ScalarType)<br>
[with ForwardIterator=thrust::detail::normal_iterator<std::complex<double><br>
*>, ScalarType=std::complex<double>]"<br>
(748): here<br>
instantiation of "void cusp::blas::scal(Array &,<br>
ScalarType) [with Array=cusp::array1d<std::complex<double>,<br>
cusp::host_memory>, ScalarType=std::complex<double>]"<br>
<a href="http://gputest.cu" target="_blank">gputest.cu</a>(25): here<br>
<br>
"<br>
There are not errors like<br>
"/opt/apps/cuda/4.0/cuda/include/thrust/detail/device/cuda/detail/launch_closure.inl(51):<br>
error: a value of type "int" cannot be assigned to an entity of type<br>
"_ZNSt7complexIdE9_ComplexTE" "<br>
<br>
Furthermore, the warning information is also different between<br>
PETSc-dev and simple codes.<br>
<br>
Could you give me some suggestion for this errors? Thank you very much.<br></blockquote><div><br></div><div>The headers are complicated to get right. The whole point of what we did is to give a way to use GPU</div><div>simply through the existing PETSc linear algebra interface.</div>
<div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Best,<br>
Yujie<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>