[petsc-dev] VecMDot_SeqCUSP improved

[Jose E. Roman]

El 26/03/2013, a las 02:41, Karl Rupp escribió:

> Hi Jose, Paul, and others,
> 
> I worked today and VecMDot and came up with an implementation which is faster than an iterated application of the standard cusp::blas::dot() (which, if I'm not mistaken, just forwards to CUBLAS) if enough vectors (>~6) are involved. For complex arithmetic, an iterated application of cusp::blas::dotc() is used, since passing complex types to CUDA kernels is fairly tricky within PETSc. Jose, any performance feedback from within SLEPc is appreciated :-)
> 
> The new implementation is based on custom kernels, only allocates a little scratchpad memory and is thus more memory efficient than the old version. Also, any unnecessary copying of data is avoided. This should speed up GMRES quite a bit, yet I haven't run any dedicated GMRES benchmarks. Paul, I guess you have some samples at hand, don't you?
> 
> Best regards,
> Karli

In my tests, the new implementation is actually slower. I tried src/vec/vec/examples/tests/ex43.c with 200 vectors of length 10000. Time increases from 4.1 to 7.2. Can anyone try to repeat the tests below?

I have an Intel Core i7 with two Tesla C2050.

Jose


master
---------------

$ ./ex43 -n 10000 -k 200 -mdot -log_summary

VecMDot             3980 1.0 3.6485e+00 1.0 7.96e+09 1.0 0.0e+00 0.0e+00 0.0e+00 11100  0  0  0  11100  0  0  0  2182

$ ./ex43 -n 10000 -k 200 -mdot -log_summary -vec_type cusp

VecMDot             3980 1.0 4.1368e+00 1.0 7.96e+09 1.0 0.0e+00 0.0e+00 0.0e+00 40100  0  0  0  40100  0  0  0  1924

$ ./ex43 -n 10000 -k 200 -log_summary

VecDot            398000 1.0 2.1585e+01 1.0 7.96e+09 1.0 0.0e+00 0.0e+00 0.0e+00 78100  0  0  0  78100  0  0  0   369

$ ./ex43 -n 10000 -k 200 -log_summary -vec_type cusp

VecDot            398000 1.0 2.9228e+01 1.0 7.96e+09 1.0 0.0e+00 0.0e+00 0.0e+00 82100  0  0  0  82100  0  0  0   272


next
---------------

$ ./ex43 -n 10000 -k 200 -mdot -log_summary

VecMDot             3980 1.0 3.6899e+00 1.0 7.96e+09 1.0 0.0e+00 0.0e+00 0.0e+00 39100  0  0  0  39100  0  0  0  2157

$ ./ex43 -n 10000 -k 200 -mdot -log_summary -vec_type cusp

VecMDot             3980 1.0 7.1823e+00 1.0 7.96e+09 1.0 0.0e+00 0.0e+00 0.0e+00 54100  0  0  0  54100  0  0  0  1108

$ ./ex43 -n 10000 -k 200 -log_summary

VecDot            398000 1.0 2.1702e+01 1.0 7.96e+09 1.0 0.0e+00 0.0e+00 0.0e+00 79100  0  0  0  79100  0  0  0   367

$ ./ex43 -n 10000 -k 200 -log_summary -vec_type cusp

VecDot            398000 1.0 2.8953e+01 1.0 7.96e+09 1.0 0.0e+00 0.0e+00 0.0e+00 82100  0  0  0  82100  0  0  0   275