[petsc-dev] programming model for PETSc

[Barry Smith]

On Nov 24, 2011, at 4:41 PM, Matthew Knepley wrote:

> On Thu, Nov 24, 2011 at 4:09 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> 
>   Jed,
> 
>   Let's stop arguing about whether MPI is or is not a good base for the next generation of HPC software but instead start a new conversation on what API (implemented on top of or not on top of MPI/pthreads etc etc) we want to build PETSc on to scale PETSc up to millions of cores with large NUMA nodes and GPU like accelerators.
> 
>    What do you want in the API?
> 
> Let's start with the "lowest" level, or at least the smallest. I think the only sane way to program for portable performance here
> is using CUDA-type vectorization. This SIMT style is explained well here http://www.yosefk.com/blog/simd-simt-smt-parallelism-in-nvidia-gpus.html
> I think this is much easier and more portable than the intrinsics for Intel, and more performant and less error prone than threads.
> I think you can show that it will accomplish anything we want to do. OpenCL seems to have capitulated on this point. Do we agree
> here?

   What syntax do you suggest for writing the code that is "vectorized"?  What tools exist, could exist, for mapping from that syntax to what is needed by the various compilers/hardware?

   For daxpy() the syntax doesn't really matter, anything will do. For other kernels: maxpy, sparse matrix vector product, triangular solves, P*A*Pt, mesh operations, sorts, indirect access .... the choice of syntax likely matters a great deal. We should test the syntax out on a wide range of kernels. For example look at VecMAXPY_kernel vs VecMAXPY_Seq vs VecMAXPY_VecCUSPMAXPY4 and the three delegators VecMAXPY_SeqPThread, VecMAXPY_MPI, and VecMAXPY_SeqCUSP;

   How does data layout relate to the vectorization you are going to do on that data and vis-versa?


   Barry

> 
>    Matt
>  
> 
>   Barry
> 
> 
> 
> 
> -- 
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener