[petsc-users] Writing a domain decomposition code with PETSc

[Jed Brown]

Åsmund Ervik <asmund.ervik at ntnu.no> writes:
>   subroutine transform_petsc_us(array,f,dof)
>     !Note: assumed shape-array does the "coordinate transformation"
>     real, intent(in), dimension(dof,1-stw:,1-stw:,1-stw:) :: array
>     real,intent(inout),dimension(1-stw:,1-stw:1-stw:,dof) :: f

This is a really bad ordering for cache-based architectures because as
you traverse the mesh, you need to pull values from segments of the
array that are far apart.  We have yet to encounter a practical project
in which indexing the first way (dofs adjacent in memory) is not faster,
sometimes by large factors.  (For example, Clawpack was recently
convinced to change and saw a 50% speedup.  The PETSc-FUN3D papers had
many comparisons.)  I recommend transforming your code to use the former
ordering, or if you must, use CPP macros so that you can change it at
compile time.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20131007/58fcfa86/attachment.pgp>