[petsc-dev] Petsc "make test" have more failures for --with-openmp=1

[Lawrence Mitchell]

> On 19 Mar 2021, at 15:20, Jed Brown <jed at jedbrown.org> wrote:
> 
>> I guess the only issue is now one needs to always remember to traverse cells in the reordered_cells order (although if you call DMPlexPermute that is automatic?).
> 
> Can DMPlexPermute handle a non-stratified permutation?

I think the answer here is still no (as per your previous statement). The way we work around this in Firedrake is as I described above. Effectively we always traverse the cells in order specified by the RCM reordering. For traversal of facets and other entities we then build a traversal order once by traversing the cells and greedily ordering the facets/edges/etc...

I think you could get this behaviour from dmplexpermute if one wrote it so that it permutes the cells with RCM and then compatibly permutes lower-dimensional entities. This would maintain stratification, but I think keep good cache locality for all entity traversals. But I do not understand the details of what dmplexpermute is doing at the moment.

Cheers,

Lawrence