<html><body>
<p>We currently use Petsc routine "<tt><font size="4">AOCreateBasic</font></tt>" to create application order --- a mapping from used id to petsc id. Then we setup a connection graph for our mesh and assemble vector and matrix based on the Petsc order. When total number of elements in our mesh is about couple million and 64 computer nodes are used, AO is created in a reasonable time. But when we have 700 million elements in the mesh and use more then 256 computer nodes, the time to create AO grows significantly (9000 secs). <br>
<br>
It seems that each process has whole permutation table about the mapping and quite expensive in time and buffer size.<br>
a) Memory requirement is quite high (AO creation failed for 1.3 billion elements on 1024 nodes due to not enough memory).<br>
b) If user id numbering on each processor is partially continuous, then time for creation AO is reduced.<br>
c) In my understanding, AOApplicationToPetsc and AOPetscToApplication routines don't use communication since AO has whole mapping on each processor.<br>
<br>
In our application, we might not need the whole mapping in each processor. We only need to ids of vertices that reside in the processor (limited ghost vertices). We wonder whether there is an efficient way to create AO and to cut its creation time and buffer size. Let us know your thought on that.<br>
</body></html>