[petsc-dev] KSP automatic reordering

[Matthew Knepley]

On Tue, Sep 18, 2012 at 8:54 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:

> On Tue, Sep 18, 2012 at 8:45 PM, Matthew Knepley <knepley at gmail.com>wrote:
>
>> But it is nasty because it changes the convergence tests, the monitor
>>> routines (they report the residual norms in the scale system not the
>>> original).  Also, does it unscale the matrix after the solve (or leave it
>>> scaled for when it is never going to be used again?). The scaling can screw
>>> up algebraic multigrid methods. Does the scaling affect Eisenstat-Walker
>>> type convergence tests for Newton's method….  It is nasty code, hard to
>>> follow and hard for users to fully appreciate.
>>>
>>> We could do the same ugly hacked up thing for reordering; using a
>>> partitioner for between processes and a low-bandwidth ordering within
>>> inside the KSPSolve/SetUp().
>>>
>>
> I think that becomes too confusing and too intrusive to our code.
>
>
>>  It would be nice to have a cleaner/clearer abstract model in terms of
>>> the software layering to handle this.  For example I played with the idea
>>> of a "diagonal scaling" PC and "reordering" PC that
>>> does the change and then has its own KSP inside for the solve. Thus
>>> you'd run with -ksp_type preonly -pc_type reorder -reorder_ksp_type gmres
>>> -reorder_pc_type ilu etc. But that seems a bit pedantic
>>> and annoying that you have to put all your "true" solver options with a
>>> prefix.
>>>
>>
> This is what I had in mind. The thing is, we don't actually want people to
> use this. Redistributing the matrix is fairly expensive. We want them to
> change their code to produce a decent ordering. A good way to convince
> people that it's worthwhile is to put the performance numbers in front of
> their face. Matrix Market will always do its silly orderings and is stored
> on disk in a stupid format so that part will always be slow (and generally
> isn't part of the benchmark result).
>
>
>> Why not make it part of the matrix? For the minute, assume we are using a
>> DM. Then the
>> matrix has the nonzero pattern already. We can use an option to compute a
>> fill-reducing ordering
>> and either permute it directly, or just use the permutations on in and
>> out. This insulates it from
>> the solver completely.
>>
>
> At what point is the vector reordered?
>

Of course, I forgot EVERYTHING goes in the DM. Once the DM computes the
ordering, the
DMLocalToGlobal is altered and we are done.

   Matt

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20120918/7d0e88c1/attachment.html>