[petsc-users] chowiluviennacl

[Patrick Sanan]

Just some more background on that algorithm for others reading (which is
obviously explained better in the paper, which Richard linked). As others
point out, I don't think it fits your use case.

The motivation for the Chow-Patel algorithm is the fact that traditional
ILU preconditioners don't work well in "fine-grained parallel" environments
like GPUs. "Triangularity" is something associated with lots of data
dependencies - think about Gaussian elimination  - the whole idea is
solving one equation at a time, based on the solutions of other equations.
The Chow-Patel approach is to approach things in a clever way (solving a
set of nonlinear equations describing the individual entries of the
factors) to simultaneously compute all the entries of the triangular
factors, asynchronously on lots of threads. That doesn't solve the problem
of how to solve the resulting triangular systems in parallel, though, so
that's done with an iterative approach (that is, you can approximate L^(-1)
with a polynomial in L). It's a new approach and thus should be considered
experimental. Key to note is that all of this is only explored or
implemented on a single node (shared-memory domain), so if you want to use
this preconditioner on multiple ranks it needs to be a sub-preconditioner
in a block Jacobi, ASM, or other method with "local subsolves".

Am Mo., 20. Jan. 2020 um 23:41 Uhr schrieb Mills, Richard Tran via
petsc-users <petsc-users at mcs.anl.gov>:

> Hi Xiangdong,
>
> Maybe I am misunderstanding you, but it sounds like you want an exact
> direct solution, so I don't understand why you are using an incomplete
> factorization solver for this. SuperLU_DIST (as Mark has suggested) or
> MUMPS are two such packages that provide MPI-parallel sparse LU
> factorization. If you need GPU support, SuperLU_DIST has such support. I
> don't know the status of our support for using the GPU capabilities of
> this, though -- I assume another developer can chime in regarding this.
>
> Note that the ILU provided by "chowiluiennacl" employs a very different
> algorithm than the standard PCILU in PETSc, and you shouldn't expect to get
> the same incomplete factorization. The algorithm is described in this paper
> by Chow and Patel:
>
> https://www.cc.gatech.edu/~echow/pubs/parilu-sisc.pdf
>
> Best regards,
> Richard
> On 1/15/20 11:39 AM, Xiangdong wrote:
>
> I just submitted the issue: https://gitlab.com/petsc/petsc/issues/535
>
> What I really want is an exact Block Tri-diagonal solver on GPU. Since for
> block tridiagonal system, ILU0 would be the same as ILU. So I tried the
> chowiluviennacl. but I found that the default parameters does not produce
> the same ILU0 factorization as the CPU ones (PCILU). My guess is that if I
> increase the number of sweeps chow_patel_ilu_config.sweeps(3), it may give
> a better result. So the option Keys would be helpful.
>
> Since Mark mentioned the Superlu's GPU feature, can I use superlu or
> hypre's GPU functionality through PETSc?
>
> Thank you.
>
> Xiangdong
>
> On Wed, Jan 15, 2020 at 2:22 PM Matthew Knepley <knepley at gmail.com> wrote:
>
>> On Wed, Jan 15, 2020 at 1:48 PM Xiangdong <epscodes at gmail.com> wrote:
>>
>>> In the ViennaCL manual
>>> http://viennacl.sourceforge.net/doc/manual-algorithms.html
>>>
>>> It did expose two parameters:
>>>
>>> // configuration of preconditioner:
>>> viennacl::linalg::chow_patel_tag chow_patel_ilu_config;
>>> chow_patel_ilu_config.sweeps(3); // three nonlinear sweeps
>>> chow_patel_ilu_config.jacobi_iters(2); // two Jacobi iterations per
>>> triangular 'solve' Rx=r
>>>
>>> and mentioned that:
>>> The number of nonlinear sweeps and Jacobi iterations need to be set
>>> problem-specific for best performance.
>>>
>>> In the PETSc' implementation:
>>>
>>> viennacl::linalg::chow_patel_tag ilu_tag;
>>>     ViennaCLAIJMatrix *mat = (ViennaCLAIJMatrix*)gpustruct->mat;
>>>     ilu->CHOWILUVIENNACL = new
>>> viennacl::linalg::chow_patel_ilu_precond<viennacl::compressed_matrix<PetscScalar>
>>> >(*mat, ilu_tag);
>>>
>>> The default is used. Is it possible to expose these two parameters so
>>> that user can change it through option keys?
>>>
>>
>> Yes. Do you mind making an issue for it? That way we can better keep
>> track.
>>
>> https://gitlab.com/petsc/petsc/issues
>>
>>   Thanks,
>>
>>     Matt
>>
>>
>>> Thank you.
>>>
>>> Xiangdong
>>>
>>> On Wed, Jan 15, 2020 at 12:40 PM Matthew Knepley <knepley at gmail.com>
>>> wrote:
>>>
>>>> On Wed, Jan 15, 2020 at 9:59 AM Xiangdong <epscodes at gmail.com> wrote:
>>>>
>>>>> Maybe I am not clear. I want to solve the block tridiagonal system
>>>>> Tx=b a few times with same T but different b. On CPU, I can have it by
>>>>> applying the ILU0 and reuse the factorization. Since it is block
>>>>> tridiagonal, ILU0 would give same results as LU.
>>>>>
>>>>> I am trying to do the same thing on GPU with chowiluviennacl, but
>>>>> found default factorization does not produce the exact factorization for
>>>>> tridiagonal system. Can we tight the drop off tolerance so that it can work
>>>>> as LU for tridiagonal system?
>>>>>
>>>>
>>>> There are no options in our implementation. You could look at the
>>>> ViennaCL manual to see if we missed something.
>>>>
>>>>   Thanks,
>>>>
>>>>     Matt
>>>>
>>>>
>>>>> Thank you.
>>>>>
>>>>> Xiangdong
>>>>>
>>>>> On Wed, Jan 15, 2020 at 9:41 AM Matthew Knepley <knepley at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> On Wed, Jan 15, 2020 at 9:36 AM Xiangdong <epscodes at gmail.com> wrote:
>>>>>>
>>>>>>> Can chowiluviennacl do ilu0?
>>>>>>>
>>>>>>> I need to solve a tri-diagonal system directly. If I apply the
>>>>>>> PCILU, I will obtain the exact solution with preonly + pcilu. However, the
>>>>>>> preonly + chowiluviennacl will not provide the exact solution. Any option
>>>>>>> keys to set the CHOWILUVIENNACL filling level or dropping off tolerance
>>>>>>> like the standard ilu?
>>>>>>>
>>>>>>
>>>>>> No. However, such a scheme makes less sense here. This algorithm
>>>>>> spawns a individual threads for individual elements. Drop tolerance
>>>>>> is not less work, it is sparser, but that should not matter for a
>>>>>> tridiagonal system. Levels also is not applicable since you have only 1
>>>>>> level.
>>>>>>
>>>>>>   Thanks,
>>>>>>
>>>>>>     Matt
>>>>>>
>>>>>>
>>>>>>> Thank you.
>>>>>>>
>>>>>>> Best,
>>>>>>> Xiangdong
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jan 14, 2020 at 10:05 PM Matthew Knepley <knepley at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> On Tue, Jan 14, 2020 at 9:56 PM Xiangdong <epscodes at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Dear Developers,
>>>>>>>>>
>>>>>>>>> I have a quick question about the chowiluviennacl. When I tried to
>>>>>>>>> use it, I found that it only works for np=1, not np>1. However, in the
>>>>>>>>> description of chowiluviennacl.cxx, it says "the ViennaCL Chow-Patel
>>>>>>>>> parallel ILU preconditioner".
>>>>>>>>>
>>>>>>>>
>>>>>>>> By parallel, this means shared memory parallelism on the GPU.
>>>>>>>>
>>>>>>>>
>>>>>>>>> I am wondering whether I am using it correctly.
>>>>>>>>> Does chowiluviennacl work for np>1?
>>>>>>>>>
>>>>>>>>
>>>>>>>> I do not believe so. I do not see why it could not be extended, but
>>>>>>>> that would mean writing some more code.
>>>>>>>>
>>>>>>>>   Thanks,
>>>>>>>>
>>>>>>>>     Matt
>>>>>>>>
>>>>>>>>
>>>>>>>>> In addition, are there option keys for the chowiluviennacl one can
>>>>>>>>> try?
>>>>>>>>> Thank you.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Xiangdong
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> What most experimenters take for granted before they begin their
>>>>>>>> experiments is infinitely more interesting than any results to which their
>>>>>>>> experiments lead.
>>>>>>>> -- Norbert Wiener
>>>>>>>>
>>>>>>>> https://www.cse.buffalo.edu/~knepley/
>>>>>>>> <http://www.cse.buffalo.edu/~knepley/>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> What most experimenters take for granted before they begin their
>>>>>> experiments is infinitely more interesting than any results to which their
>>>>>> experiments lead.
>>>>>> -- Norbert Wiener
>>>>>>
>>>>>> https://www.cse.buffalo.edu/~knepley/
>>>>>> <http://www.cse.buffalo.edu/~knepley/>
>>>>>>
>>>>>
>>>>
>>>> --
>>>> What most experimenters take for granted before they begin their
>>>> experiments is infinitely more interesting than any results to which their
>>>> experiments lead.
>>>> -- Norbert Wiener
>>>>
>>>> https://www.cse.buffalo.edu/~knepley/
>>>> <http://www.cse.buffalo.edu/~knepley/>
>>>>
>>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://www.cse.buffalo.edu/~knepley/
>> <http://www.cse.buffalo.edu/~knepley/>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200121/8cc7af65/attachment-0001.html>