Parallel ISCreateGeneral()

[Matthew Knepley]

On Nov 18, 2007 12:21 PM, Tim Stitt <timothy.stitt at ichec.ie> wrote:
> Oh...ok I am now officially confused.
>
> I have developed a serial code for getting the first k rows of an
> inverted sparse matrix..thanks to PETSC users/developers help this past
> week.
>
> In that code I was calling MatLUFactorSymbolic() and
> MatLUFactorNumeric() to factor the sparse matrix and then calling
> MatSolve for each of the first k columns in the identity matrix as the
> RHS. I then varied the matrix type from the command line to test MUMPS,
> SUPERLU etc. for the best performance.
>
> Now I just want to translate the code into a parallel version...so I now
> assemble rows in a distributed fashion and now working on translating
> the MatLUFactorSymbolic() and MatLUFactorNumeric() calls which require
> index sets...hence my original question.
>
> Are you saying that I now shouldn't be calling those routines?

You can certainly do it that way, but it is much easier to just use a KSP.
You set the Mat using KSPSetOperators, then KSPSetType(ksp, KSPPREONLY),
and PCSetType(pc, PCLU) (or MUMPS or whatever). Then KSPSolve() with the
identity columns. We handle everything else.

  Matt

> Tim.
>
> Matthew Knepley wrote:
> > On Nov 18, 2007 11:52 AM, Tim Stitt <timothy.stitt at ichec.ie> wrote:
> >
> >> Matt,
> >>
> >> It is in setup for MatLUFactorSymbolic() and MatLUFactorNumeric() calls
> >> which require index sets. I have distributed my rows across the
> >> processes and now just a bit confused about the arguments to the
> >> ISCreateGeneral() routine to set up the IS sets used by the Factor
> >> routines in parallel.
> >>
> >> So my basic question is what in general is the length and integers that
> >> get passed to ISCreateGeneral() when doing this type of calculation in
> >> parallel? Are they local index values (0..#rows on process-1) or do they
> >> refer to the distributed indices of the global matrix?
> >>
> >
> > To be consistent, these would be local sizes and global numberings. However,
> > I am not sure why you would be doing this. I do not believe any of the parallel
> > LU packages accept an ordering from the user (they calculate their own),
> > and I would really only use them from a KSP (or PC at the least).
> >
> >   Matt
> >
> >
> >> Tim.
> >>
> >>
> >> Matthew Knepley wrote:
> >>
> >>> On Nov 18, 2007 11:34 AM, Tim Stitt <timothy.stitt at ichec.ie> wrote:
> >>>
> >>>
> >>>> OK..so I should be using the aggregate length returned by
> >>>> MatGetOwnershipRange() routine?
> >>>>
> >>>>
> >>> If you are using it to permute a Mat, yes.
> >>>
> >>>   Matt
> >>>
> >>>
> >>>
> >>>> Thanks Matt for you help.
> >>>>
> >>>>
> >>>> Matthew Knepley wrote:
> >>>>
> >>>>
> >>>>> IS are not really parallel, so all the lengths, etc. only refer to local things.
> >>>>>
> >>>>>   Matt
> >>>>>
> >>>>> On Nov 18, 2007 11:22 AM, Tim Stitt <timothy.stitt at ichec.ie> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>> Hi all,
> >>>>>>
> >>>>>> Just wanted to know if the "the length of the index set" for a call to
> >>>>>> ISCreateGeneral() in a parallel code, is a global length, or the length
> >>>>>> of the local elements on each process?
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Tim.
> >>>>>>
> >>>>>> --
> >>>>>> Dr. Timothy Stitt <timothy_dot_stitt_at_ichec.ie>
> >>>>>> HPC Application Consultant - ICHEC (www.ichec.ie)
> >>>>>>
> >>>>>> Dublin Institute for Advanced Studies
> >>>>>> 5 Merrion Square - Dublin 2 - Ireland
> >>>>>>
> >>>>>> +353-1-6621333 (tel) / +353-1-6621477 (fax)
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>> --
> >>>> Dr. Timothy Stitt <timothy_dot_stitt_at_ichec.ie>
> >>>> HPC Application Consultant - ICHEC (www.ichec.ie)
> >>>>
> >>>> Dublin Institute for Advanced Studies
> >>>> 5 Merrion Square - Dublin 2 - Ireland
> >>>>
> >>>> +353-1-6621333 (tel) / +353-1-6621477 (fax)
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >> --
> >> Dr. Timothy Stitt <timothy_dot_stitt_at_ichec.ie>
> >> HPC Application Consultant - ICHEC (www.ichec.ie)
> >>
> >> Dublin Institute for Advanced Studies
> >> 5 Merrion Square - Dublin 2 - Ireland
> >>
> >> +353-1-6621333 (tel) / +353-1-6621477 (fax)
> >>
> >>
> >>
> >
> >
> >
> >
>
>
> --
> Dr. Timothy Stitt <timothy_dot_stitt_at_ichec.ie>
> HPC Application Consultant - ICHEC (www.ichec.ie)
>
> Dublin Institute for Advanced Studies
> 5 Merrion Square - Dublin 2 - Ireland
>
> +353-1-6621333 (tel) / +353-1-6621477 (fax)
>
>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which
their experiments lead.
-- Norbert Wiener