MatCreateMPIAIJ Pre-allocation Query

[Tim Stitt]

Matt,

I am getting a lot of warnings regarding incompatible pointer types when 
compiling with that option. Here is an example:

ziscoloringf90.c: In function `iscoloringgetisf90_':
ziscoloringf90.c:20: warning: passing arg 2 of `ISColoringGetIS' from 
incompatible pointer type

Is this a problem?

Tim.

On Sunday 17 June 2007 18:27, Tim Stitt wrote:
> Cheers...perfect.
>
> On Sunday 17 June 2007 18:16, Matthew Knepley wrote:
> > On 6/17/07, Tim Stitt <timothy.stitt at ichec.ie> wrote:
> > > Matt,
> > >
> > > We are investigating 32-qubit fault-tolerant quantum computation which
> > > results in 2^32 basis states and hence our large Hamiltonian matrix.
> > >
> > > I just ran into problems passing 2^32 in as the global row an column
> > > argument so I assumed I need to rebuild PETSc to accept 64-bit
> > > integers.
> >
> > Yes, reconfigure with --with-64-bit-indices and rebuild. You can do
> > this in another
> > arch if you want to keep your old build using --PETSC_ARCH=<new arch
> > name>
> >
> >    Matt
> >
> > > Tim.
> > >
> > > On Sunday 17 June 2007 17:54, Matthew Knepley wrote:
> > > > On 6/17/07, Tim Stitt <timothy.stitt at ichec.ie> wrote:
> > > > > Actually my sparse matrix has >2^32 rows and columns hence the
> > > > > global row and column arguments need to be 64-bit.
> > > >
> > > > Cool. What problem is that big?
> > > >
> > > >   Matt
> > > >
> > > > > On Sunday 17 June 2007 17:36, Barry Smith wrote:
> > > > > >   If your problem is so big that you need integers that represent
> > > > > > numbers more than about 2 billion then yes you will need to
> > > > > > rebuild PETSc.
> > > > > >
> > > > > >   Barry
> > > > > >
> > > > > > You'll be the first PETSc user who solves a problem with more
> > > > > > than 2 billion unknowns.
> > > > > >
> > > > > > On Sun, 17 Jun 2007, Tim Stitt wrote:
> > > > > > > Thanks for that Matt...will check it out.
> > > > > > >
> > > > > > > Incidentally, I need to pass a 64-bit integer to
> > > > > > > MatCreateMPIAIJ. Do I need to rebuild PETSc using a 64-bit
> > > > > > > integer option switch?
> > > > > > >
> > > > > > > Tim.
> > > > > > >
> > > > > > > On Sunday 17 June 2007 17:07, Matthew Knepley wrote:
> > > > > > > > On 6/17/07, Tim Stitt <timothy.stitt at ichec.ie> wrote:
> > > > > > > > > Hi All,
> > > > > > > > >
> > > > > > > > > Currently I am using MatCreateMPIAIJ to create a
> > > > > > > > > distributed sparse matrix for use in my parallel sparse
> > > > > > > > > eigensolver. If I understand things correctly, it is
> > > > > > > > > important to specify the correct pre-allocation values for
> > > > > > > > > (o_nz,d_nz) or (o_nnz, d_nnz) respectively to obtain
> > > > > > > > > optimum memory usage per node.
> > > > > > > > >
> > > > > > > > > For information my matrix redistribution routine generates
> > > > > > > > > each row in the sparse matrix (associated with a given
> > > > > > > > > node) "on-the-fly" and then uses MatSetValues to apply the
> > > > > > > > > elements.
> > > > > > > > >
> > > > > > > > > My question is: does anyone have a standard code template
> > > > > > > > > for the calculating the d_nz or d_nnz values (along with
> > > > > > > > > o_nz or o_nnz respectively) so that optimum pre-allocation
> > > > > > > > > is obtained? Is there a simple way of determining if a
> > > > > > > > > given non-zero value in a row is part of the diagonal or
> > > > > > > > > off-diagonal regions for a given distributed topology?
> > > > > > > > >
> > > > > > > > > Any assistance gratefully received.
> > > > > > > >
> > > > > > > > Basically, you run through the construction algorithm once to
> > > > > > > > calculate d_nnz and o_nnz, and then again to set values. You
> > > > > > > > can see my code for doing it in preallocateOperator() in
> > > > > > > > src/dm/mesh/mesh.c.
> > > > > > > >
> > > > > > > >    Matt
> > > > > > > >
> > > > > > > > > Thanks in advance,
> > > > > > > > >
> > > > > > > > > Tim.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Dr. Timothy Stitt <timothy_dot_stitt_at_ichec.ie>
> > > > > > > > > HPC Application Consultant - ICHEC (www.ichec.ie)
> > > > > > > > >
> > > > > > > > > Dublin Institute for Advanced Studies
> > > > > > > > > 5 Merrion Square - Dublin 2 - Ireland
> > > > > > > > >
> > > > > > > > > +353-1-6621333 (tel) / +353-1-6621477 (fax) /
> > > > > > > > > +353-874195427 (mobile)
> > > > >
> > > > > --
> > > > > Dr. Timothy Stitt <timothy_dot_stitt_at_ichec.ie>
> > > > > HPC Application Consultant - ICHEC (www.ichec.ie)
> > > > >
> > > > > Dublin Institute for Advanced Studies
> > > > > 5 Merrion Square - Dublin 2 - Ireland
> > > > >
> > > > > +353-1-6621333 (tel) / +353-1-6621477 (fax) / +353-874195427
> > > > > (mobile)
> > >
> > > --
> > > Dr. Timothy Stitt <timothy_dot_stitt_at_ichec.ie>
> > > HPC Application Consultant - ICHEC (www.ichec.ie)
> > >
> > > Dublin Institute for Advanced Studies
> > > 5 Merrion Square - Dublin 2 - Ireland
> > >
> > > +353-1-6621333 (tel) / +353-1-6621477 (fax) / +353-874195427 (mobile)

-- 
Dr. Timothy Stitt <timothy_dot_stitt_at_ichec.ie>
HPC Application Consultant - ICHEC (www.ichec.ie)

Dublin Institute for Advanced Studies
5 Merrion Square - Dublin 2 - Ireland

+353-1-6621333 (tel) / +353-1-6621477 (fax) / +353-874195427 (mobile)