matrix memory allocation

[Matthew Knepley]

On Wed, Mar 26, 2008 at 6:30 PM, Matt Funk <mafunk at nmsu.edu> wrote:
> Well,
>
>  it is of course possible that i am looking at this wrong.
>
>  as to your question what it would save:
>
>  - case 1) : allocating all local rows on the local process at one time:
>  in order to determine the information for the nonzero entries for the local on
>  and off diagonal matrices i have to iterate over my entire domain and do
>  logic for each node to determine whether my surrounding nodes (for the
>  Laplacian) are on the local processor or not and/or whether they are even in
>  the domain. Because i have to do the same thing again once i get to the point
>  where i insert values into the matrix this is really redundant, but necessary
>  to determine the space to allocate.
>  i would like to eliminate this.

This is what we do. If you measure it, you will find it is a
completely negligible
fraction of the runtime. I see this as premature (and unnecessary) optimization.
If I am wrong here, please send the numbers.

  Thanks,

     Matt

>  -case 2) : do not preallocate anything
>   very slow, and not really an option.
>
>  so, what it would gain me would be the best compromise between memory
>  allocation performance and logic overhead. I.e. the logic while iterating
>  through the domain layout it executed only once instead of twice.
>
>  I hope it makes sense...
>
>  but i guess it is not possible anyway. too bad ...
>
>  thanks though
>  mat
>
>
>
>
>
>
>  On Wednesday 26 March 2008 17:07, Matthew Knepley wrote:
>  > This is not possible with the current implementation. I am not sure why you
>  > would want to do it. It seems to me to save nothing. Do you have some sort
>  > of performance problem?
>  >
>  >   Matt
>  >
>  > On Wed, Mar 26, 2008 at 5:01 PM, Matt Funk <mafunk at nmsu.edu> wrote:
>  > > Maybe i should clarify a little more.
>  > >  I guess what i am looking for is something that lets me split up the
>  > > memory allocation within a given processor.
>  > >
>  > >  The example you guys often show (for example on MatCreateMPIAIJ manual
>  > > page) is where the local process owns 3 rows.
>  > >
>  > >  But consider for example if the local process owns 1000 rows. Right now
>  > > (as i can see it) i have two options. Do not allocate memory at all and
>  > > use MatSetValues and allocate memory per row. This is very slow as you
>  > > guys state on your manual pages.
>  > >  The other option is to allocate all 1000 rows at a time which means
>  > > extra overhead.
>  > >
>  > >  However, what i would like to be able to (as an example), make 2 calls
>  > > for an allocation of 200 rows and a second call for an allocation of the
>  > > remaining 800 rows.
>  > >
>  > >  I do not know if this is possible as i don't know whether all the local
>  > > stuff is stored in on continuous array?
>  > >
>  > >  Anyway, i hope this clarifies my question a little more.
>  > >
>  > >  thank
>  > >  mat
>  > >
>  > >  On Wednesday 26 March 2008 14:31, Matt Funk wrote:
>  > >  > Hi,
>  > >  >
>  > >  > in order to create a sparse MPIMatrix (with preallocated memory) it is
>  > >  > necessary to give it the number of nonzero entries per row for the
>  > >  > local and off-diagonal submatrix.
>  > >  >
>  > >  > The way i do things right now is that i need to allocate the entire
>  > >  > matrix. However, my domain is decomposed into boxes where one or more
>  > >  > boxes reside on the current processor.
>  > >  >
>  > >  > So, i was wondering if it is possible to allocate the matrix memory
>  > >  > per box. Right now what i do is this:
>  > >  > 1) iterate over the whole domain (every single box) and extract the
>  > >  > info i need from each box for memory allocation.
>  > >  > 2) call MatCreateMPIAIJ to create the matrix/allocate memory.
>  > >  > 3) iterate over the whole domain again to get the info to insert the
>  > >  > values into the matrix.
>  > >  >
>  > >  > Is it possible to instead do something like:
>  > >  > 1) get info for values and memory allocation needed per box
>  > >  > 2) allocate memory for x number of rows in global matrix and insert
>  > >  > values for them (via MatSetValues)
>  > >  > 3) start at 1) till every box in the domain is covered.
>  > >  >
>  > >  > The problem in 2) is that i can insert the values, but there is no
>  > >  > memory preallocated for it, so it would be slow. If i could allocate
>  > >  > it, i think it would solve my problem.
>  > >  >
>  > >  > I hope i am making sense. By the way, i cannot use the DA construct
>  > >  > because i have overlapping values.
>  > >  >
>  > >  > thanks
>  > >  > mat
>
>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which
their experiments lead.
-- Norbert Wiener