<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=us-ascii" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Thank you for your explanation. I did this test and it really makes
sense now. In general it looks like petsc still requires quite a lot of
memory, but it really decreases this requirement as i increase the
number of processes.<br>
<br>
Cheers,<br>
Denis<br>
<br>
Barry Smith wrote:
<blockquote cite="mid:DFD4E53E-FAE6-409E-B451-3993EBFCF754@mcs.anl.gov"
type="cite">
<pre wrap=""> Sometimes computing can be an experimental science. Run the same
size DA on 1, 2, 4, 8, 16, 32 processes and gather the information
about memory usage and make a little table.
Here is what you should find. The amount of memory depends on the
local size of the array, which for your example below is
1000*1000*1000 on one process. Thus you will see that as you increase
the number of processes you'll see the space needed per process for
the DA decreases. It increases from 1 process to 2 because it needs
all the ghost point data and VecScatter that are not needed on 1.
Note also on one process a SINGLE vector for this size mesh is 8
gigabytes so the DA is really not much of pig since it is less than
one vector.
Barry
On Nov 22, 2009, at 9:34 PM, Denis Teplyashin wrote:
</pre>
<blockquote type="cite">
<pre wrap="">So this sort of memory consumption is expected? Is it possible to
reduce is somehow? I'm not sure about underlying petsc object but it
looks like these additional objects require more memory than the
actual vector itself.
Cheers,
Denis
Matthew Knepley wrote:
</pre>
<blockquote type="cite">
<pre wrap="">It is not simple, but it is scalable, meaning in the limit of large
N, the memory will be constant on
each processor. When it is created, the VecScatter objects mapping
global to local vecs are created.
Matt
On Sun, Nov 22, 2009 at 8:29 PM, Denis Teplyashin
<a class="moz-txt-link-rfc2396E" href="mailto:denist@al.com.au"><denist@al.com.au></a> wrote:
Hi guys,
I'm a bit confused with distributed array memory consumption. I did
a simple test like this one:
ierr = DACreate3d(PETSC_COMM_WORLD, DA_NONPERIODIC,
DA_STENCIL_BOX, 1000, 1000, 1000, PETSC_DECIDE, PETSC_DECIDE,
PETSC_DECIDE, 1, 1, PETSC_NULL, PETSC_NULL, PETSC_NULL , &da);
and then checked memory with PetscMemoryGetCurrentUsage and
PetscMemoryGetMaximumUsage. Running this test using mpi on one core
gives me this result: current usage 3818Mb and maximum usage
7633Mb. And this is the result after creating just a DA without
actual vectors. Running the same test on two cores gives me even
more interesting result: rank 0 - 9552/11463Mb and rank 1 -
5735/5732Mb.
Is it what i should expect in general or am i doing something
wrong? Is there a simple formula which could show how much memory i
would need to allocate and array with given resolution?
Thanks in advance,
Denis
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to
which their experiments lead.
-- Norbert Wiener
</pre>
</blockquote>
</blockquote>
<pre wrap=""><!---->
</pre>
</blockquote>
<br>
</body>
</html>